blib/lib/Cn/Util.pm

filter: Statementstmt 92.2% 380 / 412 Branchbran 74.1% 172 / 232 Conditioncond 52.4% 53 / 101 MC/DCmcdc 38.6% 22 / 57 Subroutinesub 100% 37 / 37 Podpod 100% 10 / 10 totaltotal 79.3% 674 / 849 CC 169 SCAR 29.2
CC
169
Cov
81%
_structure_kind
69.1
_structure_describe
27.7
structure
26.5
CRAP 18.6
↑ Summary
1 # ABSTRACT: Small everyday helpers: directory listing, shell commands, JSON files, number formatting, and data-structure shapes
2 95 ▶use strict;
Subroutine BEGIN: called CC=1 SCAR=0.0
3 95 ▶use feature 'say';
Subroutine BEGIN: called CC=1 SCAR=0.0
4 95 ▶use warnings FATAL => 'all';
Subroutine BEGIN: called CC=1 SCAR=0.0
5 # Devel::Confess is loaded for its side effect, which is process-wide: it
6 # installs $SIG{__DIE__} and $SIG{__WARN__} so that every die and warn in the
7 # program carries a stack trace. `use Cn::Util` has always given the caller
8 # this, so it stays at the top, where loading the module is enough.
9 95 ▶use Devel::Confess;
Subroutine BEGIN: called CC=1 SCAR=0.0
10  
11 package Cn::Util;
12 95 require 5.010001;
13 95 our $VERSION = 0.024;
14 95 require Exporter;
15 95 our @ISA = ('Exporter');
16 95 our @EXPORT = qw(dir execute file2string format_commas json_file_to_ref random_key random_value ref_to_json_file st structure);
17  
18 # Startup cost. Everything other than the pragmas, Exporter and
19 # Devel::Confess is loaded on first use, not at `use Cn::Util`. Loading DDP,
20 # autodie, Capture::Tiny and JSON::MaybeXS eagerly cost ~40 ms of interpreter
21 # startup on every script that touched this module, even ones that only called
22 # format_commas().
23 #
24 # The rules that decide what can be deferred:
25 # - PRAGMAS (warnings, feature, autodie) act on the *compiler* for the
26 # enclosing lexical scope, so they can never be deferred. autodie was the
27 # single most expensive line here (~15 ms); this file does not use it, and
28 # the bare open()s it was covering carry explicit `or die`.
29 # - Exporter must exist at compile time (it supplies import()), but it is
30 # free -- it is already in memory as part of the core.
31 # - Everything else is a plain module, so `require` inside the sub that
32 # needs it works. `require` is idempotent and a repeat call is one %INC
33 # hash lookup, so putting it in the hot sub costs nothing measurable.
34 # - A runtime `require` does NOT run import(), so the imported short names
35 # (find, capture, decode_json, colored, blessed) are not available. Each
36 # call site below is therefore fully qualified. Calling a not-yet-loaded
37 # sub with a block prototype is also unparseable at compile time, which is
38 # why Capture::Tiny::capture is handed an explicit sub{} instead of a bare
39 # {}.
40  
41 # ---------------------------------------------------------------------------
42 # `use Cn::Util` hands the calling package three things besides @EXPORT:
43 # autodie, and DDP's p() and np().
44 #
45 # autodie is LEXICAL, and that is the hard case. It is Fatal underneath, and
46 # Fatal::import does a bare `caller()` with no $Level support: calling
47 # autodie->import() directly from in here makes Fatal believe the importing
48 # file is Cn/Util.pm. Its runtime leak-guard then sees calls arriving from a
49 # different file and correctly declines to apply -- open() in the caller just
50 # returns undef again, silently. Verified: the naive version does nothing at
51 # all.
52 #
53 # So the import has to be performed in a context where caller() reports the
54 # caller's package and file. A string eval carrying a #line directive does
55 # exactly that, and is the same trick Import::Into uses (not depended on here
56 # because it is not a core module). $line/$file come from our own caller, so
57 # autodie's scope starts at the `use Cn::Util` line, which is where a
58 # hand-written `use autodie` would have started it too.
59 #
60 # The eval only compiles a closure, and the import runs when that closure is
61 # called after the eval has returned, which is also what Import::Into does.
62 # Perls up to at least 5.12 keep a string eval's own %^H in force while the
63 # eval runs, and autodie 2.06 (core in 5.10.1 and 5.12.5) keeps the guard that
64 # un-installs it at the end of the scope in %^H: imported inside the eval,
65 # autodie was un-installed again as the eval returned, and a missing file
66 # opened in the caller returned false. t/00-load.t checks this.
67 #
68 # p() and np() are installed by _install_ddp_stubs(), below.
69 #
70 # NB: `use Cn::Util ();` skips import() entirely, per normal Perl semantics,
71 # so that is the escape hatch for a caller that wants none of this.
72 # ---------------------------------------------------------------------------
73 95 ▶sub import {
Subroutine import: called CC=2 SCAR=7.0
74 95 my $class = shift;
75 95 $class->export_to_level(1, $class, @_); # the @EXPORT subs, one level up
76  
77 95 my ($pkg, $file, $line) = caller;
78 95 _install_ddp_stubs($pkg);
79 95 require autodie;
80 95 ▶ my $autodie = eval qq{package $pkg;\n#line $line "$file"\nsub { autodie->import(':default') }}
Branch: 1 branch
BranchTrueFalseTotal
my $autodie = eval qq[package $pkg;\n#line $line "$file"\nsub { autodie->import(':default') }] or die "Cn::Util: could not propagate autodie to ${file}: $@" 95 0 95
81 or die "Cn::Util: could not propagate autodie to $file: $@";
82 95 $autodie->();
83 95 return;
84 }
85  
86 # Scripts rely on `use Cn::Util` giving them DDP's p() and np() without a
87 # `use DDP` of their own. These stubs keep that convenience without paying
88 # DDP's ~16 ms unless something actually prints. They are installed at the
89 # caller's `use` line, which is compile time for the rest of the caller, and
90 # with DDP's own prototype, because that is what makes `p @files` pass \@files
91 # rather than a flat list. On first call, DDP->import() is run as if from the
92 # caller's package, so it overwrites the stub there with the real thing.
93 #
94 # A p() or np() the caller already has -- its own, or DDP's from an earlier
95 # `use DDP` -- is left alone. Util.pm, which this module was before it was
96 # Cn::Util, compiled the stubs into main:: whenever it loaded, which redefined
97 # DDP's p() under its FATAL warnings when a script said `use DDP` first.
98 95 my %DDP_LOADED; # package => true once DDP's real p() and np() are installed there
99 my %DDP_STUB; # package => { name => the stub installed for it }, p and np
100  
101 95 ▶sub _install_ddp_stubs {
Subroutine _install_ddp_stubs: called CC=3 SCAR=11.0
102 95 my ($pkg) = @_;
103 95 ▶ no strict 'refs';
Subroutine BEGIN: called CC=1 SCAR=0.0
104 95 for my $name ('p', 'np') {
105 190 my $glob = "${pkg}::$name";
106 190 ▶ next if defined &{$glob};
Branch: 1 branch
BranchTrueFalseTotal
if defined &{$glob;} 1 189 190
107 189 ▶ *{$glob} = $DDP_STUB{$pkg}{$name} = sub (\[@$%&];%) {
Subroutine __ANON__: called CC=2 SCAR=7.0
108 4 _load_ddp($pkg);
109 # A `p` that DDP did not replace -- an `alias` in ~/.dataprinter
110 # exports it under another name -- would otherwise be this stub
111 # again, and the goto would call it forever.
112 die "Cn::Util: DDP did not install ${pkg}::$name\n"
113 2 ▶ if \&{$glob} == $DDP_STUB{$pkg}{$name};
Branch: 1 branch
BranchTrueFalseTotal
if \&{$glob;} == $DDP_STUB{$pkg}{$name} 0 2 2
114 2 goto &{$glob};
115 189 };
116 }
117 95 return;
118 }
119  
120 4 ▶sub _load_ddp {
Subroutine _load_ddp: called CC=7 SCAR=19.8
121 4 my ($pkg) = @_;
122 # Marked loaded only once the import has succeeded. Marking it first made
123 # a second call after a failed load return here at once, and the stub's
124 # goto then called the stub itself, forever.
125 4 ▶ return if $DDP_LOADED{$pkg};
Branch: 1 branch
BranchTrueFalseTotal
if $DDP_LOADED{$pkg} 0 4 4
126 4 require DDP;
127 95 ▶ no strict 'refs';
Subroutine BEGIN: called CC=1 SCAR=0.0
128 # DDP->import() exports both p() and np(), but a stub was installed only
129 # for a name the caller had no sub under, so the other -- or one the caller
130 # defined over a stub after its `use` line -- is the caller's own. Its
131 # glob is localised for the import, so DDP writes into a temporary that is
132 # discarded, and the caller's sub comes back untouched.
133 my %own = map {
134 2 my $glob = "${pkg}::$_";
135 4 ▶ $_ => (defined &{$glob} && \&{$glob} != ($DDP_STUB{$pkg}{$_} // 0));
Condition80%4/5
defined &{$glob;} && \&{$glob;} != ($DDP_STUB{$pkg}{$_} // 0)
!ll&&!rl&&r
031
$DDP_STUB{$pkg}{$_} // 0
l!l
31
MC/DC50%1/2
defined &{$glob;} && \&{$glob;} != ($DDP_STUB{$pkg}{$_} // 0)
defined &{$glob;} \&{$glob;} != ($DDP_STUB{$pkg}{$_} // 0)
MC/DC100%1/1
$DDP_STUB{$pkg}{$_} // 0
$DDP_STUB{$pkg}{$_}
Truth table66%2/3
A && B
A= defined &{$glob;}B= \&{$glob;} != ($DDP_STUB{$pkg}{$_} // 0)
ABresult
0X0
100
111
Truth table100%2/2
A
A= $DDP_STUB{$pkg}{$_}
Aresult
11
00
136 } 'p', 'np';
137 2 ▶ local *{"${pkg}::p"} if $own{p};
Branch: 1 branch
BranchTrueFalseTotal
if $own{"p"} 1 1 2
138 0 ▶ local *{"${pkg}::np"} if $own{np};
Branch: 1 branch
BranchTrueFalseTotal
if $own{"np"} 0 2 2
139 3 ▶ local $SIG{__WARN__} = sub {
Subroutine __ANON__: called CC=2 SCAR=7.6
140 3 ▶ warn @_ unless $_[0] =~ m{\ASubroutine \Q$pkg\E::n?p redefined};
Branch: 1 branch
BranchTrueFalseTotal
unless $_[0] =~ /\ASubroutine \Q$pkg\E::n?p redefined/ 0 3 3
141 2 };
142 2 my $ok = eval qq{package $pkg;\nDDP->import({output => 'STDOUT', array_max => 10, show_memsize => 1, show_dualvar => 'off'}); 1};
143 2 ▶ die "Cn::Util: could not load DDP into $pkg: $@" unless $ok;
Branch: 1 branch
BranchTrueFalseTotal
unless $ok 0 2 2
144 2 $DDP_LOADED{$pkg} = 1;
145 2 return;
146 }
147  
148 1599 ▶sub dir {
Subroutine dir: called CC=26 SCAR=32.7
Pod documented
149 1599 my %args = (
150 tests => 'f',
151 dir => '.',
152 regex => '.',
153 recursive => 0,
154 on_match => undef, # optional callback: streams results, keeps RAM flat
155 cpu => 1, # worker processes for a recursive walk; 1 = no forking
156 @_,
157 );
158  
159 1599 my %allowed = map { $_ => 1 } qw(tests dir regex recursive on_match cpu);
160 1599 ▶ if (my @bad = grep { !$allowed{$_} } keys %args) {
Branch: 1 branch
BranchTrueFalseTotal
if (my(@bad) = grep({not $allowed{$_};} keys %args)) 1 1598 1599
161 1 die "Unrecognized arguments: @bad\nAccepted args: "
162 . join(', ', sort keys %allowed);
163 }
164 # Added 's' (non-zero size), removed 't' (tty test - meaningless on a path)
165 # Checked here so that an undef gets this message rather than "Use of
166 # uninitialized value" from the first line that reads it. recursive is
167 # left out: undef is as false there as 0.
168 1598 for my $name (qw(tests dir regex cpu)) {
169 6389 ▶ die "$name must be defined" unless defined $args{$name};
Branch: 1 branch
BranchTrueFalseTotal
unless defined $args{$name} 1 6388 6389
170 }
171 die "on_match must be a code reference"
172 1597 ▶ if defined $args{on_match} && ref $args{on_match} ne 'CODE';
Branch: 1 branch
BranchTrueFalseTotal
if defined $args{"on_match"} and ref $args{"on_match"} ne "CODE" 1 1596 1597
Condition100%3/3
defined $args{"on_match"} and ref $args{"on_match"} ne "CODE"
!ll&&!rl&&r
14901061
MC/DC100%2/2
defined $args{"on_match"} and ref $args{"on_match"} ne "CODE"
defined $args{"on_match"} ref $args{"on_match"} ne "CODE"
Truth table100%3/3
A and B
A= defined $args{"on_match"}B= ref $args{"on_match"} ne "CODE"
ABresult
0X0
100
111
173 die "Unrecognized characters in tests: $args{tests}"
174 1596 ▶ if $args{tests} =~ /[^rwxoRWXOezsfdlpSbcugkTB]/;
Branch: 1 branch
BranchTrueFalseTotal
if $args{"tests"} =~ /[^rwxoRWXOezsfdlpSbcugkTB]/ 1 1595 1596
175 die "cpu must be a positive integer, got '$args{cpu}'"
176 1595 ▶ unless $args{cpu} =~ /\A[1-9][0-9]*\z/;
Branch: 1 branch
BranchTrueFalseTotal
unless $args{"cpu"} =~ /\A[1-9][0-9]*\z/ 2 1593 1595
177 1593 ▶ my $re = eval { qr/$args{regex}/ }
Branch: 1 branch
BranchTrueFalseTotal
my $re = eval { do { qr/$args{"regex"}/ } } or die qq[Invalid regex '$args{"regex"}': $@] 1592 1 1593
178 or die "Invalid regex '$args{regex}': $@";
179 # Compile the tests once. The first test stats the path; every later test
180 # reuses the stat buffer via the special '_' filehandle, so N tests cost
181 # one stat(2) syscall instead of N (~2.4x faster for 4 tests).
182 # -l needs lstat, so it is hoisted to the front, and the test after it
183 # must re-stat the real path (otherwise '_' holds lstat data, or -l _
184 # would be a fatal error).
185 1592 my @t = split //, $args{tests};
186 1592 @t = ((grep { $_ eq 'l' } @t), (grep { $_ ne 'l' } @t));
187 1592 my @parts;
188 1592 my $fresh = 1; # 1 = the next test must stat the path itself rather than reuse '_'
189 1592 for my $t (@t) {
190 0 ▶ if ($t eq 'l') { push @parts, '-l $_[0]'; $fresh = 1 }
Branch: 2 branches
BranchTrueFalseTotal
if ($t eq 'l') { } 0 1348 1348
elsif ($fresh) { } 1348 0 1348
191 1348 elsif ($fresh) { push @parts, "-$t \$_[0]"; $fresh = 0 }
192 0 else { push @parts, "-$t _" }
193 }
194 my $tester = @parts
195 ? (eval 'sub { ' . join(' && ', @parts) . ' }'
196 or die "Failed to compile tests: $@")
197 1592 ▶ : sub { 1 }; # empty tests string now means "no filtering", not "nothing matches"
Subroutine __ANON__: called CC=1 SCAR=0.0
Branch: 1 branch
BranchTrueFalseTotal
@parts ? : 1348 244 1592
Condition50%1/2
eval 'sub { ' . join(" && ", @parts) . ' }' or die "Failed to compile tests: $@"
l!l
13480
MC/DC0%0/1
eval 'sub { ' . join(" && ", @parts) . ' }' or die "Failed to compile tests: $@"
eval 'sub { ' . join(" && ", @parts) . ' }'
Truth table50%1/2
A
A= eval 'sub { ' . join(" && ", @parts) . ' }'
Aresult
11
00
198  
199 1592 my $target_dir = $args{dir};
200 # On Windows, File::Find turns every "\" in its starting path into "/"
201 # (the "canonicalize directory separators" step of _find_opt in 1.44, the
202 # version on the Strawberry Perl 5.42 smoker that failed t/dir.t; 1.14, in
203 # perl 5.10.1, does not), so a recursive listing of "C:\x" reported paths under
204 # "C:/x", while the non-recursive listing and the parallel walk's top
205 # levels joined onto "C:\x"; and since "C:/x" ne "C:\x", the walk's check
206 # that leaves the root itself out never matched. Doing the same here first
207 # gives every mode the same paths. Only on MSWin32, as in File::Find:
208 # elsewhere "\" is an ordinary character in a file name.
209 1592 my $is_win32 = $^O eq 'MSWin32';
210 1592 ▶ $target_dir =~ tr{\\}{/} if $is_win32;
Branch: 1 branch
BranchTrueFalseTotal
if $is_win32 0 1592 1592
211 # Strip trailing slashes, but keep a root intact: "/", and on Windows "C:/",
212 # because "C:" alone is the current directory on drive C, not its root.
213 # Collapsing first is what turns "///" into "/"; it used to be left as it
214 # was, to be joined as "////etc".
215 1592 $target_dir =~ s{/+\z}{/}s;
216 1592 ▶ $target_dir =~ s{/\z}{}s
Branch: 2 branches
BranchTrueFalseTotal
unless $target_dir =~ ($is_win32 ? qr"\A(?:[A-Za-z]:)?/\z"s : qr"\A/\z"s) 1422 170 1592
$is_win32 ? : 0 1592 1592
217 unless $target_dir =~ ($is_win32 ? qr{\A(?:[A-Za-z]:)?/\z}s : qr{\A/\z}s);
218 # "." is left off the front of results; a root, the only $d that still
219 # ends in "/", must not become "//etc" or "C://etc".
220 12161 ▶ my $join = sub {
Subroutine __ANON__: called CC=3 SCAR=11.0
221 12161 my ($d, $item) = @_;
222 12161 ▶ $d eq '.' ? $item : $d =~ m{/\z} ? "$d$item" : "$d/$item"
Branch: 2 branches
BranchTrueFalseTotal
$d eq '.' ? : 117 12044 12161
$d =~ m[/\z] ? : 3570 8474 12044
223 1592 };
224  
225 1592 my $cb = $args{on_match};
226 1592 my @items;
227 1592 ▶ my $emit = $cb // sub { push @items, $_[0] };
Subroutine __ANON__: called CC=1 SCAR=0.1
Condition66%2/3
$cb // sub { push @items, $_[0]; }
l!l&&r!l&&!r
10614860
MC/DC0%0/2
$cb // sub { push @items, $_[0]; }
$cb sub { push @items, $_[0]; }
Truth table66%2/3
A // B
A= $cbB= sub { push @items, $_[0]; }
ABresult
1X1
011
000
228  
229 1592 ▶ if ($args{recursive}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($args{'recursive'}) { } 967 625 1592
230 967 require File::Find; # ~1.6 ms, and only the recursive branch needs it
231 # Walk everything strictly below $start, handing each match to $sink.
232 601 ▶ my $find = sub {
Subroutine __ANON__: called CC=4 SCAR=13.9
233 601 my ($start, $sink) = @_;
234 601 my $strip = $start eq '.';
235 # File::Find lstat()s its starting path, so a $start that is a
236 # symlink to a directory was taken for a file and nothing was
237 # found under it -- although opendir, which the non-recursive
238 # listing and the parallel walk's top levels use, follows it. So
239 # that level is listed here with opendir, and File::Find starts
240 # from its subdirectories instead. Passing "$start/" made POSIX
241 # lstat resolve the link, but not win32_lstat (perl 5.42
242 # win32/win32.c), which asks GetFileAttributes, and Windows drops a
243 # trailing separator; Strawberry Perl 5.42 still found nothing.
244 601 ▶ my @top = -l $start && -d $start
Branch: 1 branch
BranchTrueFalseTotal
-l $start && -d $start ? : 50 551 601
Condition66%2/3
-l $start and -d $start
!ll&&!rl&&r
551050
MC/DC50%1/2
-l $start and -d $start
-l $start -d $start
Truth table66%2/3
A and B
A= -l $startB= -d $start
ABresult
0X0
100
111
245 ? _dir_level($start, $re, $tester, $join, $sink)
246 : ($start);
247 601 ▶ return unless @top;
Branch: 1 branch
BranchTrueFalseTotal
unless @top 0 601 601
248 601 my %top = map { $_ => 1 } @top;
249 File::Find::find({
250 no_chdir => 1,
251 25732 ▶ wanted => sub {
Subroutine __ANON__: called CC=5 SCAR=16.1
252 # A starting point is not "below" $start, or, under a
253 # symlinked $start, _dir_level has already reported it.
254 25732 ▶ return if $top{$_};
Branch: 1 branch
BranchTrueFalseTotal
if $top{$_} 1151 24581 25732
255 24581 my ($file) = m{([^/]+)\z}s; # basename, much cheaper than File::Spec
256 24581 ▶ if ($file =~ $re && $tester->($_)) {
Branch: 1 branch
BranchTrueFalseTotal
if ($file =~ /$re/ and &$tester($_)) 14178 10403 24581
Condition100%3/3
$file =~ /$re/ and &$tester($_)
!ll&&!rl&&r
2941746214178
MC/DC100%2/2
$file =~ /$re/ and &$tester($_)
$file =~ /$re/ &$tester($_)
Truth table100%3/3
A and B
A= $file =~ /$re/B= &$tester($_)
ABresult
0X0
100
111
257 14178 my $path = $_;
258 14178 ▶ $path =~ s{^\./}{} if $strip;
Branch: 1 branch
BranchTrueFalseTotal
if $strip 324 13854 14178
259 14178 $sink->($path);
260 }
261 },
262 601 }, @top);
263 967 };
264 967 ▶ if ($args{cpu} == 1) {
Branch: 1 branch
BranchTrueFalseTotal
if ($args{'cpu'} == 1) { } 481 486 967
265 481 $find->($target_dir, $emit);
266 } else {
267 486 _dir_parallel($target_dir, $args{cpu}, $re, $tester, $join, $find, $emit);
268 }
269 } else {
270 # cpu is ignored here on purpose: one readdir plus one stat per entry
271 # is ~6 us warm, far below the cost of forking and shipping results back.
272 625 ▶ opendir my $dh, $target_dir
Branch: 1 branch
BranchTrueFalseTotal
opendir my $dh, $target_dir or die "Cannot open directory ${target_dir}: $!" 624 1 625
273 or die "Cannot open directory $target_dir: $!";
274 624 ▶ while (defined(my $item = readdir $dh)) { # defined(): a file named "0" is falsy
Branch: 1 branch
BranchTrueFalseTotal
if (defined(my $item = readdir $dh)) 7320 624 7944
275 7320 ▶ next if $item =~ /\A\.\.?\z/; # \A..\z: "$" would match before a trailing \n
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /\A\.\.?\z/ 1248 6072 7320
276 6072 ▶ next if $item !~ $re;
Branch: 1 branch
BranchTrueFalseTotal
unless $item =~ /$re/ 879 5193 6072
277 5193 my $path = $join->($target_dir, $item);
278 5193 ▶ $emit->($path) if $tester->($path);
Branch: 1 branch
BranchTrueFalseTotal
if &$tester($path) 4598 595 5193
279 }
280 624 closedir $dh;
281 }
282 1507 ▶ return $cb ? () : @items;
Branch: 1 branch
BranchTrueFalseTotal
$cb ? : 94 1413 1507
283 }
284  
285 # Jobs handed out per worker. Parallel::ForkManager starts a new job whenever
286 # a worker finishes, so more jobs than workers is what evens out a tree whose
287 # subdirectories differ wildly in size; each job costs one fork plus one
288 # Storable round-trip through a temp file, so too many jobs loses again.
289 # Measured 2026-09-23, 20-core NVMe box, warm cache, best of 3, seconds:
290 #
291 # tree (files) serial k=1 k=2 k=4 k=8 k=16
292 # ~/perl5 (150k), cpu=8 1.23 0.95 0.25 0.29 0.36 0.47
293 # ~/perl5 (150k), cpu=4 1.23 1.03 0.82 0.39 0.45 0.61
294 # perlbrew/build (120k), cpu=8 0.75 0.19 0.24 0.25 0.33 0.44
295 #
296 # k=1 and k=2 collapse on the skewed tree; 4 is the smallest value that does
297 # not. Re-run: edit the value below, then time
298 # perl -Ilib -MCn::Util -e 'dir(dir => $ARGV[0], recursive => 1, cpu => 8)' TREE
299 95 my $DIR_JOBS_PER_CPU = 4;
300  
301 # Hand each entry of directory $d that matches $re and $tester to $emit, and
302 # return its subdirectories, leaving out symlinks to directories as File::Find
303 # does. One level of a recursive dir(), for the parallel walk's breadth-first
304 # levels and for a starting directory that is itself a symlink.
305 536 ▶sub _dir_level {
Subroutine _dir_level: called CC=8 SCAR=20.8
306 536 my ($d, $re, $tester, $join, $emit) = @_;
307 536 ▶ opendir my $dh, $d or die "Cannot open directory $d: $!";
Branch: 1 branch
BranchTrueFalseTotal
opendir my $dh, $d or die "Cannot open directory ${d}: $!" 536 0 536
308 536 my @sub;
309 536 ▶ while (defined(my $item = readdir $dh)) {
Branch: 1 branch
BranchTrueFalseTotal
if (defined(my $item = readdir $dh)) 8040 536 8576
310 8040 ▶ next if $item =~ /\A\.\.?\z/;
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /\A\.\.?\z/ 1072 6968 8040
311 6968 my $path = $join->($d, $item);
312 6968 ▶ $emit->($path) if $item =~ $re && $tester->($path);
Branch: 1 branch
BranchTrueFalseTotal
if $item =~ /$re/ and &$tester($path) 2860 4108 6968
Condition100%3/3
$item =~ /$re/ and &$tester($path)
!ll&&!rl&&r
158625222860
MC/DC100%2/2
$item =~ /$re/ and &$tester($path)
$item =~ /$re/ &$tester($path)
Truth table100%3/3
A and B
A= $item =~ /$re/B= &$tester($path)
ABresult
0X0
100
111
313 6968 ▶ push @sub, $path if !-l $path && -d _; # -l: lstat, so -d _ is the entry itself, not its target
Branch: 1 branch
BranchTrueFalseTotal
if not -l $path and -d _ 6432 536 6968
Condition66%2/3
not -l $path and -d _
!ll&&!rl&&r
53606432
MC/DC50%1/2
not -l $path and -d _
not -l $path -d _
Truth table66%2/3
not(A) and B
A= not -l $pathB= -d _
ABresult
0X0
100
111
314 }
315 536 closedir $dh;
316 536 return @sub;
317 }
318  
319 # Recursive dir() across $cpu worker processes. Results come back to this
320 # process and go through $emit here, so an on_match closure still sees and
321 # mutates the caller's variables -- it just runs after a whole job's batch
322 # arrives rather than per file, and match order differs from the serial walk.
323 486 ▶sub _dir_parallel {
Subroutine _dir_parallel: called CC=10 SCAR=24.0
324 486 my ($root, $cpu, $re, $tester, $join, $find, $emit) = @_;
325 486 require Parallel::ForkManager; # not core, and only cpu > 1 needs it
326  
327 # One subtree per job would starve the pool whenever a single directory
328 # holds most of the tree (~/perl5: one of 3 top-level dirs has 99% of the
329 # files, and a top-level split gave no speedup at any cpu). So walk
330 # breadth-first here until a level is wide enough to share out, testing
331 # the entries of every level walked; the final level's directories are
332 # already tested and become the jobs.
333 486 my $want = $cpu * $DIR_JOBS_PER_CPU;
334 486 my @level = ($root);
335 486 ▶ while (@level && @level < $want) {
Branch: 1 branch
BranchTrueFalseTotal
if (@level and @level < $want) 486 486 972
Condition66%2/3
@level and @level < $want
!ll&&!rl&&r
0486486
MC/DC50%1/2
@level and @level < $want
@level @level < $want
Truth table66%2/3
A and B
A= @levelB= @level < $want
ABresult
0X0
100
111
336 486 @level = map { _dir_level($_, $re, $tester, $join, $emit) } @level;
337 }
338 486 ▶ return unless @level;
Branch: 1 branch
BranchTrueFalseTotal
unless @level 0 486 486
339  
340 486 my @jobs;
341 486 push @{ $jobs[$_ % $want] }, $level[$_] for 0 .. $#level;
342  
343 486 my @errors;
344 486 my $pm = Parallel::ForkManager->new($cpu);
345 # The default 1 s poll in wait_all_children added a flat 1.01 s to every
346 # call, measured; 0 makes it block in waitpid instead.
347 486 $pm->set_waitpid_blocking_sleep(0);
348 3490 ▶ $pm->run_on_finish(sub {
Subroutine __ANON__: called CC=6 SCAR=22.8
349 3490 my ($pid, $exit, $id, $signal, $core, $data) = @_;
350 3490 ▶ if ($signal || $exit || !$data) {
Branch: 1 branch
BranchTrueFalseTotal
if ($signal or $exit or not $data) 0 3490 3490
Condition33%2/6
$signal or $exit or not $data
l!l&&r!l&&!r
003490
$signal or $exit
l!l&&r!l&&!r
003490
MC/DC0%0/3
$signal or $exit or not $data
$signal $exit not $data
Truth table25%1/4
A or B or not(C)
A= $signalB= $exitC= not $data
ABCresult
1XX1
01X1
0011
0000
351 0 push @errors, "dir: worker $pid died (exit $exit, signal $signal)";
352 0 return;
353 }
354 3490 ▶ push @errors, $data->{error} if defined $data->{error};
Branch: 1 branch
BranchTrueFalseTotal
if defined $data->{"error"} 0 3490 3490
355 3490 $emit->($_) for @{ $data->{paths} };
356 486 });
357 486 for my $job (@jobs) {
358 3654 ▶ $pm->start and next;
Branch: 1 branch
BranchTrueFalseTotal
$pm->start and next 3570 84 3654
359 84 my @found;
360 84 ▶ my $ok = eval { $find->($_, sub { push @found, $_[0] }) for @$job; 1 };
Subroutine __ANON__: called CC=1 SCAR=0.0
361 84 ▶ $pm->finish(0, { paths => \@found, error => $ok ? undef : "$@" });
Branch: 1 branch
BranchTrueFalseTotal
$ok ? : 84 0 84
362 }
363 402 $pm->wait_all_children;
364 402 ▶ die join "\n", @errors if @errors;
Branch: 1 branch
BranchTrueFalseTotal
if @errors 0 402 402
365 402 return;
366 }
367  
368 14 ▶sub execute {
Subroutine execute: called CC=17 SCAR=28.5
Pod documented
369 14 my ($cmd, $return, $die) = @_;
370 14 ▶ $return = 'exit' unless defined $return;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $return 4 10 14
371 14 ▶ $die = 1 unless defined $die;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $die 11 3 14
372 # \A..\z: "$" also matches before a trailing newline, so "exit\n" passed
373 # here, ran the command, and only then died with "broke pigeonholes".
374 14 ▶ if ($return !~ m/\A(?:exit|stdout|stderr|all)\z/) {
Branch: 1 branch
BranchTrueFalseTotal
unless ($return =~ /\A(?:exit|stdout|stderr|all)\z/) 2 12 14
375 2 die "you gave \$return = \"$return\", while this subroutine only accepts exit, stdout, stderr or all";
376 }
377 12 require Capture::Tiny; # ~7.5 ms
378 # Capture::Tiny redirects fd 1 and fd 2 and nothing else, so the child still
379 # inherits the parent's fd 0. A child that prompts -- rm on a
380 # write-protected file, cp -i, apt -- prompts only when fd 0 is a tty, then
381 # writes its question into the captured stderr where nobody can see it and
382 # blocks on the terminal for an answer the user does not know is wanted.
383 # (Observed 2026-09-13: `rm -r` on read-only cl.tex hung jobsearch/tex/tar.pl
384 # indefinitely.) Pointing fd 0 at /dev/null turns the prompt into an instant
385 # EOF, so such a child declines and exits instead of hanging.
386 #
387 # What goes on fd 0 is the read end of an empty pipe whose write end is
388 # closed, and only the descriptor is swapped, with dup2; the STDIN handle is
389 # left alone. Before system() forks, perl flushes every handle, and flushing
390 # one it reads from lseek()s its descriptor back to the position perl has
391 # read up to and, if that seek succeeds, discards the read-ahead buffer. A
392 # pipe cannot seek, so the buffer survives; the null device can, so with it
393 # on fd 0, as in 0.023, the buffered input was lost:
394 # `printf 'a\nb\nc\n' | perl -e 'while (<STDIN>) { print; execute("true") }'`
395 # printed only "a". Whatever holds fd 0 -- STDIN, or another handle when
396 # STDIN is closed -- is saved and put back, and fd 0 is closed again if it
397 # was closed before.
398 #
399 # fd 0 is saved before the pipe is made: when it is free, pipe() takes it,
400 # and a dup of fd 0 made afterwards saved the pipe, which was then
401 # "restored", leaving fd 0 open on it.
402 12 require POSIX;
403 12 my $saved_fd0 = POSIX::dup(0); # undef = fd 0 was closed
404 12 ▶ pipe my $eof, my $writer or die "cannot make a pipe for the command's input: $!";
Branch: 1 branch
BranchTrueFalseTotal
pipe my $eof, my $writer or die "cannot make a pipe for the command's input: $!" 12 0 12
405 12 close $writer;
406 12 my $eof_on_fd0 = fileno $eof == 0; # 1 = fd 0 was free and pipe() took it
407 12 ▶ unless ($eof_on_fd0) {
Branch: 1 branch
BranchTrueFalseTotal
unless ($eof_on_fd0) 11 1 12
408 11 ▶ defined POSIX::dup2(fileno $eof, 0) or die "cannot point fd 0 at an empty pipe: $!";
Branch: 1 branch
BranchTrueFalseTotal
defined POSIX::dup2(fileno $eof, 0) or die "cannot point fd 0 at an empty pipe: $!" 11 0 11
409 11 close $eof;
410 }
411 # An explicit sub{} rather than capture { ... }: the prototype that makes
412 # the block form legal is not visible at compile time under a runtime
413 # require, so perl would try to parse the block as an anonymous hash.
414 # The eval is so that fd 0 is restored even when capture() dies; 0.023 left
415 # it on the null device for the rest of the program.
416 12 my ($stdout, $stderr, $exit);
417 12 my $ok = eval {
418 11 ▶ ($stdout, $stderr, $exit) = Capture::Tiny::capture(sub {
Subroutine __ANON__: called CC=1 SCAR=0.0
419 11 system( $cmd )
420 12 });
421 11 1;
422 };
423 12 my $err = $@;
424 12 ▶ if (defined $saved_fd0) {
Branch: 2 branches
BranchTrueFalseTotal
if (defined $saved_fd0) { } 11 1 12
elsif ($eof_on_fd0) { } 1 0 1
425 11 ▶ defined POSIX::dup2($saved_fd0, 0) or die "cannot restore fd 0: $!";
Branch: 1 branch
BranchTrueFalseTotal
defined POSIX::dup2($saved_fd0, 0) or die "cannot restore fd 0: $!" 11 0 11
426 11 POSIX::close($saved_fd0);
427 } elsif ($eof_on_fd0) {
428 1 close $eof;
429 } else {
430 0 POSIX::close(0);
431 }
432 12 ▶ die $err unless $ok;
Branch: 1 branch
BranchTrueFalseTotal
unless $ok 1 11 12
433 # Truthiness, as the documentation describes it. `$die == 1` made 'yes' die
434 # with "isn't numeric" and made 2 mean "do not die".
435 11 ▶ if ($die && $exit != 0) {
Branch: 1 branch
BranchTrueFalseTotal
if ($die and $exit != 0) 3 8 11
Condition100%3/3
$die and $exit != 0
!ll&&!rl&&r
173
MC/DC100%2/2
$die and $exit != 0
$die $exit != 0
Truth table100%3/3
A and B
A= $dieB= $exit != 0
ABresult
0X0
100
111
436 3 say STDERR "exit = $exit";
437 3 say STDERR "STDOUT = $stdout";
438 3 say STDERR "STDERR = $stderr";
439 3 die "$cmd\n failed";
440 }
441 8 ▶ if ($return eq 'exit') {
Branch: 4 branches
BranchTrueFalseTotal
if ($return eq 'exit') { } 3 5 8
elsif ($return eq 'stderr') { } 1 4 5
elsif ($return eq 'stdout') { } 3 1 4
elsif ($return eq 'all') { } 1 0 1
442 3 return $exit
443 } elsif ($return eq 'stderr') {
444 1 chomp $stderr;
445 1 return $stderr
446 } elsif ($return eq 'stdout') {
447 3 chomp $stdout;
448 3 return $stdout
449 } elsif ($return eq 'all') {
450 1 chomp $stdout;
451 1 chomp $stderr;
452 return {
453 1 exit => $exit,
454 stdout => $stdout,
455 stderr => $stderr
456 }
457 } else {
458 0 die "$return broke pigeonholes"
459 }
460 0 return $stdout
461 }
462  
463 3 ▶sub file2string {
Subroutine file2string: called CC=2 SCAR=6.9
Pod documented
464 3 my ($file) = @_;
465 3 ▶ open my $fh, '<', $file or die "can't read $file: $!"; # was autodie
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, "<", $file or die "can't read ${file}: $!" 2 1 3
466 2 return do { local $/; <$fh> };
467 }
468  
469 31 ▶sub format_commas { # https://stackoverflow.com/questions/33442240/perl-printf-to-use-commas-as-thousands-separator
Subroutine format_commas: called CC=9 SCAR=22.0
Pod documented
470 31 my ($n, $format) = @_;
471 31 ▶ $format = '.%02d' unless defined $format;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $format 20 11 31
472 # '%.0u' is the documented way to ask for a whole number, but sprintf
473 # prints nothing for it only when the hundredths are 0: 1234.56 came out
474 # "1,23456". So '%.0u' is recognised here, rounds half up to a whole
475 # number, and prints no fraction at all.
476 31 my $whole = $format eq '%.0u';
477 # Infinity and NaN are returned as perl prints them. Run through the
478 # arithmetic below they came out "Inf.NaN", "-Inf.NaN" and "NaN.NaN".
479 # $n - $n is NaN for exactly these, and NaN is the one value not equal to
480 # itself.
481 31 my $diff = $n - $n;
482 31 ▶ return "$n" if $diff != $diff;
Branch: 1 branch
BranchTrueFalseTotal
if $diff != $diff 3 28 31
483 28 my $neg = $n < 0;
484 28 my $abs = abs $n;
485 28 my ($int, $cents);
486 28 ▶ if ($whole) {
Branch: 1 branch
BranchTrueFalseTotal
if ($whole) { } 11 17 28
487 11 $int = int($abs + .5);
488 11 $cents = 0;
489 } else {
490 17 $int = int $abs;
491 # int(100 * (.005 + frac)) is the original expression, and is exactly
492 # round-half-up to hundredths -- .005 * 100 is the +0.5 before truncation.
493 # It is kept verbatim so no already-correct result shifts by a cent.
494 17 $cents = int(100 * (.005 + ($abs - $int)));
495 # ...but it can land on 100 (.999 rounds up to 1.00), which used to be
496 # printed straight into the fraction: 999999.999 came out "999,999.100".
497 # The carry has to go into the integer part instead.
498 17 ▶ if ($cents >= 100) {
Branch: 1 branch
BranchTrueFalseTotal
if ($cents >= 100) 5 12 17
499 5 $int++;
500 5 $cents -= 100;
501 }
502 }
503 # Split off the sign and work on the magnitude: the old code fed a negative
504 # through int()/sprintf and produced "-1,234.-49" for -1234.5, and lost the
505 # sign entirely for -0.5. A value that rounds to zero stays unsigned, so
506 # -0.001 is "0.00" rather than "-0.00".
507 28 ▶ my $sign = ($neg && ($int || $cents)) ? '-' : '';
Branch: 1 branch
BranchTrueFalseTotal
$neg && ($int || $cents) ? : 5 23 28
Condition100%6/6
$neg and $int || $cents
!ll&&!rl&&r
2125
$int or $cents
l!l&&r!l&&!r
412
MC/DC100%3/3
$neg and $int || $cents
$neg $int $cents
Truth table100%4/4
A and B or C
A= $negB= $intC= $cents
ABCresult
0XX0
1000
11X1
1011
508 # '%.0f' rather than interpolating $int: a large magnitude stringifies in
509 # scientific notation, and the reverse/unpack grouping shredded that into
510 # "1e,+20" for 1e20.
511 28 my $digits = sprintf '%.0f', $int;
512 28 ▶ return $sign . reverse(join(',', unpack('(A3)*', reverse $digits))) . ($whole ? '' : sprintf($format, $cents));
Branch: 1 branch
BranchTrueFalseTotal
$whole ? : 11 17 28
513 }
514  
515 5 ▶sub json_file_to_ref {
Subroutine json_file_to_ref: called CC=4 SCAR=13.9
Pod documented
516 5 my ($json_filename) = @_;
517 5 ▶ die "$json_filename doesn't exist or isn't a file" unless -f $json_filename;
Branch: 1 branch
BranchTrueFalseTotal
unless -f $json_filename 1 4 5
518 4 ▶ die "$json_filename has 0 size" if -s $json_filename == 0;
Branch: 1 branch
BranchTrueFalseTotal
if -s $json_filename == 0 1 3 4
519 3 ▶ open my $fh, '<:raw', $json_filename or die "can't read $json_filename: $!";
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, "<:raw", $json_filename or die "can't read ${json_filename}: $!" 3 0 3
520 3 my $json = do { local $/; <$fh> }; # slurp; $/ restored immediately
521 3 close $fh;
522 # Python/NumPy emit bare NaN / Infinity / -Infinity, which are not valid
523 # JSON. Rewrite them to null (-> undef) only when they sit in JSON value
524 # position (after '[', ',' or ':'). That test alone also matched text
525 # inside a string -- "values: [1, NaN, 2]" became "values: [1, null, 2]"
526 # -- so the first alternative matches each whole string literal and
527 # (*SKIP)(*FAIL) resumes the scan after its closing quote.
528 3 $json =~ s/"(?:[^"\\]++|\\.)*+"(*SKIP)(*FAIL)|[\[,:]\s*\K-?(?:NaN|Infinity)(?=\s*[,\]}])/null/gs;
529 3 require JSON::MaybeXS; # ~4.3 ms
530 3 return JSON::MaybeXS::decode_json($json);
531 }
532  
533 # These return *some* key/value from the hash, not a random one -- whichever
534 # pair the hash's internal iterator yields first.
535 #
536 # They used to be `foreach my $key (keys %{$hash}) { return $key }`, which is
537 # O(n): `keys` in list context materializes EVERY key into a temporary list,
538 # and then the loop throws all but the first away. On a 50,000-key hash that
539 # was 1.48 ms per call; via each() it is 0.15 us, ~10,000x faster.
540 #
541 # The bare `keys` in void context resets the iterator, so repeated calls keep
542 # returning the first pair instead of walking off the end of the hash. It is
543 # reset again after the each(), which would otherwise leave it one pair in, so
544 # that a caller's own `while (each %h)` afterwards skipped the first pair.
545 # The old `keys`-in-list-context version left it at the start, and so do these.
546 #
547 # Falling off a foreach also returned '' rather than undef in scalar context;
548 # each() on an empty hash returns an empty list, so these now yield a true
549 # undef, which is what t/random.t always asserted.
550 4 ▶sub random_key {
Subroutine random_key: called CC=1 SCAR=0.0
Pod documented
551 4 my ($hash) = @_;
552 4 keys %{ $hash }; # reset the iterator
553 4 my ($key) = each %{ $hash };
554 4 keys %{ $hash };
555 4 return $key;
556 }
557  
558 4 ▶sub random_value {
Subroutine random_value: called CC=1 SCAR=0.0
Pod documented
559 4 my ($hash) = @_;
560 4 keys %{ $hash }; # reset the iterator
561 4 my (undef, $value) = each %{ $hash };
562 4 keys %{ $hash };
563 4 return $value;
564 }
565  
566 3 ▶sub ref_to_json_file {
Subroutine ref_to_json_file: called CC=4 SCAR=13.9
Pod documented
567 3 my ($ref, $json_filename) = @_;
568 3 my $ref_json_filename = ref $json_filename;
569 3 ▶ unless ($ref_json_filename eq '') {
Branch: 1 branch
BranchTrueFalseTotal
unless ($ref_json_filename eq "") 1 2 3
570 1 die "$json_filename isn't a scalar/string";
571 }
572 2 require JSON::MaybeXS;
573 2 require Term::ANSIColor;
574 2 ▶ open my $fh, '>:raw', $json_filename # Write it unmangled
Branch: 1 branch
BranchTrueFalseTotal
open my $fh, ">:raw", $json_filename or die "can't write ${json_filename}: $!" 2 0 2
575 or die "can't write $json_filename: $!"; # was autodie
576 2 say $fh JSON::MaybeXS::encode_json($ref);
577 # A full disk shows up at close, when the buffer is flushed. Unchecked,
578 # perl up to 5.20 reported "Wrote" and returned the name of a truncated
579 # file, and later perls died with "unable to close filehandle" from the
580 # implicit close under FATAL warnings.
581 2 ▶ close $fh or die "can't write $json_filename: $!";
Branch: 1 branch
BranchTrueFalseTotal
close $fh or die "can't write ${json_filename}: $!" 1 1 2
582 1 say 'Wrote ' . Term::ANSIColor::colored(['blue on_red'], $json_filename);
583 1 return $json_filename;
584 }
585  
586 # ----------------------------------------------------------------------------
587 # structure(): print the *shape* of a data structure instead of its values.
588 #
589 # p() from Data::Printer shows you every value; when a 50,000-element array
590 # holds the same kind of record over and over, that is the wrong tool. This
591 # merges all elements of an array into ONE representative node, so you see the
592 # schema once. Keys absent from some of the merged records are flagged '?';
593 # values that disagree about their type become a UNION.
594 #
595 # Two entry points, mirroring DDP's own p()/np() split:
596 # st() - prototyped, so `st %hash` works like `p %hash`. Use this
597 # interactively.
598 # structure() - plain sub, takes a reference: structure(\%hash), or the
599 # return value of another call. Use this from other code.
600 #
601 # Scalar::Util and Term::ANSIColor are require'd in structure(), which st()
602 # also funnels through, so the per-value helpers below can call them fully
603 # qualified without re-checking %INC on every node. Scalar::Util is not loaded
604 # at `use` time because it drags in List::Util's XS bootstrap (~4 ms) and only
605 # this family ever touches it.
606  
607 95 my %STRUCTURE_COLOR = (
608 key => 'bold', # hash key names
609 opt => 'red', # the '?' optional marker
610 HASH => 'bold cyan',
611 ARRAY => 'bold yellow',
612 UNION => 'bold magenta',
613 elem => 'bright_black', # the '[*]' / '<alt>' / '(empty)' markers
614 class => 'bold white', # blessed package names
615 number => 'green',
616 string => 'bright_blue',
617 bool => 'bright_magenta',
618 null => 'bright_black',
619 CODE => 'yellow',
620 other => 'red', # anything unexpected
621 );
622  
623 43 ▶sub _structure_color {
Subroutine _structure_color: called CC=3 SCAR=16.5
624 43 my ($name, $text, $use_color) = @_;
625 43 ▶ return $text unless $use_color;
Branch: 1 branch
BranchTrueFalseTotal
unless $use_color 43 0 43
626 0 ▶ return Term::ANSIColor::colored([$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{other}], $text);
Condition0%0/3
$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{"other"}
l!l&&r!l&&!r
000
MC/DC0%0/2
$STRUCTURE_COLOR{$name} // $STRUCTURE_COLOR{"other"}
$STRUCTURE_COLOR{$name} $STRUCTURE_COLOR{"other"}
Truth table0%0/3
A // B
A= $STRUCTURE_COLOR{$name}B= $STRUCTURE_COLOR{"other"}
ABresult
1X1
011
000
627 }
628  
629 # Classify one value. Blessed containers keep their package name so that
630 # Foo=HASH and a plain HASH don't get merged into the same node.
631 25 ▶sub _structure_kind {
Subroutine _structure_kind: called CC=13 SCAR=42.4
632 25 my ($v) = @_;
633 25 ▶ return 'null' unless defined $v;
Branch: 1 branch
BranchTrueFalseTotal
unless defined $v 0 25 25
634 25 my $r = ref $v;
635 25 ▶ return (Scalar::Util::looks_like_number($v) ? 'number' : 'string') if $r eq '';
Branch: 2 branches
BranchTrueFalseTotal
if $r eq "" 10 15 25
&Scalar::Util::looks_like_number($v) ? : 8 2 10
636 15 ▶ if (my $class = Scalar::Util::blessed($v)) {
Branch: 1 branch
BranchTrueFalseTotal
if (my $class = builtin::blessed($v)) 0 15 15
637 0 ▶ return 'bool' if $class eq 'JSON::PP::Boolean' || $class eq 'Types::Serialiser::Boolean';
Branch: 1 branch
BranchTrueFalseTotal
if $class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean" 0 0 0
Condition0%0/3
$class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean"
l!l&&r!l&&!r
000
MC/DC0%0/2
$class eq "JSON::PP::Boolean" or $class eq "Types::Serialiser::Boolean"
$class eq "JSON::PP::Boolean" $class eq "Types::Serialiser::Boolean"
Truth table0%0/3
A or B
A= $class eq "JSON::PP::Boolean"B= $class eq "Types::Serialiser::Boolean"
ABresult
1X1
011
000
638 0 ▶ return 'Regexp' if $class eq 'Regexp';
Branch: 1 branch
BranchTrueFalseTotal
if $class eq "Regexp" 0 0 0
639 0 ▶ my $rt = Scalar::Util::reftype($v) // '';
Condition0%0/2
builtin::reftype($v) // ""
l!l
00
MC/DC0%0/1
builtin::reftype($v) // ""
builtin::reftype($v)
Truth table0%0/2
A
A= builtin::reftype($v)
Aresult
11
00
640 0 ▶ return "$class=$rt" if $rt eq 'HASH' || $rt eq 'ARRAY' || $rt eq 'SCALAR' || $rt eq 'REF';
Branch: 1 branch
BranchTrueFalseTotal
if $rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF" 0 0 0
Condition0%0/9
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF"
l!l&&r!l&&!r
000
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR"
l!l&&r!l&&!r
000
$rt eq "HASH" or $rt eq "ARRAY"
l!l&&r!l&&!r
000
MC/DC0%0/4
$rt eq "HASH" or $rt eq "ARRAY" or $rt eq "SCALAR" or $rt eq "REF"
$rt eq "HASH" $rt eq "ARRAY" $rt eq "SCALAR" $rt eq "REF"
Truth table0%0/5
A or B or C or D
A= $rt eq "HASH"B= $rt eq "ARRAY"C= $rt eq "SCALAR"D= $rt eq "REF"
ABCDresult
1XXX1
01XX1
001X1
00011
00000
641 0 return $class;
642 }
643 15 return $r; # HASH ARRAY SCALAR REF CODE GLOB
644 }
645  
646 # Merge a list of same-position values into a single structural node:
647 # { kind => 'hash', keys => { k => { node => .., optional => 0|1 } }, n, class }
648 # { kind => 'array', node => .., n => #instances, lens => [lengths], class }
649 # { kind => 'sref', node => .. } # \$x and \\$x
650 # { kind => 'leaf', types => { typename => 1, ... } }
651 # { kind => 'union', alts => { kindname => node } }
652 # $stack holds the refaddrs of the containers we are currently inside, so a
653 # self-referential structure terminates instead of recursing forever.
654 23 ▶sub _structure_describe {
Subroutine _structure_describe: called CC=22 SCAR=33.2
655 23 my ($values, $opt, $depth, $stack) = @_;
656 23 my %by_kind;
657 23 push @{ $by_kind{ _structure_kind($_) } }, $_ for @{ $values };
658 23 my @kinds = sort keys %by_kind;
659  
660 23 ▶ if (@kinds > 1) {
Branch: 1 branch
BranchTrueFalseTotal
if (@kinds > 1) 0 23 23
661 return {
662 kind => 'union',
663 0 alts => { map { $_ => _structure_describe($by_kind{$_}, $opt, $depth, $stack) } @kinds },
664 };
665 }
666 23 ▶ my $k = $kinds[0] // 'null';
Condition50%1/2
$kinds[0] // "null"
l!l
230
MC/DC0%0/1
$kinds[0] // "null"
$kinds[0]
Truth table50%1/2
A
A= $kinds[0]
Aresult
11
00
667 23 my ($class, $reftype) = $k =~ /\A(.+)=(HASH|ARRAY|SCALAR|REF)\z/;
668 23 ▶ $reftype //= $k;
Condition33%1/3
$reftype //= $k
l!l&&r!l&&!r
0230
MC/DC0%0/2
$reftype //= $k
$reftype $k
Truth table33%1/3
A //= B
A= $reftypeB= $k
ABresult
1X1
011
000
669  
670 23 ▶ if ($reftype eq 'HASH' || $reftype eq 'ARRAY' || $reftype eq 'SCALAR' || $reftype eq 'REF') {
Branch: 1 branch
BranchTrueFalseTotal
if ($reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF") 15 8 23
Condition100%9/9
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF"
l!l&&r!l&&!r
1148
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR"
l!l&&r!l&&!r
10112
$reftype eq "HASH" or $reftype eq "ARRAY"
l!l&&r!l&&!r
7313
MC/DC100%4/4
$reftype eq "HASH" or $reftype eq "ARRAY" or $reftype eq "SCALAR" or $reftype eq "REF"
$reftype eq "HASH" $reftype eq "ARRAY" $reftype eq "SCALAR" $reftype eq "REF"
Truth table100%5/5
A or B or C or D
A= $reftype eq "HASH"B= $reftype eq "ARRAY"C= $reftype eq "SCALAR"D= $reftype eq "REF"
ABCDresult
1XXX1
01XX1
001X1
00011
00000
671 # Drop anything we're already inside: its shape is an ancestor's shape.
672 15 my @fresh = grep { !$stack->{ Scalar::Util::refaddr($_) } } @{ $values };
673 15 ▶ return { kind => 'leaf', types => { '(cycle)' => 1 } } unless @fresh;
Branch: 1 branch
BranchTrueFalseTotal
unless @fresh 0 15 15
674 15 ▶ if ($opt->{max_depth} && $depth >= $opt->{max_depth}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($opt->{"max_depth"} and $depth >= $opt->{"max_depth"}) 0 15 15
Condition33%1/3
$opt->{"max_depth"} and $depth >= $opt->{"max_depth"}
!ll&&!rl&&r
1500
MC/DC0%0/2
$opt->{"max_depth"} and $depth >= $opt->{"max_depth"}
$opt->{"max_depth"} $depth >= $opt->{"max_depth"}
Truth table33%1/3
A and B
A= $opt->{"max_depth"}B= $depth >= $opt->{"max_depth"}
ABresult
0X0
100
111
675 0 return { kind => 'leaf', types => { '(...)' => 1 } };
676 }
677 15 local @{ $stack }{ map { Scalar::Util::refaddr($_) } @fresh } = (1) x @fresh;
678  
679 15 ▶ if ($reftype eq 'HASH') {
Branch: 2 branches
BranchTrueFalseTotal
if ($reftype eq 'HASH') { } 7 8 15
elsif ($reftype eq 'ARRAY') { } 3 5 8
680 7 my %keys; # key => { vals => [...], seen => n }
681 7 for my $h (@fresh) {
682 7 for my $key (keys %{ $h }) {
683 9 push @{ $keys{$key}{vals} }, $h->{$key};
684 9 $keys{$key}{seen}++;
685 }
686 }
687 # A wide hash used as a dictionary has no interesting key names:
688 # merge every value into one representative entry instead.
689 7 ▶ if ($opt->{hash_max} && keys %keys > $opt->{hash_max}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}) 0 7 7
Condition33%1/3
$opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}
!ll&&!rl&&r
070
MC/DC0%0/2
$opt->{"hash_max"} and keys %keys > $opt->{"hash_max"}
$opt->{"hash_max"} keys %keys > $opt->{"hash_max"}
Truth table33%1/3
A and B
A= $opt->{"hash_max"}B= keys %keys > $opt->{"hash_max"}
ABresult
0X0
100
111
690 0 my @all = map { @{ $keys{$_}{vals} } } keys %keys;
691 return {
692 0 kind => 'hash',
693 class => $class,
694 n => scalar @fresh,
695 wide => scalar keys %keys,
696 keys => { '*' => {
697 node => _structure_describe(\@all, $opt, $depth + 1, $stack),
698 optional => 0,
699 }},
700 };
701 }
702 7 my %out;
703 7 for my $key (keys %keys) {
704 $out{$key} = {
705 node => _structure_describe($keys{$key}{vals}, $opt, $depth + 1, $stack),
706 9 ▶ optional => ($keys{$key}{seen} < @fresh ? 1 : 0),
Branch: 1 branch
BranchTrueFalseTotal
$keys{$key}{'seen'} < @fresh ? : 0 9 9
707 };
708 }
709 7 return { kind => 'hash', class => $class, n => scalar @fresh, keys => \%out };
710 }
711 elsif ($reftype eq 'ARRAY') {
712 3 my (@elems, @lens);
713 3 for my $a (@fresh) {
714 3 push @lens, scalar @{ $a };
715 3 push @elems, @{ $a };
716 }
717 return {
718 3 ▶ kind => 'array',
Branch: 1 branch
BranchTrueFalseTotal
@elems ? : 3 0 3
719 class => $class,
720 node => (@elems ? _structure_describe(\@elems, $opt, $depth + 1, $stack) : undef),
721 n => scalar @fresh,
722 lens => \@lens,
723 };
724 }
725 return { # SCALAR / REF
726 kind => 'sref',
727 class => $class,
728 5 node => _structure_describe([map { ${ $_ } } @fresh], $opt, $depth + 1, $stack),
729 };
730 }
731 8 return { kind => 'leaf', types => { $k => 1 } };
732 }
733  
734 3 ▶sub _structure_array_desc {
Subroutine _structure_array_desc: called CC=6 SCAR=18.5
735 3 my ($node, $c) = @_;
736 3 my @lens = @{ $node->{lens} };
737 3 my ($min, $max) = ($lens[0], $lens[0]);
738 3 ▶ for (@lens) { $min = $_ if $_ < $min; $max = $_ if $_ > $max }
Branch: 2 branches
BranchTrueFalseTotal
if $_ < $min 0 3 3
if $_ > $max 0 3 3
739 3 ▶ my $range = $min == $max ? $min : "$min-$max";
Branch: 1 branch
BranchTrueFalseTotal
$min == $max ? : 3 0 3
740 3 my $out = $c->('ARRAY', "ARRAY[$range]");
741 3 ▶ $out .= " (from $node->{n} instances)" if $node->{n} > 1;
Branch: 1 branch
BranchTrueFalseTotal
if $node->{"n"} > 1 0 3 3
742 3 return $out;
743 }
744  
745 23 ▶sub _structure_render {
Subroutine _structure_render: called CC=17 SCAR=31.7
746 23 my ($label, $node, $indent, $opt, $c, $lines) = @_;
747 23 ▶ my $pad = (' ' x $indent) . ($label // '');
Condition100%2/2
$label // ""
l!l
176
MC/DC100%1/1
$label // ""
$label
Truth table100%2/2
A
A= $label
Aresult
11
00
748 23 ▶ my $class = defined $node->{class} ? $c->('class', "$node->{class}=") : '';
Branch: 1 branch
BranchTrueFalseTotal
defined $node->{'class'} ? : 0 23 23
749  
750 23 ▶ if ($node->{kind} eq 'hash') {
Branch: 4 branches
BranchTrueFalseTotal
if ($node->{'kind'} eq 'hash') { } 7 16 23
elsif ($node->{'kind'} eq 'array') { } 3 13 16
elsif ($node->{'kind'} eq 'sref') { } 5 8 13
elsif ($node->{'kind'} eq 'union') { } 0 8 8
751 7 my @k = sort keys %{ $node->{keys} };
752 7 ▶ my $count = $node->{wide} // scalar @k;
Condition50%1/2
$node->{"wide"} // scalar @k
l!l
07
MC/DC0%0/1
$node->{"wide"} // scalar @k
$node->{"wide"}
Truth table50%1/2
A
A= $node->{"wide"}
Aresult
11
00
753 7 ▶ push @{ $lines }, $pad . $class . $c->('HASH', "HASH{$count}")
Branch: 2 branches
BranchTrueFalseTotal
$node->{'n'} > 1 ? : 0 7 7
$node->{'wide'} ? : 0 7 7
754 . ($node->{n} > 1 ? " (from $node->{n} instances)" : '')
755 7 . ($node->{wide} ? ' ' . $c->('elem', '(keys merged)') : '');
756 7 for my $key (@k) {
757 9 my $entry = $node->{keys}{$key};
758 my $name = ($key eq '*' && $node->{wide})
759 ? $c->('elem', '<*>') . ': '
760 9 ▶ : $c->('key', $key) . ($entry->{optional} ? $c->('opt', '?') : '') . ': ';
Branch: 2 branches
BranchTrueFalseTotal
$key eq '*' && $node->{'wide'} ? : 0 9 9
$entry->{'optional'} ? : 0 9 9
Condition33%1/3
$key eq "*" and $node->{"wide"}
!ll&&!rl&&r
900
MC/DC0%0/2
$key eq "*" and $node->{"wide"}
$key eq "*" $node->{"wide"}
Truth table33%1/3
A and B
A= $key eq "*"B= $node->{"wide"}
ABresult
0X0
100
111
761 9 _structure_render($name, $entry->{node}, $indent + 1, $opt, $c, $lines);
762 }
763 }
764 elsif ($node->{kind} eq 'array') {
765 3 push @{ $lines }, $pad . $class . _structure_array_desc($node, $c);
766 3 ▶ if ($node->{node}) {
Branch: 1 branch
BranchTrueFalseTotal
if ($node->{'node'}) { } 3 0 3
767 3 _structure_render($c->('elem', '[*]') . ': ', $node->{node}, $indent + 1, $opt, $c, $lines);
768 } else {
769 0 push @{ $lines }, (' ' x ($indent + 1)) . $c->('elem', '(empty)');
770 }
771 }
772 elsif ($node->{kind} eq 'sref') { # \$x: stay on the same line, no extra indent level
773 # The '\ ' joins the label rather than the pad, and $indent passes
774 # through unchanged, so whatever the referent holds is still indented
775 # from this line. Passing $pad with indent 0 put the line right but
776 # dropped the referent's own children back to depth 1.
777 5 ▶ _structure_render(($label // '') . $class . $c->('elem', '\\') . ' ', $node->{node}, $indent, $opt, $c, $lines);
Condition100%2/2
$label // ""
l!l
41
MC/DC100%1/1
$label // ""
$label
Truth table100%2/2
A
A= $label
Aresult
11
00
778 }
779 elsif ($node->{kind} eq 'union') {
780 0 push @{ $lines }, $pad . $c->('UNION', 'UNION');
781 0 for my $alt (sort keys %{ $node->{alts} }) {
782 0 _structure_render($c->('elem', "<$alt>") . ' ', $node->{alts}{$alt}, $indent + 1, $opt, $c, $lines);
783 }
784 }
785 else { # leaf
786 8 push @{ $lines }, $pad . join($c->('elem', '|'),
787 8 map { $c->($_, $_) } sort keys %{ $node->{types} });
788 }
789 23 return;
790 }
791  
792 7 ▶sub structure {
Subroutine structure: called CC=14 SCAR=32.8
Pod documented
793 7 my ($ref, %args) = @_;
794 7 my %opt = (
795 output => 'stdout', # 'stdout' | 'stderr' | 'string' (string: return it, print nothing)
796 color => 'auto', # 'auto' | 1 | 0
797 max_depth => 0, # 0 = unlimited
798 hash_max => 32, # hashes wider than this collapse to one '<*>' entry; 0 = never
799 as => undef, # optional header line, like DDP's "as"
800 %args,
801 );
802 7 my %allowed = map { $_ => 1 } qw(output color max_depth hash_max as);
803 7 ▶ if (my @bad = grep { !$allowed{$_} } keys %opt) {
Branch: 1 branch
BranchTrueFalseTotal
if (my(@bad) = grep({not $allowed{$_};} keys %opt)) 0 7 7
804 0 die "Unrecognized arguments: @bad\nAccepted args: " . join(', ', sort keys %allowed);
805 }
806 # undef as in dir(): this message, not "Use of uninitialized value".
807 7 for my $name (qw(output color)) {
808 14 ▶ die "$name must be defined" unless defined $opt{$name};
Branch: 1 branch
BranchTrueFalseTotal
unless defined $opt{$name} 1 13 14
809 }
810 6 ▶ die "output must be one of stdout, stderr, string" if $opt{output} !~ m/\A(stdout|stderr|string)\z/;
Branch: 1 branch
BranchTrueFalseTotal
unless $opt{"output"} =~ /\A(stdout|stderr|string)\z/ 0 6 6
811  
812 6 require Scalar::Util; # ~4 ms (List::Util XS bootstrap); see header note
813  
814 6 ▶ my $fh = $opt{output} eq 'stderr' ? \*STDERR : \*STDOUT;
Branch: 1 branch
BranchTrueFalseTotal
$opt{'output'} eq 'stderr' ? : 0 6 6
815 my $use_color = $opt{color} eq 'auto'
816 ? ((($ENV{CLICOLOR_FORCE} || ($opt{output} ne 'string' && -t $fh)) && !$ENV{NO_COLOR}) ? 1 : 0)
817 6 ▶ : $opt{color};
Branch: 2 branches
BranchTrueFalseTotal
$opt{'color'} eq 'auto' ? : 0 6 6
($ENV{'CLICOLOR_FORCE'} || $opt{'output'} ne 'string' && -t $fh) && !$ENV{'NO_COLOR'} ? : 0 0 0
Condition0%0/9
$ENV{"CLICOLOR_FORCE"} || $opt{"output"} ne "string" && -t $fh and not $ENV{"NO_COLOR"}
!ll&&!rl&&r
000
$ENV{"CLICOLOR_FORCE"} or $opt{"output"} ne "string" and -t $fh
l!l&&r!l&&!r
000
$opt{"output"} ne "string" and -t $fh
!ll&&!rl&&r
000
MC/DC0%0/4
$ENV{"CLICOLOR_FORCE"} || $opt{"output"} ne "string" && -t $fh and not $ENV{"NO_COLOR"}
$ENV{"CLICOLOR_FORCE"} $opt{"output"} ne "string" -t $fh not $ENV{"NO_COLOR"}
Truth table0%0/6
A or B and C and not(D)
A= $ENV{"CLICOLOR_FORCE"}B= $opt{"output"} ne "string"C= -t $fhD= not $ENV{"NO_COLOR"}
ABCDresult
00XX0
010X0
1XX00
01100
1XX11
01111
818 6 ▶ require Term::ANSIColor if $use_color; # ~0.8 ms, skipped entirely when piping
Branch: 1 branch
BranchTrueFalseTotal
if $use_color 0 6 6
819 6 ▶ my $c = sub { _structure_color($_[0], $_[1], $use_color) };
Subroutine __ANON__: called CC=1 SCAR=0.0
820  
821 6 my @lines;
822 6 ▶ push @lines, $c->('key', $opt{as}) if defined $opt{as};
Branch: 1 branch
BranchTrueFalseTotal
if defined $opt{"as"} 0 6 6
823 6 _structure_render(undef, _structure_describe([$ref], \%opt, 0, {}), 0, \%opt, $c, \@lines);
824 6 my $out = join("\n", @lines) . "\n";
825 0 ▶ print {$fh} $out unless $opt{output} eq 'string';
Branch: 1 branch
BranchTrueFalseTotal
unless $opt{"output"} eq "string" 0 6 6
826 6 return $out;
827 }
828  
829 # st(): structure() with DDP's calling convention, so you can write
830 #
831 # st %hash; st @array; st $ref; st %hash, max_depth => 2;
832 #
833 # instead of backslashing by hand. The \[@$%&] prototype makes perl pass a
834 # reference to the caller's variable, which is the whole trick -- but it also
835 # means the argument must BE a variable: st(\%h), st({...}) and
836 # st(func()) are compile-time errors, exactly as they are for DDP's p().
837 # Call structure() directly in those cases.
838 #
839 # NOTE: the obvious short name 's' is impossible. perl's tokenizer resolves a
840 # bare 's' to the substitution operator before it ever looks for a sub of that
841 # name, so `s $ref` dies with "Substitution pattern not terminated" and even
842 # `s($ref)` is parsed as s(pat)(repl). Only `&s($ref)` would work, which
843 # defeats the point. Hence 'st'.
844 1 ▶sub st (\[@$%&];%) {
Subroutine st: called CC=3 SCAR=13.3
Pod documented
845 1 my ($item, @args) = @_;
846 # `st %h` / `st @a` / `st &c` hand us the HASH/ARRAY/CODE ref directly, but
847 # `st $x` hands us a ref to the scalar $x -- one layer more than we want.
848 # The prototype rejects everything except a plain variable, so a SCALAR/REF
849 # here is always that added layer: peel exactly one. `st $hashref` then
850 # describes the hash, and `my $x = 42; st $x` prints 'number', not '\ number'.
851 0 ▶ my $ref = (ref($item) eq 'SCALAR' || ref($item) eq 'REF') ? ${ $item } : $item;
Branch: 1 branch
BranchTrueFalseTotal
ref $item eq 'SCALAR' || ref $item eq 'REF' ? : 0 1 1
Condition33%1/3
ref $item eq "SCALAR" or ref $item eq "REF"
l!l&&r!l&&!r
001
MC/DC0%0/2
ref $item eq "SCALAR" or ref $item eq "REF"
ref $item eq "SCALAR" ref $item eq "REF"
Truth table33%1/3
A or B
A= ref $item eq "SCALAR"B= ref $item eq "REF"
ABresult
1X1
011
000
852 1 return structure($ref, @args);
853 }
854  
855 95 1;
856 # from md2pod.pl πατερ ημων ο εν τοις ουρανοις, ἁγιασθήτω τὸ ὄνομά σου
857 =encoding utf8
858  
859 =head1 NAME
860  
861 Cn::Util - Small everyday helpers: directory listing, shell commands, JSON files, number formatting, and data-structure shapes
862  
863 =head1 Synopsis
864  
865 use Cn::Util;
866
867 my @scripts = dir(dir => '.', regex => '\.pl$', recursive => 1);
868 my $branch = execute('git rev-parse --abbrev-ref HEAD', 'stdout');
869 my $text = file2string('notes.txt');
870 print format_commas(1234567.891), "\n"; # 1,234,567.89
871
872 my $data = json_file_to_ref('results.json');
873 ref_to_json_file($data, 'copy.json');
874
875 st $data; # print the shape of $data, not its values
876 p $data; # Data::Printer's p(), loaded on first use
877  
878 =head1 Description
879  
880 C<Cn::Util> is a collection of the functions I reach for in nearly every script: listing a directory with file tests and a regex, running a shell command and keeping its output, slurping a file, reading and writing JSON, putting commas in a number, and seeing the I<shape> of a large data structure without scrolling through every value in it.
881  
882 Loading it is meant to be cheap. Everything that is not needed to compile the caller is loaded the first time a function needs it, so a script that only calls C<format_commas> does not pay for C<JSON::MaybeXS>, C<Capture::Tiny> or C<Data::Printer>.
883  
884 =head2 What C<use Cn::Util> does
885  
886 Besides exporting the ten functions below, C<use Cn::Util> does three things to the file and package that loads it:
887  
888 =over
889  
890 =item * It turns on C<autodie ':default'> in that file, from the C<use> line onward, so a bare C<open> or C<close> that fails dies with a message rather than returning false.
891  
892 =item * It gives that package C<p> and C<np> from C<Data::Printer> (C<DDP>). They are stubs until first called, so C<DDP> is loaded only by a script that prints something. A C<p> or C<np> the package already has, such as one from an earlier C<use DDP>, is left alone.
893  
894 =item * It loads C<Devel::Confess>, which is process-wide: every C<die> and C<warn> in the program carries a stack trace.
895  
896 =back
897  
898 C<use Cn::Util ();> imports nothing and skips the first two; C<Devel::Confess> is still loaded. C<use Cn::Util 'format_commas';> imports only the names given, and still does all three.
899  
900 =head1 Functions
901  
902 All functions are exported by default.
903  
904 =head2 dir
905  
906 my @files = dir(dir => '/data', regex => '\.csv$');
907 my @tree = dir(dir => '/data', regex => '\.csv$', recursive => 1, cpu => 8);
908 dir(dir => '/data', recursive => 1, on_match => sub { say $_[0] });
909  
910 Returns the paths under C<dir> whose base name matches C<regex> and that pass every file test in C<tests>. The paths are C<dir> joined to the entry with C</>; with the default C<< dir =E<gt> '.' >>, the leading C<./> is left off. C<.> and C<..> are never returned.
911  
912 It takes these named arguments:
913  
914 =over
915  
916 =item * C<dir> (default C<.>): The directory to list. Trailing slashes are removed, but C</> stays C</>. A C<dir> that is a symbolic link to a directory is listed through the link, recursively or not.
917  
918 =item * C<regex> (default C<.>): A pattern the base name must match, given as a string or a C<qr//>. An invalid pattern dies, naming it.
919  
920 =item * C<tests> (default C<f>): One or more of perl's file tests, as letters: C<rwxoRWXOezsfdlpSbcugkTB>. Every one must pass. C<'d'> lists directories; C<''> applies no test at all.
921  
922 =item * C<recursive> (default C<0>): C<1> descends into subdirectories, through C<File::Find>, without following symbolic links.
923  
924 =item * C<cpu> (default C<1>): For a recursive listing, the number of worker processes (through C<Parallel::ForkManager>). Ignored when C<recursive> is C<0>.
925  
926 =item * C<on_match> (no default): A code reference called with each path as it is found; C<dir> then returns an empty list, so a very large tree is never held in memory.
927  
928 =back
929  
930 The file tests share one C<stat> per path, so asking for four of them costs no more than asking for one. C<-l> is checked first, with C<lstat>.
931  
932 With C<cpu> greater than 1, C<dir> walks the tree breadth-first until one level holds four directories per worker, then shares those directories out. The results are the same as a serial walk, in a different order. C<on_match> still runs in the calling process, so it can change the caller's variables, but it is called once a worker's whole batch arrives rather than as each file is found.
933  
934 An unknown argument, an undefined C<tests>, C<dir>, C<regex> or C<cpu>, a letter that is not a file test, a C<cpu> that is not a positive integer, an C<on_match> that is not a code reference, and a directory that cannot be opened all die.
935  
936 =head2 execute
937  
938 my $exit = execute('make test'); # 0, or dies
939 my $out = execute('git log -1 --format=%H', 'stdout');
940 my $err = execute('perl -c script.pl', 'stderr');
941 my $all = execute('ls /nonexistent', 'all', 0); # does not die
942 # $all = { exit => 512, stdout => '', stderr => "ls: cannot access ..." }
943  
944 Runs C<$cmd> through the shell with C<system>, capturing its standard output and standard error with C<Capture::Tiny>. The second argument says what to return:
945  
946 =over
947  
948 =item * C<exit> (the default): the wait status, C<$?> -- 0 for success, 256 for C<exit 1>
949  
950 =item * C<stdout>: standard output, chomped
951  
952 =item * C<stderr>: standard error, chomped
953  
954 =item * C<all>: a hash reference with C<exit>, C<stdout> and C<stderr>
955  
956 =back
957  
958 The third argument, C<$die>, is true by default: a command that exits non-zero prints its exit status, standard output and standard error to C<STDERR> and dies. Pass a false value, such as 0, to have the status returned instead.
959  
960 The command's standard input is an empty pipe. A command that would prompt -- C<rm> on a write-protected file, C<cp -i> -- reads end-of-file and declines rather than waiting, unseen, for an answer. The caller's C<STDIN> is not touched, so input it has already read ahead is still there afterwards, even when C<execute> dies.
961  
962 =head2 file2string
963  
964 my $text = file2string('notes.txt');
965  
966 Returns the whole of a file as one string, read without any decoding layer. A file that cannot be opened dies, naming it.
967  
968 =head2 format_commas
969  
970 format_commas(1234567.891); # '1,234,567.89'
971 format_commas(1234567, '%.0u'); # '1,234,567'
972 format_commas(999999.999); # '1,000,000.00'
973 format_commas(-0.5); # '-0.50'
974 format_commas(1e20); # '100,000,000,000,000,000,000.00'
975  
976 Puts a comma between each group of three digits of the integer part and appends the hundredths, rounded half up. The second argument is the C<sprintf> format the hundredths are printed with, C<'.%02d'> by default; C<'%.0u'> rounds half up to a whole number and prints no fraction. Rounding that carries into the integer part does so (C<0.999> is C<1.00>), a negative number keeps its sign, a number that rounds to zero has none, and a large number is grouped as digits rather than printed in exponent form. Infinity and NaN are returned as perl prints them.
977  
978 =head2 json_file_to_ref
979  
980 my $ref = json_file_to_ref('results.json');
981  
982 Reads a JSON file and returns the decoded data, through C<JSON::MaybeXS>. Python's C<json> module and NumPy write C<NaN>, C<Infinity> and C<-Infinity> as bare words, which are not JSON; where one of those stands as a value it is read as C<null>, so it becomes C<undef>. The same words inside a string are left alone. A file that does not exist, or is empty, dies.
983  
984 =head2 ref_to_json_file
985  
986 ref_to_json_file($data, 'results.json'); # prints "Wrote results.json"
987  
988 Writes a data structure to a file as JSON, through C<JSON::MaybeXS>, prints C<Wrote> and the file name, and returns the file name. The file name must be a plain string; a reference dies, and so does a write that fails, such as on a full disk, before anything is printed.
989  
990 =head2 random_key, random_value
991  
992 my $key = random_key(\%hash);
993 my $value = random_value(\%hash);
994  
995 Return one key, or one value, of a hash -- whichever pair perl's hash iterator yields first, which is quick no matter how big the hash is. This is I<any> key, not a randomly chosen one: the same hash returns the same pair each time. It is useful for looking at one record of a hash of records. An empty hash returns C<undef>.
996  
997 Both reset the hash's iterator, so do not call them on a hash in the middle of an C<each> loop.
998  
999 =head2 structure
1000  
1001 my $text = structure($ref, output => 'string');
1002 structure(\%config, max_depth => 2, as => 'config');
1003  
1004 Prints the I<shape> of a data structure rather than its values: C<p> shows every element of a 50,000-element array of records, while C<structure> merges them into one and shows the record once.
1005  
1006 my @people = (
1007 { name => 'Ann', age => 31, langs => ['perl', 'R'] },
1008 { name => 'Bo', age => 28, langs => [], email => undef },
1009 );
1010 st @people;
1011  
1012 prints
1013  
1014 ARRAY[2]
1015 [*]: HASH{4} (from 2 instances)
1016 age: number
1017 email?: null
1018 langs: ARRAY[0-2] (from 2 instances)
1019 [*]: string
1020 name: string
1021  
1022 A key that only some of the merged hashes have is marked C<?>. Values in the same place that are of different kinds are shown as a C<UNION> of each kind. Arrays show the range of their lengths. Blessed references show their class. A reference back to a containing structure is shown as C<(cycle)> rather than followed.
1023  
1024 It takes these options after the reference:
1025  
1026 =over
1027  
1028 =item * C<output> (default C<stdout>): C<stdout> or C<stderr> to print there; C<string> to print nothing. The text is returned in every case.
1029  
1030 =item * C<color> (default C<auto>): C<1> or C<0>. C<auto> colours the output when it goes to a terminal, or when C<CLICOLOR_FORCE> is set, unless C<NO_COLOR> is set.
1031  
1032 =item * C<max_depth> (default C<0>): How many levels to descend before printing C<(...)>; C<0> is no limit.
1033  
1034 =item * C<hash_max> (default C<32>): A hash with more keys than this is taken to be a lookup table, and its values are merged into one entry, C<< E<lt>*E<gt> >>, rather than listed by key. C<0> lists every key.
1035  
1036 =item * C<as> (no default): A line to print above the structure.
1037  
1038 =back
1039  
1040 =head2 st
1041  
1042 st %hash;
1043 st @array;
1044 st $ref, max_depth => 2;
1045  
1046 C<structure>, called the way C<p> is called: the argument is a variable, not a reference to one, and the options follow it. As with C<p>, it must be a variable -- C<st(\%h)> and C<st(func())> do not compile; call C<structure> for those.
1047  
1048 =head1 Installation
1049  
1050 cpanm Cn::Util
1051  
1052 or, from a copy of the repository:
1053  
1054 perl Makefile.PL
1055 make
1056 make test
1057 make install
1058  
1059 =head1 Compatibility
1060  
1061 C<Cn::Util> runs on perl 5.10.1 and later, and is tested there. C<execute> hands its command to the shell, so the command has to be written for the shell of the system it runs on. On Windows, the paths C<dir> returns are joined with C</>, which perl accepts there, and a C<\> in the C<dir> argument is returned as C</>, so C<< dir =E<gt> q{C:\Temp} >> lists C<C:/Temp/a.txt>; recursive listings with C<cpu> rely on C<Parallel::ForkManager>'s emulated C<fork>.
1062  
1063 =head1 Author
1064  
1065 David E. Condon L<mailto:dec986@gmail.com>
1066  
1067 =head1 License
1068  
1069 This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
1070  
1071 =head1 Thanks
1072  
1073 A lot of this work used Claude AI, which was paid for by the University of Idaho's IMCI
↑ Summary