Detrans.pm

filter: Statementstmt 85.7% 126 / 147 Branchbran 78.3% 47 / 60 Conditioncond 50.0% 6 / 12 MC/DCmcdc 25.0% 2 / 8 Subroutinesub 85.0% 17 / 20 Podpod 46.6% 7 / 15 totaltotal 78.2% 205 / 262 CC 40 SCAR 18.7
CC
40
Cov
82%
add880s
14.3
convert
8.5
add066
5.6
CRAP 6.5
« Detrans/Rules.pm ↑ Summary Detrans/Config.pm »
1 package MARC::Detrans;
2  
3 6 ▶use strict;
Subroutine BEGIN: called CC=1 SCAR=0.0
4 6 ▶use warnings;
Subroutine BEGIN: called CC=1 SCAR=0.0
5 6 ▶use Carp qw( croak );
Subroutine BEGIN: called CC=1 SCAR=0.0
6 6 ▶use MARC::Detrans::Config;
Subroutine BEGIN: called CC=1 SCAR=0.0
7  
8 6 our $VERSION = '1.41';
9  
10 =head1 NAME
11  
12 MARC::Detrans - De-transliterate text and MARC records
13  
14 =head1 SYNOPSIS
15  
16 use MARC::Batch;
17 use MARC::Detrans;
18  
19 my $batch = MARC::Batch->new( 'marc.dat' );
20 my $detrans = MARC::Detrans->new( 'config.xml' );
21  
22 while ( my $record = $batch->next() ) {
23 my $newRecord = $detrans->convert( $record );
24 }
25  
26 =head1 DESCRIPTION
27  
28 MARC::Detrans is an eclectic addition to the already eclectic MARC::Record
29 distribution for de-transliterating MARC::Records. What is detransliteration
30 you ask? Well it's the opposite of transliteration, which according to the
31 Merriam-Webster:
32  
33 to represent or spell in the characters of another alphabet
34  
35 Traditionally when librarians catalog an item that has a title in a non-Roman
36 script they will follow transliteration rules for converting the title
37 into the Roman alphabet, so that the bibliographic record could be filed
38 into the card catalog or database index appropriately. These Romanization
39 Rules are published by the Library of Congress
40 http://www.loc.gov/catdir/cpso/roman.html.
41  
42 Now that computer screens can display Unicode fairly well it is now
43 desirable to display the original script for library users who are
44 more familiar with the original script. MARC::Detrans provides a framework
45 for detransliterating MARC records so that the orginal script is available
46 MARC-8 encoded in 880 fields. Very esoteric right?
47  
48 =head1 CONFIGURATION
49  
50 MARC::Detrans behavior is controlled by an XML configuration file. An
51 example of this configuration file can be found in the examples directory
52 of the MARC::Detrans distribution. The configuration determines the
53 detransliteration rules that will be used to add 880 fields to existing
54 records. It is hoped that people will contribute their configurations
55 for various languages to the MARC::Detrans project so that they can
56 be distributed with this package. For more information about the
57 configuration file see L<MARC::Detrans::Config>.
58  
59 In addition a sample driver program which uses MARC::Detrans has also
60 been included in the examples directory. This script is meant as a
61 jumping off point showing how to use the MARC::Detrans framework.
62  
63 =head1 METHODS
64  
65 =head2 new()
66  
67 The constructor which you should pass the path to your configuration file.
68  
69 my $detrans = MARC::Detrans->new( config => 'config.xml' );
70  
71 =cut
72  
73 6 ▶sub new {
Subroutine new: called CC=4 SCAR=13.9
Pod documented
74 6 my ($class,%args) = @_;
75 6 ▶ croak( "must supply config parameter" ) if ! exists $args{config};
Branch: 1 branch
BranchTrueFalseTotal
unless exists $args{"config"} 1 5 6
76 5 ▶ croak( "config file doesn't exist" ) if ! -f $args{config};
Branch: 1 branch
BranchTrueFalseTotal
unless -f $args{"config"} 1 4 5
77 4 my $config = MARC::Detrans::Config->new( $args{config} );
78 ## verify a few things
79 4 ▶ croak( $args{config} . ": missing code attribute in language element" )
Branch: 1 branch
BranchTrueFalseTotal
unless $config->languageCode 0 4 4
80 if ! $config->languageCode();
81 4 return _init( $class, $config );
82 }
83  
84 =head2 newFromConfig()
85  
86 If you want to supply your own MARC::Detran::Config object instead of
87 an XML file configuration as in new() you can use newFromConfig().
88 It's unlikely that you'll ever need to use this method.
89  
90 =cut
91  
92 2 ▶sub newFromConfig {
Subroutine newFromConfig: called CC=3 SCAR=12.5
Pod documented
93 2 my ($class,$config) = @_;
94 2 ▶ croak( "must supply MARC::Detrans::Config object" )
Branch: 1 branch
BranchTrueFalseTotal
unless ref $config and $config->isa("MARC::Detrans::Config") 0 2 2
Condition33%1/3
ref $config and $config->isa("MARC::Detrans::Config")
!ll&&!rl&&r
002
MC/DC0%0/2
ref $config and $config->isa("MARC::Detrans::Config")
ref $config $config->isa("MARC::Detrans::Config")
Truth table33%1/3
A and B
A= ref $configB= $config->isa("MARC::Detrans::Config")
ABresult
0X0
100
111
95 if ! ref($config) or ! $config->isa( 'MARC::Detrans::Config' );
96 2 return _init( $class, $config );
97 }
98  
99 ## helper to initialize an object
100 6 ▶sub _init {
Subroutine _init: called CC=2 SCAR=8.1
101 6 my ($class,$config) = @_;
102 6 ▶ return bless {
Condition33%1/3
ref $class || $class
l!l&&r!l&&!r
060
MC/DC0%0/2
ref $class || $class
ref $class $class
Truth table33%1/3
A || B
A= ref $classB= $class
ABresult
1X1
011
000
103 config => $config,
104 errors => [],
105 tallyAdd880 => 0,
106 tallyDetrans => {},
107 tallyCopy => {},
108 }, ref($class) || $class;
109 }
110  
111 =head2 convert()
112  
113 Pass a MARC::Record into convert() and you will be returned a
114 the same object with portions of it modified according to your
115 configuration file.
116  
117 IMPORTANT: if the record was not modified or an error was encountered
118 you will be returned undef instead of the MARC::Record object. You
119 will want to use the errors() method for diagnosing what happened.
120  
121 =cut
122  
123 24 ▶sub convert {
Subroutine convert: called CC=7 SCAR=21.4
Pod documented
124 24 my ($self,$record) = @_;
125 24 ▶ croak( "must pass in MARC::Record object" )
Branch: 1 branch
BranchTrueFalseTotal
unless ref $record and $record->isa("MARC::Record") 0 24 24
Condition33%1/3
ref $record and $record->isa("MARC::Record")
!ll&&!rl&&r
0024
MC/DC0%0/2
ref $record and $record->isa("MARC::Record")
ref $record $record->isa("MARC::Record")
Truth table33%1/3
A and B
A= ref $recordB= $record->isa("MARC::Record")
ABresult
0X0
100
111
126 if ! ref($record) or ! $record->isa( 'MARC::Record' );
127 24 my $config = $self->{config};
128  
129 ## make sure the script isn't already present
130 24 ▶ if ( $self->scriptAlreadyPresent($record) ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($self->scriptAlreadyPresent($record)) 1 23 24
131 1 $self->addError( "target script already present" );
132 1 return;
133 }
134  
135 ## check the language of the record
136 23 my $f008 = $record->field( '008' );
137 23 ▶ if ( ! $f008 ) {
Branch: 1 branch
BranchTrueFalseTotal
unless ($f008) 0 23 23
138 0 $self->addError( "can't determine language in record: missing 008" );
139 0 return;
140 }
141 23 my $lang = substr( $f008->data(), 35, 3 );
142 23 ▶ if ( $lang ne $config->languageCode() ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($lang ne $config->languageCode) 0 23 23
143 0 $self->addError( "record is not correct language: $lang instead of ".
144 $config->languageCode() );
145 0 return;
146 }
147  
148 ## add 880 fields and return if the record was edited
149 23 ▶ return $record if $self->add880s( $record );
Branch: 1 branch
BranchTrueFalseTotal
if $self->add880s($record) 19 4 23
150  
151 ## otherwise return undef since the record was not modified
152 4 return;
153 }
154  
155 ## internal helper for adding 880 fields to a record
156 ## will return 1 if the record is modified and 0 if it isn't
157  
158 23 ▶sub add880s {
Subroutine add880s: called CC=14 SCAR=26.6
Pod undocumented
159 23 my ($self,$r) = @_;
160 23 my $config = $self->{config};
161 23 my $rules = $config->rules();
162 23 my $names = $config->names();
163 23 my $scriptCode = $config->scriptCode();
164 23 my $scriptOrientation = $config->scriptOrientation();
165 23 my $count = 0;
166 23 my $edited = 0;
167  
168 ## see if the record is for a translation
169 ## since we'll need to skip some fields below if it is
170 23 my $isTranslation = isTranslation( $r );
171  
172 23 foreach my $tag ( $config->detransFields() ) {
173 42 FIELD: foreach my $field ( $r->field($tag) ) {
174 39 my @newSubfields = ();
175  
176 ## we don't process parallel titles
177 39 ▶ if ( isParallelTitle($field) ) {
Branch: 1 branch
BranchTrueFalseTotal
if (isParallelTitle($field)) 3 36 39
178 3 $self->addError( "field=$tag: skipped parallel title" );
179 3 next FIELD;
180 }
181  
182 ## we don't process 1XX and 7XX fields the record
183 ### is for a translation
184 36 ▶ if ( $isTranslation and $tag =~ /(1|7)\d\d/ ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($isTranslation and $tag =~ /(1|7)\d\d/) 3 33 36
Condition100%3/3
$isTranslation and $tag =~ /(1|7)\d\d/
!ll&&!rl&&r
3213
MC/DC100%2/2
$isTranslation and $tag =~ /(1|7)\d\d/
$isTranslation $tag =~ /(1|7)\d\d/
Truth table100%3/3
A and B
A= $isTranslationB= $tag =~ /(1|7)\d\d/
ABresult
0X0
100
111
185 3 $self->addError( "field=$tag: skipped because of translation" );
186 3 next FIELD;
187 }
188
189 ## if it's a field that might contain a name look it up
190 ## to see if it has a non-standard detransliteration
191 33 ▶ if ( isNameField($tag) ) {
Branch: 1 branch
BranchTrueFalseTotal
if (isNameField($tag)) 15 18 33
192 15 my $nameData = $names->convert( $field );
193 15 ▶ if ( $nameData ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($nameData) 0 15 15
194 0 $self->{tallyAdd880}++;
195 0 $count++;
196 0 add880( $r, $count, $field, $nameData, $scriptCode,
197 $scriptOrientation );
198 0 $edited = 1;
199 0 next FIELD;
200 }
201 }
202  
203 33 SUBFIELD: foreach my $subfield ( $field->subfields() ) {
204 39 my ($code,$data) = @$subfield;
205 39 ▶ if ($config->needsDetrans(field=>$tag,subfield=>$code)) {
Branch: 2 branches
BranchTrueFalseTotal
if ($config->needsDetrans('field', $tag, 'subfield', $code)) { } 35 4 39
elsif ($config->needsCopy('field', $tag, 'subfield', $code)) { } 4 0 4
206 35 my $new = $rules->convert( $data );
207 35 ▶ if ( ! defined $new ) {
Branch: 1 branch
BranchTrueFalseTotal
unless (defined $new) 1 34 35
208 1 $self->addError( "field=$tag subfield=$code: " .
209 $rules->error() );
210 1 next FIELD;
211 }
212 34 $self->{tallyDetrans}{"$tag-$code"}++;
213 34 push( @newSubfields, $code, $rules->convert($data) );
214 }
215 elsif ($config->needsCopy(field=>$tag,subfield=>$code)) {
216 4 $self->{tallyCopy}{"$tag-$code"}++;
217 4 push( @newSubfields, $code, $data);
218 }
219 }
220  
221 32 ▶ if ( @newSubfields ) {
Branch: 1 branch
BranchTrueFalseTotal
if (@newSubfields) 32 0 32
222 32 $self->{tallyAdd880}++;
223 32 $count++;
224 32 add880($r, $count, $field, \@newSubfields, $scriptCode,
225 $scriptOrientation );
226 32 $edited = 1;
227 }
228 }
229  
230 }
231  
232 23 ▶ if ( $edited ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($edited) 19 4 23
233 19 $self->add066($r);
234 }
235  
236 23 return $edited;
237 }
238  
239 24 ▶sub scriptAlreadyPresent {
Subroutine scriptAlreadyPresent: called CC=4 SCAR=14.0
Pod undocumented
240 24 my ($self,$record ) = @_;
241 24 my $config = $self->{config};
242 24 my $f066 = $record->field( '066' );
243 24 ▶ return 0 if ! $f066;
Branch: 1 branch
BranchTrueFalseTotal
unless $f066 23 1 24
244 1 foreach my $subfield( $f066->subfields() ) {
245 1 ▶ return 1 if grep { $_ eq $subfield->[1] } $config->allEscapeCodes();
Branch: 1 branch
BranchTrueFalseTotal
if grep {$_ eq $subfield->[1];} $config->allEscapeCodes 1 0 1
246 }
247 0 return 0;
248 }
249  
250 33 ▶sub isNameField {
Subroutine isNameField: called CC=1 SCAR=0.0
Pod undocumented
251 33 my $tag = shift;
252 33 return grep /^$tag$/, qw( 100 110 600 700 810 800 );
253 }
254  
255 39 ▶sub isParallelTitle {
Subroutine isParallelTitle: called CC=4 SCAR=14.1
Pod undocumented
256 39 my $field = shift;
257 39 ▶ return if $field->tag() ne 246;
Branch: 1 branch
BranchTrueFalseTotal
if $field->tag ne 246 36 3 39
258 3 ▶ return 1 if $field->indicator(2) =~ /1|5/;
Branch: 1 branch
BranchTrueFalseTotal
if $field->indicator(2) =~ /1|5/ 2 1 3
259 1 ▶ return 1 if ( $field->subfields() )[0]->[0] eq 'i';
Branch: 1 branch
BranchTrueFalseTotal
if +($field->subfields)[0]->[0] eq "i" 1 0 1
260 0 return;
261 }
262  
263 23 ▶sub isTranslation {
Subroutine isTranslation: called CC=3 SCAR=11.0
Pod undocumented
264 23 my $r = shift;
265 23 my $f041 = $r->field( '041' );
266 23 ▶ return if ! $f041;
Branch: 1 branch
BranchTrueFalseTotal
unless $f041 21 2 23
267 2 ▶ return if ! $f041->subfield( 'h' );
Branch: 1 branch
BranchTrueFalseTotal
unless $f041->subfield("h") 0 2 2
268 2 return 1;
269 }
270  
271 ## private helper function to add a single 880 based on the
272 ## tag and indicators of another field
273  
274 32 ▶sub add880 {
Subroutine add880: called CC=3 SCAR=11.0
Pod undocumented
275 32 my ( $record, $count, $field, $subfields, $scriptCode, $orientation ) = @_;
276 32 my $tag = $field->tag();
277 32 my $occurrence = sprintf( '%02d', $count );
278 32 my $sub6 = "$tag-$occurrence";
279 32 ▶ $sub6 .= "/$scriptCode" if defined $scriptCode;
Branch: 1 branch
BranchTrueFalseTotal
if defined $scriptCode 31 1 32
280 32 ▶ $sub6 .= "/$orientation" if defined $orientation;
Branch: 1 branch
BranchTrueFalseTotal
if defined $orientation 31 1 32
281 32 my $f880 = MARC::Field->new(
282 '880',
283 $field->indicator(1),
284 $field->indicator(2),
285 6 => $sub6, ## subfield 6
286 @$subfields ## the reset of the subfields
287 );
288 32 $record->insert_grouped_field( $f880 );
289  
290 ## now add to the original field
291 ## by creating a new field with the subfield 6
292 ## and replacing the old field with it
293 32 my @subfields = map { $_->[0], $_->[1] } $field->subfields();
294 32 unshift( @subfields, '6' => "880-$occurrence" );
295 32 $field->replace_with(
296 MARC::Field->new(
297 $tag,
298 $field->indicator(1), $field->indicator(2),
299 @subfields
300 )
301 );
302 }
303  
304 ## private helper function for adding a 066 indicating which
305 ## additional character sets were used in this record
306  
307 19 ▶sub add066 {
Subroutine add066: called CC=5 SCAR=17.2
Pod undocumented
308 19 my ($self,$record) = @_;
309 19 my $config = $self->{config};
310  
311 ## get a list of all the 066 fields used in this mapping
312 ## techically we should probably only list here the ones
313 ## that are *actually* used in this record...but there's
314 ## probably no harm in listing all of the ones used in this
315 ## configuration.
316 19 my @subfields;
317 19 foreach ( $config->allEscapeCodes() ) {
318 ## ignore (B
319 36 ▶ next if $_ eq '(B';
Branch: 1 branch
BranchTrueFalseTotal
if $_ eq "(B" 0 36 36
320 36 push( @subfields, 'c', $_ );
321 }
322  
323 19 ▶ return if @subfields == 0;
Branch: 1 branch
BranchTrueFalseTotal
if @subfields == 0 1 18 19
324  
325 ## don't obliterate an 066 that's already present
326 18 my $f066 = $record->field( '066' );
327 18 ▶ if ( $f066 ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($f066) { } 0 18 18
328 0 unshift( @subfields, map { $_->[0], $_->[1] } $f066->subfields() );
329 0 my $new066 = MARC::Field->new( '066', '', '', @subfields );
330 0 $f066->replace_with( $new066 );
331 } else {
332 18 $f066 = MARC::Field->new( '066', '', '', @subfields );
333 18 $record->insert_grouped_field( $f066 );
334 }
335 }
336  
337  
338 =head2 errors()
339  
340 Will return the latest errors encountered during a call to convert(). Can
341 be useful for determining why a call to convert() returned undef. A side
342 effect of calling errors() is that the errors storage is reset.
343  
344 =cut
345  
346 7 ▶sub errors {
Subroutine errors: called CC=1 SCAR=0.0
Pod documented
347 7 my $self = shift;
348 7 my @errors = @{ $self->{errors} };
349 7 $self->{errors} = [];
350 7 return @errors;
351 }
352  
353 ## this really should just be used internally...hence no POD
354 8 ▶sub addError {
Subroutine addError: called CC=1 SCAR=0.0
Pod undocumented
355 8 my ($self,$msg) = @_;
356 8 push( @{ $self->{errors} }, $msg );
357 }
358  
359 =head2 stats880sAdded()
360  
361 Returns the total amount of 880 fields added to records so far by
362 this MARC::Detrans object.
363  
364 =cut
365  
366 0 ▶sub stats880sAdded {
Subroutine stats880sAdded: not called CC=1 SCAR=6.9
Pod documented
367 0 my $self = shift;
368 0 return $self->{tallyAdd880};
369 }
370  
371 =head2 statsDetransliterated()
372  
373 Returns a hash of stats on the field_subfield combinations that
374 have been detransliterated by a MARC::Detrans object.
375  
376 =cut
377  
378 0 ▶sub statsDetransliterated {
Subroutine statsDetransliterated: not called CC=1 SCAR=6.9
Pod documented
379 0 return %{ shift->{tallyDetrans} };
380 }
381  
382 =head2 statsCopied()
383  
384 Returns a hash of stats on the field_subfield combinations that
385 have been copied by a MARC::Detrans object.
386  
387 =cut
388  
389 0 ▶sub statsCopied {
Subroutine statsCopied: not called CC=1 SCAR=6.9
Pod documented
390 0 return %{ shift->{tallyCopy} };
391 }
392  
393 =head1 AUTHORS
394  
395 MARC::Detrans was developed as part of a project funded by the Queens
396 Borough Public Library in New York City under the direction of Jane Jacobs.
397 It is their generosity that allowed this package to be released on CPAN.
398  
399 =over 4
400  
401 =item * Ed Summers <ehs@pobox.com>
402  
403 =back
404  
405 =cut
406  
407 6 1;
« Detrans/Rules.pm ↑ Summary Detrans/Config.pm »