File Coverage

blib/lib/Scalar/String.pm

Criterion	Covered	Total	%
statement	32	32	100.0
branch	1	2	50.0
condition	2	2	100.0
subroutine	14	14	100.0
pod	3	3	100.0
total	52	53	98.1

line	stmt	bran	sub	time	code
1					=head1 NAME
2
3					Scalar::String - string aspects of scalars
4
5					=head1 SYNOPSIS
6
7					use Scalar::String
8					qw(sclstr_is_upgraded sclstr_is_downgraded);
9
10					if(sclstr_is_upgraded($value)) { ...
11					if(sclstr_is_downgraded($value)) { ...
12
13					use Scalar::String qw(
14					sclstr_upgrade_inplace sclstr_upgraded
15					sclstr_downgrade_inplace sclstr_downgraded
16					);
17
18					sclstr_upgrade_inplace($value);
19					$value = sclstr_upgraded($value);
20					sclstr_downgrade_inplace($value);
21					$value = sclstr_downgraded($value);
22
23					=head1 DESCRIPTION
24
25					This module is about the string part of plain Perl scalars. A scalar has
26					a string value, which is notionally a sequence of Unicode codepoints, but
27					may be internally encoded in either ISO-8859-1 or UTF-8. In places, and
28					more so in older versions of Perl, the internal encoding shows through.
29					To fully understand Perl strings it is necessary to understand these
30					implementation details.
31
32					This module provides functions to classify a string by encoding and to
33					encode a string in a desired way.
34
35					This module is implemented in XS, with a pure Perl backup version for
36					systems that can't handle XS.
37
38					=head1 STRING ENCODING
39
40					ISO-8859-1 is a simple 8-bit character encoding, which represents the
41					first 256 Unicode characters (codepoints 0x00 to 0xff) in one octet each.
42					This is how strings were historically represented in Perl. A string
43					represented this way is referred to as "downgraded".
44
45					UTF-8 is a variable-width character encoding, which represents all
46					possible Unicode codepoints in differing numbers of octets. A design
47					feature of UTF-8 is that ASCII characters (codepoints 0x00 to 0x7f)
48					are each represented in a single octet, identically to their ISO-8859-1
49					encoding. Perl has its own variant of UTF-8, which can handle a wider
50					range of codepoints than Unicode formally allows. A string represented
51					in this variant UTF-8 is referred to as "upgraded".
52
53					A Perl string is physically represented as a string of octets along with
54					a flag that says whether the string is downgraded or upgraded. At this
55					level, to determine the Unicode codepoints that are represented requires
56					examining both parts of the representation. If the string contains only
57					ASCII characters then the octet sequence is identical in either encoding,
58					but Perl still maintains an encoding flag on such a string. A string
59					is always either downgraded or upgraded; it is never both or neither.
60
61					When handling string input, it is good form to operate only on the Unicode
62					characters represented by the string, ignoring the manner in which they
63					are encoded. Basic string operations such as concatenation work this way
64					(except for a bug in perl 5.6.0), so simple code written in pure Perl is
65					generally safe on this front. Pieces of character-based code can pass
66					around strings among themselves, and always get consistent behaviour,
67					without worrying about the way in which the characters are encoded.
68
69					However, due to an historical accident, a lot of C code that interfaces
70					with Perl looks at the octets used to represent a string without also
71					examining the encoding flag. Such code gives inconsistent behaviour for
72					the same character sequence represented in the different ways. In perl
73					5.6, many pure Perl operations (such as regular expression matching)
74					also work this way, though some of them can be induced to work correctly
75					by using the L pragma. In perl 5.8, regular expression matching
76					is character-based by default, but many I/O functions (such as C)
77					are still octet-based.
78
79					Where code that operates on the octets of a string must be used by code
80					that operates on characters, the latter needs to pay attention to the
81					encoding of its strings. Commonly, the octet-based code expects its
82					input to be represented in a particular encoding, in which case the
83					character-based code must oblige by forcing strings to that encoding
84					before they are passed in. There are other usage patterns too.
85
86					You will be least confused if you think about a Perl string as a character
87					sequence plus an encoding flag. You should normally operate on the
88					character sequence and not care about the encoding flag. Occasionally you
89					must pay attention to the flag in addition to the characters. Unless you
90					are writing C code, you should try not to think about a string the other
91					way round, as an octet sequence plus encoding flag.
92
93					=cut
94
95					package Scalar::String;
96
97	2		2	26243	{ use 5.006; }
	2			7
	2			140
98	2		2	12	use warnings;
	2			5
	2			65
99	2		2	27	use strict;
	2			3
	2			180
100
101					our $VERSION = "0.002";
102
103	2		2	1886	use parent "Exporter";
	2			642
	2			12
104					our @EXPORT_OK = qw(
105					sclstr_is_upgraded sclstr_is_downgraded
106					sclstr_upgrade_inplace sclstr_upgraded
107					sclstr_downgrade_inplace sclstr_downgraded
108					);
109
110					eval { local $SIG{__DIE__};
111					require XSLoader;
112					XSLoader::load(__PACKAGE__, $VERSION);
113					};
114
115					if($@ eq "") {
116					close(DATA);
117					} else {
118					(my $filename = __FILE__) =~ tr# -~##cd;
119					local $/ = undef;
120					my $pp_code = "#line 128 \"$filename\"\n".;
121					close(DATA);
122					{
123					local $SIG{__DIE__};
124					eval $pp_code;
125					}
126					die $@ if $@ ne "";
127					}
128
129	1		1	6	1;
	1			2
	1			82
130	1	50	1	1018
131					__DATA__