File Coverage

lib/JSON/XS.pm
Criterion Covered Total %
statement 1 3 33.3
branch n/a
condition n/a
subroutine 1 1 100.0
pod n/a
total 2 4 50.0


line stmt bran cond sub pod time code
1             =head1 NAME
2              
3             JSON::XS - JSON serialising/deserialising, done correctly and fast
4              
5             =encoding utf-8
6              
7             JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ
8             (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html)
9              
10             =head1 SYNOPSIS
11              
12             use JSON::XS;
13 1     1   369  
  0            
  0            
14             # exported functions, they croak on error
15             # and expect/generate UTF-8
16              
17             $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref;
18             $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text;
19              
20             # OO-interface
21              
22             $coder = JSON::XS->new->ascii->pretty->allow_nonref;
23             $pretty_printed_unencoded = $coder->encode ($perl_scalar);
24             $perl_scalar = $coder->decode ($unicode_json_text);
25              
26             # Note that JSON version 2.0 and above will automatically use JSON::XS
27             # if available, at virtually no speed overhead either, so you should
28             # be able to just:
29            
30             use JSON;
31              
32             # and do the same things, except that you have a pure-perl fallback now.
33              
34             =head1 DESCRIPTION
35              
36             This module converts Perl data structures to JSON and vice versa. Its
37             primary goal is to be I and its secondary goal is to be
38             I. To reach the latter goal it was written in C.
39              
40             Beginning with version 2.0 of the JSON module, when both JSON and
41             JSON::XS are installed, then JSON will fall back on JSON::XS (this can be
42             overridden) with no overhead due to emulation (by inheriting constructor
43             and methods). If JSON::XS is not available, it will fall back to the
44             compatible JSON::PP module as backend, so using JSON instead of JSON::XS
45             gives you a portable JSON API that can be fast when you need and doesn't
46             require a C compiler when that is a problem.
47              
48             As this is the n-th-something JSON module on CPAN, what was the reason
49             to write yet another JSON module? While it seems there are many JSON
50             modules, none of them correctly handle all corner cases, and in most cases
51             their maintainers are unresponsive, gone missing, or not listening to bug
52             reports for other reasons.
53              
54             See MAPPING, below, on how JSON::XS maps perl values to JSON values and
55             vice versa.
56              
57             =head2 FEATURES
58              
59             =over 4
60              
61             =item * correct Unicode handling
62              
63             This module knows how to handle Unicode, documents how and when it does
64             so, and even documents what "correct" means.
65              
66             =item * round-trip integrity
67              
68             When you serialise a perl data structure using only data types supported
69             by JSON and Perl, the deserialised data structure is identical on the Perl
70             level. (e.g. the string "2.0" doesn't suddenly become "2" just because
71             it looks like a number). There I minor exceptions to this, read the
72             MAPPING section below to learn about those.
73              
74             =item * strict checking of JSON correctness
75              
76             There is no guessing, no generating of illegal JSON texts by default,
77             and only JSON is accepted as input by default (the latter is a security
78             feature).
79              
80             =item * fast
81              
82             Compared to other JSON modules and other serialisers such as Storable,
83             this module usually compares favourably in terms of speed, too.
84              
85             =item * simple to use
86              
87             This module has both a simple functional interface as well as an object
88             oriented interface.
89              
90             =item * reasonably versatile output formats
91              
92             You can choose between the most compact guaranteed-single-line format
93             possible (nice for simple line-based protocols), a pure-ASCII format
94             (for when your transport is not 8-bit clean, still supports the whole
95             Unicode range), or a pretty-printed format (for when you want to read that
96             stuff). Or you can combine those features in whatever way you like.
97              
98             =back
99              
100             =cut
101              
102             package JSON::XS;
103              
104             use common::sense;
105              
106             our $VERSION = 3.01;
107             our @ISA = qw(Exporter);
108              
109             our @EXPORT = qw(encode_json decode_json);
110              
111             use Exporter;
112             use XSLoader;
113              
114             use Types::Serialiser ();
115              
116             =head1 FUNCTIONAL INTERFACE
117              
118             The following convenience methods are provided by this module. They are
119             exported by default:
120              
121             =over 4
122              
123             =item $json_text = encode_json $perl_scalar
124              
125             Converts the given Perl data structure to a UTF-8 encoded, binary string
126             (that is, the string contains octets only). Croaks on error.
127              
128             This function call is functionally identical to:
129              
130             $json_text = JSON::XS->new->utf8->encode ($perl_scalar)
131              
132             Except being faster.
133              
134             =item $perl_scalar = decode_json $json_text
135              
136             The opposite of C: expects an UTF-8 (binary) string and tries
137             to parse that as an UTF-8 encoded JSON text, returning the resulting
138             reference. Croaks on error.
139              
140             This function call is functionally identical to:
141              
142             $perl_scalar = JSON::XS->new->utf8->decode ($json_text)
143              
144             Except being faster.
145              
146             =back
147              
148              
149             =head1 A FEW NOTES ON UNICODE AND PERL
150              
151             Since this often leads to confusion, here are a few very clear words on
152             how Unicode works in Perl, modulo bugs.
153              
154             =over 4
155              
156             =item 1. Perl strings can store characters with ordinal values > 255.
157              
158             This enables you to store Unicode characters as single characters in a
159             Perl string - very natural.
160              
161             =item 2. Perl does I associate an encoding with your strings.
162              
163             ... until you force it to, e.g. when matching it against a regex, or
164             printing the scalar to a file, in which case Perl either interprets your
165             string as locale-encoded text, octets/binary, or as Unicode, depending
166             on various settings. In no case is an encoding stored together with your
167             data, it is I that decides encoding, not any magical meta data.
168              
169             =item 3. The internal utf-8 flag has no meaning with regards to the
170             encoding of your string.
171              
172             Just ignore that flag unless you debug a Perl bug, a module written in
173             XS or want to dive into the internals of perl. Otherwise it will only
174             confuse you, as, despite the name, it says nothing about how your string
175             is encoded. You can have Unicode strings with that flag set, with that
176             flag clear, and you can have binary data with that flag set and that flag
177             clear. Other possibilities exist, too.
178              
179             If you didn't know about that flag, just the better, pretend it doesn't
180             exist.
181              
182             =item 4. A "Unicode String" is simply a string where each character can be
183             validly interpreted as a Unicode code point.
184              
185             If you have UTF-8 encoded data, it is no longer a Unicode string, but a
186             Unicode string encoded in UTF-8, giving you a binary string.
187              
188             =item 5. A string containing "high" (> 255) character values is I a UTF-8 string.
189              
190             It's a fact. Learn to live with it.
191              
192             =back
193              
194             I hope this helps :)
195              
196              
197             =head1 OBJECT-ORIENTED INTERFACE
198              
199             The object oriented interface lets you configure your own encoding or
200             decoding style, within the limits of supported formats.
201              
202             =over 4
203              
204             =item $json = new JSON::XS
205              
206             Creates a new JSON::XS object that can be used to de/encode JSON
207             strings. All boolean flags described below are by default I.
208              
209             The mutators for flags all return the JSON object again and thus calls can
210             be chained:
211              
212             my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]})
213             => {"a": [1, 2]}
214              
215             =item $json = $json->ascii ([$enable])
216              
217             =item $enabled = $json->get_ascii
218              
219             If C<$enable> is true (or missing), then the C method will not
220             generate characters outside the code range C<0..127> (which is ASCII). Any
221             Unicode characters outside that range will be escaped using either a
222             single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence,
223             as per RFC4627. The resulting encoded JSON text can be treated as a native
224             Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string,
225             or any other superset of ASCII.
226              
227             If C<$enable> is false, then the C method will not escape Unicode
228             characters unless required by the JSON syntax or other flags. This results
229             in a faster and more compact format.
230              
231             See also the section I later in this
232             document.
233              
234             The main use for this flag is to produce JSON texts that can be
235             transmitted over a 7-bit channel, as the encoded JSON texts will not
236             contain any 8 bit characters.
237              
238             JSON::XS->new->ascii (1)->encode ([chr 0x10401])
239             => ["\ud801\udc01"]
240              
241             =item $json = $json->latin1 ([$enable])
242              
243             =item $enabled = $json->get_latin1
244              
245             If C<$enable> is true (or missing), then the C method will encode
246             the resulting JSON text as latin1 (or iso-8859-1), escaping any characters
247             outside the code range C<0..255>. The resulting string can be treated as a
248             latin1-encoded JSON text or a native Unicode string. The C method
249             will not be affected in any way by this flag, as C by default
250             expects Unicode, which is a strict superset of latin1.
251              
252             If C<$enable> is false, then the C method will not escape Unicode
253             characters unless required by the JSON syntax or other flags.
254              
255             See also the section I later in this
256             document.
257              
258             The main use for this flag is efficiently encoding binary data as JSON
259             text, as most octets will not be escaped, resulting in a smaller encoded
260             size. The disadvantage is that the resulting JSON text is encoded
261             in latin1 (and must correctly be treated as such when storing and
262             transferring), a rare encoding for JSON. It is therefore most useful when
263             you want to store data structures known to contain binary data efficiently
264             in files or databases, not when talking to other JSON encoders/decoders.
265              
266             JSON::XS->new->latin1->encode (["\x{89}\x{abc}"]
267             => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not)
268              
269             =item $json = $json->utf8 ([$enable])
270              
271             =item $enabled = $json->get_utf8
272              
273             If C<$enable> is true (or missing), then the C method will encode
274             the JSON result into UTF-8, as required by many protocols, while the
275             C method expects to be handled an UTF-8-encoded string. Please
276             note that UTF-8-encoded strings do not contain any characters outside the
277             range C<0..255>, they are thus useful for bytewise/binary I/O. In future
278             versions, enabling this option might enable autodetection of the UTF-16
279             and UTF-32 encoding families, as described in RFC4627.
280              
281             If C<$enable> is false, then the C method will return the JSON
282             string as a (non-encoded) Unicode string, while C expects thus a
283             Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs
284             to be done yourself, e.g. using the Encode module.
285              
286             See also the section I later in this
287             document.
288              
289             Example, output UTF-16BE-encoded JSON:
290              
291             use Encode;
292             $jsontext = encode "UTF-16BE", JSON::XS->new->encode ($object);
293              
294             Example, decode UTF-32LE-encoded JSON:
295              
296             use Encode;
297             $object = JSON::XS->new->decode (decode "UTF-32LE", $jsontext);
298              
299             =item $json = $json->pretty ([$enable])
300              
301             This enables (or disables) all of the C, C and
302             C (and in the future possibly more) flags in one call to
303             generate the most readable (or most compact) form possible.
304              
305             Example, pretty-print some simple structure:
306              
307             my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]})
308             =>
309             {
310             "a" : [
311             1,
312             2
313             ]
314             }
315              
316             =item $json = $json->indent ([$enable])
317              
318             =item $enabled = $json->get_indent
319              
320             If C<$enable> is true (or missing), then the C method will use a multiline
321             format as output, putting every array member or object/hash key-value pair
322             into its own line, indenting them properly.
323              
324             If C<$enable> is false, no newlines or indenting will be produced, and the
325             resulting JSON text is guaranteed not to contain any C.
326              
327             This setting has no effect when decoding JSON texts.
328              
329             =item $json = $json->space_before ([$enable])
330              
331             =item $enabled = $json->get_space_before
332              
333             If C<$enable> is true (or missing), then the C method will add an extra
334             optional space before the C<:> separating keys from values in JSON objects.
335              
336             If C<$enable> is false, then the C method will not add any extra
337             space at those places.
338              
339             This setting has no effect when decoding JSON texts. You will also
340             most likely combine this setting with C.
341              
342             Example, space_before enabled, space_after and indent disabled:
343              
344             {"key" :"value"}
345              
346             =item $json = $json->space_after ([$enable])
347              
348             =item $enabled = $json->get_space_after
349              
350             If C<$enable> is true (or missing), then the C method will add an extra
351             optional space after the C<:> separating keys from values in JSON objects
352             and extra whitespace after the C<,> separating key-value pairs and array
353             members.
354              
355             If C<$enable> is false, then the C method will not add any extra
356             space at those places.
357              
358             This setting has no effect when decoding JSON texts.
359              
360             Example, space_before and indent disabled, space_after enabled:
361              
362             {"key": "value"}
363              
364             =item $json = $json->relaxed ([$enable])
365              
366             =item $enabled = $json->get_relaxed
367              
368             If C<$enable> is true (or missing), then C will accept some
369             extensions to normal JSON syntax (see below). C will not be
370             affected in anyway. I
371             JSON texts as if they were valid!>. I suggest only to use this option to
372             parse application-specific files written by humans (configuration files,
373             resource files etc.)
374              
375             If C<$enable> is false (the default), then C will only accept
376             valid JSON texts.
377              
378             Currently accepted extensions are:
379              
380             =over 4
381              
382             =item * list items can have an end-comma
383              
384             JSON I array elements and key-value pairs with commas. This
385             can be annoying if you write JSON texts manually and want to be able to
386             quickly append elements, so this extension accepts comma at the end of
387             such items not just between them:
388              
389             [
390             1,
391             2, <- this comma not normally allowed
392             ]
393             {
394             "k1": "v1",
395             "k2": "v2", <- this comma not normally allowed
396             }
397              
398             =item * shell-style '#'-comments
399              
400             Whenever JSON allows whitespace, shell-style comments are additionally
401             allowed. They are terminated by the first carriage-return or line-feed
402             character, after which more white-space and comments are allowed.
403              
404             [
405             1, # this comment not allowed in JSON
406             # neither this one...
407             ]
408              
409             =back
410              
411             =item $json = $json->canonical ([$enable])
412              
413             =item $enabled = $json->get_canonical
414              
415             If C<$enable> is true (or missing), then the C method will output JSON objects
416             by sorting their keys. This is adding a comparatively high overhead.
417              
418             If C<$enable> is false, then the C method will output key-value
419             pairs in the order Perl stores them (which will likely change between runs
420             of the same script, and can change even within the same run from 5.18
421             onwards).
422              
423             This option is useful if you want the same data structure to be encoded as
424             the same JSON text (given the same overall settings). If it is disabled,
425             the same hash might be encoded differently even if contains the same data,
426             as key-value pairs have no inherent ordering in Perl.
427              
428             This setting has no effect when decoding JSON texts.
429              
430             This setting has currently no effect on tied hashes.
431              
432             =item $json = $json->allow_nonref ([$enable])
433              
434             =item $enabled = $json->get_allow_nonref
435              
436             If C<$enable> is true (or missing), then the C method can convert a
437             non-reference into its corresponding string, number or null JSON value,
438             which is an extension to RFC4627. Likewise, C will accept those JSON
439             values instead of croaking.
440              
441             If C<$enable> is false, then the C method will croak if it isn't
442             passed an arrayref or hashref, as JSON texts must either be an object
443             or array. Likewise, C will croak if given something that is not a
444             JSON object or array.
445              
446             Example, encode a Perl scalar as JSON value with enabled C,
447             resulting in an invalid JSON text:
448              
449             JSON::XS->new->allow_nonref->encode ("Hello, World!")
450             => "Hello, World!"
451              
452             =item $json = $json->allow_unknown ([$enable])
453              
454             =item $enabled = $json->get_allow_unknown
455              
456             If C<$enable> is true (or missing), then C will I throw an
457             exception when it encounters values it cannot represent in JSON (for
458             example, filehandles) but instead will encode a JSON C value. Note
459             that blessed objects are not included here and are handled separately by
460             c.
461              
462             If C<$enable> is false (the default), then C will throw an
463             exception when it encounters anything it cannot encode as JSON.
464              
465             This option does not affect C in any way, and it is recommended to
466             leave it off unless you know your communications partner.
467              
468             =item $json = $json->allow_blessed ([$enable])
469              
470             =item $enabled = $json->get_allow_blessed
471              
472             See L for details.
473              
474             If C<$enable> is true (or missing), then the C method will not
475             barf when it encounters a blessed reference that it cannot convert
476             otherwise. Instead, a JSON C value is encoded instead of the object.
477              
478             If C<$enable> is false (the default), then C will throw an
479             exception when it encounters a blessed object that it cannot convert
480             otherwise.
481              
482             This setting has no effect on C.
483              
484             =item $json = $json->convert_blessed ([$enable])
485              
486             =item $enabled = $json->get_convert_blessed
487              
488             See L for details.
489              
490             If C<$enable> is true (or missing), then C, upon encountering a
491             blessed object, will check for the availability of the C method
492             on the object's class. If found, it will be called in scalar context and
493             the resulting scalar will be encoded instead of the object.
494              
495             The C method may safely call die if it wants. If C
496             returns other blessed objects, those will be handled in the same
497             way. C must take care of not causing an endless recursion cycle
498             (== crash) in this case. The name of C was chosen because other
499             methods called by the Perl core (== not by the user of the object) are
500             usually in upper case letters and to avoid collisions with any C
501             function or method.
502              
503             If C<$enable> is false (the default), then C will not consider
504             this type of conversion.
505              
506             This setting has no effect on C.
507              
508             =item $json = $json->allow_tags ([$enable])
509              
510             =item $enabled = $json->allow_tags
511              
512             See L for details.
513              
514             If C<$enable> is true (or missing), then C, upon encountering a
515             blessed object, will check for the availability of the C method on
516             the object's class. If found, it will be used to serialise the object into
517             a nonstandard tagged JSON value (that JSON decoders cannot decode).
518              
519             It also causes C to parse such tagged JSON values and deserialise
520             them via a call to the C method.
521              
522             If C<$enable> is false (the default), then C will not consider
523             this type of conversion, and tagged JSON values will cause a parse error
524             in C, as if tags were not part of the grammar.
525              
526             =item $json = $json->filter_json_object ([$coderef->($hashref)])
527              
528             When C<$coderef> is specified, it will be called from C each
529             time it decodes a JSON object. The only argument is a reference to the
530             newly-created hash. If the code references returns a single scalar (which
531             need not be a reference), this value (i.e. a copy of that scalar to avoid
532             aliasing) is inserted into the deserialised data structure. If it returns
533             an empty list (NOTE: I C, which is a valid scalar), the
534             original deserialised hash will be inserted. This setting can slow down
535             decoding considerably.
536              
537             When C<$coderef> is omitted or undefined, any existing callback will
538             be removed and C will not change the deserialised hash in any
539             way.
540              
541             Example, convert all JSON objects into the integer 5:
542              
543             my $js = JSON::XS->new->filter_json_object (sub { 5 });
544             # returns [5]
545             $js->decode ('[{}]')
546             # throw an exception because allow_nonref is not enabled
547             # so a lone 5 is not allowed.
548             $js->decode ('{"a":1, "b":2}');
549              
550             =item $json = $json->filter_json_single_key_object ($key [=> $coderef->($value)])
551              
552             Works remotely similar to C, but is only called for
553             JSON objects having a single key named C<$key>.
554              
555             This C<$coderef> is called before the one specified via
556             C, if any. It gets passed the single value in the JSON
557             object. If it returns a single value, it will be inserted into the data
558             structure. If it returns nothing (not even C but the empty list),
559             the callback from C will be called next, as if no
560             single-key callback were specified.
561              
562             If C<$coderef> is omitted or undefined, the corresponding callback will be
563             disabled. There can only ever be one callback for a given key.
564              
565             As this callback gets called less often then the C
566             one, decoding speed will not usually suffer as much. Therefore, single-key
567             objects make excellent targets to serialise Perl objects into, especially
568             as single-key JSON objects are as close to the type-tagged value concept
569             as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not
570             support this in any way, so you need to make sure your data never looks
571             like a serialised Perl hash.
572              
573             Typical names for the single object key are C<__class_whatever__>, or
574             C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even
575             things like C<__class_md5sum(classname)__>, to reduce the risk of clashing
576             with real hashes.
577              
578             Example, decode JSON objects of the form C<< { "__widget__" => } >>
579             into the corresponding C<< $WIDGET{} >> object:
580              
581             # return whatever is in $WIDGET{5}:
582             JSON::XS
583             ->new
584             ->filter_json_single_key_object (__widget__ => sub {
585             $WIDGET{ $_[0] }
586             })
587             ->decode ('{"__widget__": 5')
588              
589             # this can be used with a TO_JSON method in some "widget" class
590             # for serialisation to json:
591             sub WidgetBase::TO_JSON {
592             my ($self) = @_;
593              
594             unless ($self->{id}) {
595             $self->{id} = ..get..some..id..;
596             $WIDGET{$self->{id}} = $self;
597             }
598              
599             { __widget__ => $self->{id} }
600             }
601              
602             =item $json = $json->shrink ([$enable])
603              
604             =item $enabled = $json->get_shrink
605              
606             Perl usually over-allocates memory a bit when allocating space for
607             strings. This flag optionally resizes strings generated by either
608             C or C to their minimum size possible. This can save
609             memory when your JSON texts are either very very long or you have many
610             short strings. It will also try to downgrade any strings to octet-form
611             if possible: perl stores strings internally either in an encoding called
612             UTF-X or in octet-form. The latter cannot store everything but uses less
613             space in general (and some buggy Perl or C code might even rely on that
614             internal representation being used).
615              
616             The actual definition of what shrink does might change in future versions,
617             but it will always try to save space at the expense of time.
618              
619             If C<$enable> is true (or missing), the string returned by C will
620             be shrunk-to-fit, while all strings generated by C will also be
621             shrunk-to-fit.
622              
623             If C<$enable> is false, then the normal perl allocation algorithms are used.
624             If you work with your data, then this is likely to be faster.
625              
626             In the future, this setting might control other things, such as converting
627             strings that look like integers or floats into integers or floats
628             internally (there is no difference on the Perl level), saving space.
629              
630             =item $json = $json->max_depth ([$maximum_nesting_depth])
631              
632             =item $max_depth = $json->get_max_depth
633              
634             Sets the maximum nesting level (default C<512>) accepted while encoding
635             or decoding. If a higher nesting level is detected in JSON text or a Perl
636             data structure, then the encoder and decoder will stop and croak at that
637             point.
638              
639             Nesting level is defined by number of hash- or arrayrefs that the encoder
640             needs to traverse to reach a given point or the number of C<{> or C<[>
641             characters without their matching closing parenthesis crossed to reach a
642             given character in a string.
643              
644             Setting the maximum depth to one disallows any nesting, so that ensures
645             that the object is only a single hash/object or array.
646              
647             If no argument is given, the highest possible setting will be used, which
648             is rarely useful.
649              
650             Note that nesting is implemented by recursion in C. The default value has
651             been chosen to be as large as typical operating systems allow without
652             crashing.
653              
654             See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
655              
656             =item $json = $json->max_size ([$maximum_string_size])
657              
658             =item $max_size = $json->get_max_size
659              
660             Set the maximum length a JSON text may have (in bytes) where decoding is
661             being attempted. The default is C<0>, meaning no limit. When C
662             is called on a string that is longer then this many bytes, it will not
663             attempt to decode the string but throw an exception. This setting has no
664             effect on C (yet).
665              
666             If no argument is given, the limit check will be deactivated (same as when
667             C<0> is specified).
668              
669             See SECURITY CONSIDERATIONS, below, for more info on why this is useful.
670              
671             =item $json_text = $json->encode ($perl_scalar)
672              
673             Converts the given Perl value or data structure to its JSON
674             representation. Croaks on error.
675              
676             =item $perl_scalar = $json->decode ($json_text)
677              
678             The opposite of C: expects a JSON text and tries to parse it,
679             returning the resulting simple scalar or reference. Croaks on error.
680              
681             =item ($perl_scalar, $characters) = $json->decode_prefix ($json_text)
682              
683             This works like the C method, but instead of raising an exception
684             when there is trailing garbage after the first JSON object, it will
685             silently stop parsing there and return the number of characters consumed
686             so far.
687              
688             This is useful if your JSON texts are not delimited by an outer protocol
689             and you need to know where the JSON text ends.
690              
691             JSON::XS->new->decode_prefix ("[1] the tail")
692             => ([], 3)
693              
694             =back
695              
696              
697             =head1 INCREMENTAL PARSING
698              
699             In some cases, there is the need for incremental parsing of JSON
700             texts. While this module always has to keep both JSON text and resulting
701             Perl data structure in memory at one time, it does allow you to parse a
702             JSON stream incrementally. It does so by accumulating text until it has
703             a full JSON object, which it then can decode. This process is similar to
704             using C to see if a full JSON object is available, but
705             is much more efficient (and can be implemented with a minimum of method
706             calls).
707              
708             JSON::XS will only attempt to parse the JSON text once it is sure it
709             has enough text to get a decisive result, using a very simple but
710             truly incremental parser. This means that it sometimes won't stop as
711             early as the full parser, for example, it doesn't detect mismatched
712             parentheses. The only thing it guarantees is that it starts decoding as
713             soon as a syntactically valid JSON text has been seen. This means you need
714             to set resource limits (e.g. C) to ensure the parser will stop
715             parsing in the presence if syntax errors.
716              
717             The following methods implement this incremental parser.
718              
719             =over 4
720              
721             =item [void, scalar or list context] = $json->incr_parse ([$string])
722              
723             This is the central parsing function. It can both append new text and
724             extract objects from the stream accumulated so far (both of these
725             functions are optional).
726              
727             If C<$string> is given, then this string is appended to the already
728             existing JSON fragment stored in the C<$json> object.
729              
730             After that, if the function is called in void context, it will simply
731             return without doing anything further. This can be used to add more text
732             in as many chunks as you want.
733              
734             If the method is called in scalar context, then it will try to extract
735             exactly I JSON object. If that is successful, it will return this
736             object, otherwise it will return C. If there is a parse error,
737             this method will croak just as C would do (one can then use
738             C to skip the erroneous part). This is the most common way of
739             using the method.
740              
741             And finally, in list context, it will try to extract as many objects
742             from the stream as it can find and return them, or the empty list
743             otherwise. For this to work, there must be no separators between the JSON
744             objects or arrays, instead they must be concatenated back-to-back. If
745             an error occurs, an exception will be raised as in the scalar context
746             case. Note that in this case, any previously-parsed JSON texts will be
747             lost.
748              
749             Example: Parse some JSON arrays/objects in a given string and return
750             them.
751              
752             my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]");
753              
754             =item $lvalue_string = $json->incr_text
755              
756             This method returns the currently stored JSON fragment as an lvalue, that
757             is, you can manipulate it. This I works when a preceding call to
758             C in I successfully returned an object. Under
759             all other circumstances you must not call this function (I mean it.
760             although in simple tests it might actually work, it I fail under
761             real world conditions). As a special exception, you can also call this
762             method before having parsed anything.
763              
764             This function is useful in two cases: a) finding the trailing text after a
765             JSON object or b) parsing multiple JSON objects separated by non-JSON text
766             (such as commas).
767              
768             =item $json->incr_skip
769              
770             This will reset the state of the incremental parser and will remove
771             the parsed text from the input buffer so far. This is useful after
772             C died, in which case the input buffer and incremental parser
773             state is left unchanged, to skip the text parsed so far and to reset the
774             parse state.
775              
776             The difference to C is that only text until the parse error
777             occurred is removed.
778              
779             =item $json->incr_reset
780              
781             This completely resets the incremental parser, that is, after this call,
782             it will be as if the parser had never parsed anything.
783              
784             This is useful if you want to repeatedly parse JSON objects and want to
785             ignore any trailing data, which means you have to reset the parser after
786             each successful decode.
787              
788             =back
789              
790             =head2 LIMITATIONS
791              
792             All options that affect decoding are supported, except
793             C. The reason for this is that it cannot be made to work
794             sensibly: JSON objects and arrays are self-delimited, i.e. you can
795             concatenate them back to back and still decode them perfectly. This does
796             not hold true for JSON numbers, however.
797              
798             For example, is the string C<1> a single JSON number, or is it simply the
799             start of C<12>? Or is C<12> a single JSON number, or the concatenation
800             of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS
801             takes the conservative route and disallows this case.
802              
803             =head2 EXAMPLES
804              
805             Some examples will make all this clearer. First, a simple example that
806             works similarly to C: We want to decode the JSON object at
807             the start of a string and identify the portion after the JSON object:
808              
809             my $text = "[1,2,3] hello";
810              
811             my $json = new JSON::XS;
812              
813             my $obj = $json->incr_parse ($text)
814             or die "expected JSON object or array at beginning of string";
815              
816             my $tail = $json->incr_text;
817             # $tail now contains " hello"
818              
819             Easy, isn't it?
820              
821             Now for a more complicated example: Imagine a hypothetical protocol where
822             you read some requests from a TCP stream, and each request is a JSON
823             array, without any separation between them (in fact, it is often useful to
824             use newlines as "separators", as these get interpreted as whitespace at
825             the start of the JSON text, which makes it possible to test said protocol
826             with C...).
827              
828             Here is how you'd do it (it is trivial to write this in an event-based
829             manner):
830              
831             my $json = new JSON::XS;
832              
833             # read some data from the socket
834             while (sysread $socket, my $buf, 4096) {
835              
836             # split and decode as many requests as possible
837             for my $request ($json->incr_parse ($buf)) {
838             # act on the $request
839             }
840             }
841              
842             Another complicated example: Assume you have a string with JSON objects
843             or arrays, all separated by (optional) comma characters (e.g. C<[1],[2],
844             [3]>). To parse them, we have to skip the commas between the JSON texts,
845             and here is where the lvalue-ness of C comes in useful:
846              
847             my $text = "[1],[2], [3]";
848             my $json = new JSON::XS;
849              
850             # void context, so no parsing done
851             $json->incr_parse ($text);
852              
853             # now extract as many objects as possible. note the
854             # use of scalar context so incr_text can be called.
855             while (my $obj = $json->incr_parse) {
856             # do something with $obj
857              
858             # now skip the optional comma
859             $json->incr_text =~ s/^ \s* , //x;
860             }
861              
862             Now lets go for a very complex example: Assume that you have a gigantic
863             JSON array-of-objects, many gigabytes in size, and you want to parse it,
864             but you cannot load it into memory fully (this has actually happened in
865             the real world :).
866              
867             Well, you lost, you have to implement your own JSON parser. But JSON::XS
868             can still help you: You implement a (very simple) array parser and let
869             JSON decode the array elements, which are all full JSON objects on their
870             own (this wouldn't work if the array elements could be JSON numbers, for
871             example):
872              
873             my $json = new JSON::XS;
874              
875             # open the monster
876             open my $fh, "
877             or die "bigfile: $!";
878              
879             # first parse the initial "["
880             for (;;) {
881             sysread $fh, my $buf, 65536
882             or die "read error: $!";
883             $json->incr_parse ($buf); # void context, so no parsing
884              
885             # Exit the loop once we found and removed(!) the initial "[".
886             # In essence, we are (ab-)using the $json object as a simple scalar
887             # we append data to.
888             last if $json->incr_text =~ s/^ \s* \[ //x;
889             }
890              
891             # now we have the skipped the initial "[", so continue
892             # parsing all the elements.
893             for (;;) {
894             # in this loop we read data until we got a single JSON object
895             for (;;) {
896             if (my $obj = $json->incr_parse) {
897             # do something with $obj
898             last;
899             }
900              
901             # add more data
902             sysread $fh, my $buf, 65536
903             or die "read error: $!";
904             $json->incr_parse ($buf); # void context, so no parsing
905             }
906              
907             # in this loop we read data until we either found and parsed the
908             # separating "," between elements, or the final "]"
909             for (;;) {
910             # first skip whitespace
911             $json->incr_text =~ s/^\s*//;
912              
913             # if we find "]", we are done
914             if ($json->incr_text =~ s/^\]//) {
915             print "finished.\n";
916             exit;
917             }
918              
919             # if we find ",", we can continue with the next element
920             if ($json->incr_text =~ s/^,//) {
921             last;
922             }
923              
924             # if we find anything else, we have a parse error!
925             if (length $json->incr_text) {
926             die "parse error near ", $json->incr_text;
927             }
928              
929             # else add more data
930             sysread $fh, my $buf, 65536
931             or die "read error: $!";
932             $json->incr_parse ($buf); # void context, so no parsing
933             }
934              
935             This is a complex example, but most of the complexity comes from the fact
936             that we are trying to be correct (bear with me if I am wrong, I never ran
937             the above example :).
938              
939              
940              
941             =head1 MAPPING
942              
943             This section describes how JSON::XS maps Perl values to JSON values and
944             vice versa. These mappings are designed to "do the right thing" in most
945             circumstances automatically, preserving round-tripping characteristics
946             (what you put in comes out as something equivalent).
947              
948             For the more enlightened: note that in the following descriptions,
949             lowercase I refers to the Perl interpreter, while uppercase I
950             refers to the abstract Perl language itself.
951              
952              
953             =head2 JSON -> PERL
954              
955             =over 4
956              
957             =item object
958              
959             A JSON object becomes a reference to a hash in Perl. No ordering of object
960             keys is preserved (JSON does not preserve object key ordering itself).
961              
962             =item array
963              
964             A JSON array becomes a reference to an array in Perl.
965              
966             =item string
967              
968             A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON
969             are represented by the same codepoints in the Perl string, so no manual
970             decoding is necessary.
971              
972             =item number
973              
974             A JSON number becomes either an integer, numeric (floating point) or
975             string scalar in perl, depending on its range and any fractional parts. On
976             the Perl level, there is no difference between those as Perl handles all
977             the conversion details, but an integer may take slightly less memory and
978             might represent more values exactly than floating point numbers.
979              
980             If the number consists of digits only, JSON::XS will try to represent
981             it as an integer value. If that fails, it will try to represent it as
982             a numeric (floating point) value if that is possible without loss of
983             precision. Otherwise it will preserve the number as a string value (in
984             which case you lose roundtripping ability, as the JSON number will be
985             re-encoded to a JSON string).
986              
987             Numbers containing a fractional or exponential part will always be
988             represented as numeric (floating point) values, possibly at a loss of
989             precision (in which case you might lose perfect roundtripping ability, but
990             the JSON number will still be re-encoded as a JSON number).
991              
992             Note that precision is not accuracy - binary floating point values cannot
993             represent most decimal fractions exactly, and when converting from and to
994             floating point, JSON::XS only guarantees precision up to but not including
995             the least significant bit.
996              
997             =item true, false
998              
999             These JSON atoms become C and
1000             C, respectively. They are overloaded to act
1001             almost exactly like the numbers C<1> and C<0>. You can check whether
1002             a scalar is a JSON boolean by using the C
1003             function (after C, of course).
1004              
1005             =item null
1006              
1007             A JSON null atom becomes C in Perl.
1008              
1009             =item shell-style comments (C<< # I >>)
1010              
1011             As a nonstandard extension to the JSON syntax that is enabled by the
1012             C setting, shell-style comments are allowed. They can start
1013             anywhere outside strings and go till the end of the line.
1014              
1015             =item tagged values (C<< (I)I >>).
1016              
1017             Another nonstandard extension to the JSON syntax, enabled with the
1018             C setting, are tagged values. In this implementation, the
1019             I must be a perl package/class name encoded as a JSON string, and the
1020             I must be a JSON array encoding optional constructor arguments.
1021              
1022             See L, below, for details.
1023              
1024             =back
1025              
1026              
1027             =head2 PERL -> JSON
1028              
1029             The mapping from Perl to JSON is slightly more difficult, as Perl is a
1030             truly typeless language, so we can only guess which JSON type is meant by
1031             a Perl value.
1032              
1033             =over 4
1034              
1035             =item hash references
1036              
1037             Perl hash references become JSON objects. As there is no inherent
1038             ordering in hash keys (or JSON objects), they will usually be encoded
1039             in a pseudo-random order. JSON::XS can optionally sort the hash keys
1040             (determined by the I flag), so the same datastructure will
1041             serialise to the same JSON text (given same settings and version of
1042             JSON::XS), but this incurs a runtime overhead and is only rarely useful,
1043             e.g. when you want to compare some JSON text against another for equality.
1044              
1045             =item array references
1046              
1047             Perl array references become JSON arrays.
1048              
1049             =item other references
1050              
1051             Other unblessed references are generally not allowed and will cause an
1052             exception to be thrown, except for references to the integers C<0> and
1053             C<1>, which get turned into C and C atoms in JSON.
1054              
1055             Since C uses the boolean model from L, you
1056             can also C and then use C
1057             and C to improve readability.
1058              
1059             use Types::Serialiser;
1060             encode_json [\0, Types::Serialiser::true] # yields [false,true]
1061              
1062             =item Types::Serialiser::true, Types::Serialiser::false
1063              
1064             These special values from the L module become JSON true
1065             and JSON false values, respectively. You can also use C<\1> and C<\0>
1066             directly if you want.
1067              
1068             =item blessed objects
1069              
1070             Blessed objects are not directly representable in JSON, but C
1071             allows various ways of handling objects. See L,
1072             below, for details.
1073              
1074             =item simple scalars
1075              
1076             Simple Perl scalars (any scalar that is not a reference) are the most
1077             difficult objects to encode: JSON::XS will encode undefined scalars as
1078             JSON C values, scalars that have last been used in a string context
1079             before encoding as JSON strings, and anything else as number value:
1080              
1081             # dump as number
1082             encode_json [2] # yields [2]
1083             encode_json [-3.0e17] # yields [-3e+17]
1084             my $value = 5; encode_json [$value] # yields [5]
1085              
1086             # used as string, so dump as string
1087             print $value;
1088             encode_json [$value] # yields ["5"]
1089              
1090             # undef becomes null
1091             encode_json [undef] # yields [null]
1092              
1093             You can force the type to be a JSON string by stringifying it:
1094              
1095             my $x = 3.1; # some variable containing a number
1096             "$x"; # stringified
1097             $x .= ""; # another, more awkward way to stringify
1098             print $x; # perl does it for you, too, quite often
1099              
1100             You can force the type to be a JSON number by numifying it:
1101              
1102             my $x = "3"; # some variable containing a string
1103             $x += 0; # numify it, ensuring it will be dumped as a number
1104             $x *= 1; # same thing, the choice is yours.
1105              
1106             You can not currently force the type in other, less obscure, ways. Tell me
1107             if you need this capability (but don't forget to explain why it's needed
1108             :).
1109              
1110             Note that numerical precision has the same meaning as under Perl (so
1111             binary to decimal conversion follows the same rules as in Perl, which
1112             can differ to other languages). Also, your perl interpreter might expose
1113             extensions to the floating point numbers of your platform, such as
1114             infinities or NaN's - these cannot be represented in JSON, and it is an
1115             error to pass those in.
1116              
1117             =back
1118              
1119             =head2 OBJECT SERIALISATION
1120              
1121             As JSON cannot directly represent Perl objects, you have to choose between
1122             a pure JSON representation (without the ability to deserialise the object
1123             automatically again), and a nonstandard extension to the JSON syntax,
1124             tagged values.
1125              
1126             =head3 SERIALISATION
1127              
1128             What happens when C encounters a Perl object depends on the
1129             C, C and C settings, which are
1130             used in this order:
1131              
1132             =over 4
1133              
1134             =item 1. C is enabled and the object has a C method.
1135              
1136             In this case, C uses the L object
1137             serialisation protocol to create a tagged JSON value, using a nonstandard
1138             extension to the JSON syntax.
1139              
1140             This works by invoking the C method on the object, with the first
1141             argument being the object to serialise, and the second argument being the
1142             constant string C to distinguish it from other serialisers.
1143              
1144             The C method can return any number of values (i.e. zero or
1145             more). These values and the paclkage/classname of the object will then be
1146             encoded as a tagged JSON value in the following format:
1147              
1148             ("classname")[FREEZE return values...]
1149              
1150             e.g.:
1151              
1152             ("URI")["http://www.google.com/"]
1153             ("MyDate")[2013,10,29]
1154             ("ImageData::JPEG")["Z3...VlCg=="]
1155              
1156             For example, the hypothetical C C method might use the
1157             objects C and C members to encode the object:
1158              
1159             sub My::Object::FREEZE {
1160             my ($self, $serialiser) = @_;
1161              
1162             ($self->{type}, $self->{id})
1163             }
1164              
1165             =item 2. C is enabled and the object has a C method.
1166              
1167             In this case, the C method of the object is invoked in scalar
1168             context. It must return a single scalar that can be directly encoded into
1169             JSON. This scalar replaces the object in the JSON text.
1170              
1171             For example, the following C method will convert all L
1172             objects to JSON strings when serialised. The fatc that these values
1173             originally were L objects is lost.
1174              
1175             sub URI::TO_JSON {
1176             my ($uri) = @_;
1177             $uri->as_string
1178             }
1179              
1180             =item 3. C is enabled.
1181              
1182             The object will be serialised as a JSON null value.
1183              
1184             =item 4. none of the above
1185              
1186             If none of the settings are enabled or the respective methods are missing,
1187             C throws an exception.
1188              
1189             =back
1190              
1191             =head3 DESERIALISATION
1192              
1193             For deserialisation there are only two cases to consider: either
1194             nonstandard tagging was used, in which case C decides,
1195             or objects cannot be automatically be deserialised, in which
1196             case you can use postprocessing or the C or
1197             C callbacks to get some real objects our of
1198             your JSON.
1199              
1200             This section only considers the tagged value case: I a tagged JSON object
1201             is encountered during decoding and C is disabled, a parse
1202             error will result (as if tagged values were not part of the grammar).
1203              
1204             If C is enabled, C will look up the C method
1205             of the package/classname used during serialisation (it will not attempt
1206             to load the package as a Perl module). If there is no such method, the
1207             decoding will fail with an error.
1208              
1209             Otherwise, the C method is invoked with the classname as first
1210             argument, the constant string C as second argument, and all the
1211             values from the JSON array (the values originally returned by the
1212             C method) as remaining arguments.
1213              
1214             The method must then return the object. While technically you can return
1215             any Perl scalar, you might have to enable the C setting to
1216             make that work in all cases, so better return an actual blessed reference.
1217              
1218             As an example, let's implement a C function that regenerates the
1219             C from the C example earlier:
1220              
1221             sub My::Object::THAW {
1222             my ($class, $serialiser, $type, $id) = @_;
1223              
1224             $class->new (type => $type, id => $id)
1225             }
1226              
1227              
1228             =head1 ENCODING/CODESET FLAG NOTES
1229              
1230             The interested reader might have seen a number of flags that signify
1231             encodings or codesets - C, C and C. There seems to be
1232             some confusion on what these do, so here is a short comparison:
1233              
1234             C controls whether the JSON text created by C (and expected
1235             by C) is UTF-8 encoded or not, while C and C only
1236             control whether C escapes character values outside their respective
1237             codeset range. Neither of these flags conflict with each other, although
1238             some combinations make less sense than others.
1239              
1240             Care has been taken to make all flags symmetrical with respect to
1241             C and C, that is, texts encoded with any combination of
1242             these flag values will be correctly decoded when the same flags are used
1243             - in general, if you use different flag settings while encoding vs. when
1244             decoding you likely have a bug somewhere.
1245              
1246             Below comes a verbose discussion of these flags. Note that a "codeset" is
1247             simply an abstract set of character-codepoint pairs, while an encoding
1248             takes those codepoint numbers and I them, in our case into
1249             octets. Unicode is (among other things) a codeset, UTF-8 is an encoding,
1250             and ISO-8859-1 (= latin 1) and ASCII are both codesets I encodings at
1251             the same time, which can be confusing.
1252              
1253             =over 4
1254              
1255             =item C flag disabled
1256              
1257             When C is disabled (the default), then C/C generate
1258             and expect Unicode strings, that is, characters with high ordinal Unicode
1259             values (> 255) will be encoded as such characters, and likewise such
1260             characters are decoded as-is, no changes to them will be done, except
1261             "(re-)interpreting" them as Unicode codepoints or Unicode characters,
1262             respectively (to Perl, these are the same thing in strings unless you do
1263             funny/weird/dumb stuff).
1264              
1265             This is useful when you want to do the encoding yourself (e.g. when you
1266             want to have UTF-16 encoded JSON texts) or when some other layer does
1267             the encoding for you (for example, when printing to a terminal using a
1268             filehandle that transparently encodes to UTF-8 you certainly do NOT want
1269             to UTF-8 encode your data first and have Perl encode it another time).
1270              
1271             =item C flag enabled
1272              
1273             If the C-flag is enabled, C/C will encode all
1274             characters using the corresponding UTF-8 multi-byte sequence, and will
1275             expect your input strings to be encoded as UTF-8, that is, no "character"
1276             of the input string must have any value > 255, as UTF-8 does not allow
1277             that.
1278              
1279             The C flag therefore switches between two modes: disabled means you
1280             will get a Unicode string in Perl, enabled means you get an UTF-8 encoded
1281             octet/binary string in Perl.
1282              
1283             =item C or C flags enabled
1284              
1285             With C (or C) enabled, C will escape characters
1286             with ordinal values > 255 (> 127 with C) and encode the remaining
1287             characters as specified by the C flag.
1288              
1289             If C is disabled, then the result is also correctly encoded in those
1290             character sets (as both are proper subsets of Unicode, meaning that a
1291             Unicode string with all character values < 256 is the same thing as a
1292             ISO-8859-1 string, and a Unicode string with all character values < 128 is
1293             the same thing as an ASCII string in Perl).
1294              
1295             If C is enabled, you still get a correct UTF-8-encoded string,
1296             regardless of these flags, just some more characters will be escaped using
1297             C<\uXXXX> then before.
1298              
1299             Note that ISO-8859-1-I strings are not compatible with UTF-8
1300             encoding, while ASCII-encoded strings are. That is because the ISO-8859-1
1301             encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I being
1302             a subset of Unicode), while ASCII is.
1303              
1304             Surprisingly, C will ignore these flags and so treat all input
1305             values as governed by the C flag. If it is disabled, this allows you
1306             to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of
1307             Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings.
1308              
1309             So neither C nor C are incompatible with the C flag -
1310             they only govern when the JSON output engine escapes a character or not.
1311              
1312             The main use for C is to relatively efficiently store binary data
1313             as JSON, at the expense of breaking compatibility with most JSON decoders.
1314              
1315             The main use for C is to force the output to not contain characters
1316             with values > 127, which means you can interpret the resulting string
1317             as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and
1318             8-bit-encoding, and still get the same data structure back. This is useful
1319             when your channel for JSON transfer is not 8-bit clean or the encoding
1320             might be mangled in between (e.g. in mail), and works because ASCII is a
1321             proper subset of most 8-bit and multibyte encodings in use in the world.
1322              
1323             =back
1324              
1325              
1326             =head2 JSON and ECMAscript
1327              
1328             JSON syntax is based on how literals are represented in javascript (the
1329             not-standardised predecessor of ECMAscript) which is presumably why it is
1330             called "JavaScript Object Notation".
1331              
1332             However, JSON is not a subset (and also not a superset of course) of
1333             ECMAscript (the standard) or javascript (whatever browsers actually
1334             implement).
1335              
1336             If you want to use javascript's C function to "parse" JSON, you
1337             might run into parse errors for valid JSON texts, or the resulting data
1338             structure might not be queryable:
1339              
1340             One of the problems is that U+2028 and U+2029 are valid characters inside
1341             JSON strings, but are not allowed in ECMAscript string literals, so the
1342             following Perl fragment will not output something that can be guaranteed
1343             to be parsable by javascript's C:
1344              
1345             use JSON::XS;
1346              
1347             print encode_json [chr 0x2028];
1348              
1349             The right fix for this is to use a proper JSON parser in your javascript
1350             programs, and not rely on C (see for example Douglas Crockford's
1351             F parser).
1352              
1353             If this is not an option, you can, as a stop-gap measure, simply encode to
1354             ASCII-only JSON:
1355              
1356             use JSON::XS;
1357              
1358             print JSON::XS->new->ascii->encode ([chr 0x2028]);
1359              
1360             Note that this will enlarge the resulting JSON text quite a bit if you
1361             have many non-ASCII characters. You might be tempted to run some regexes
1362             to only escape U+2028 and U+2029, e.g.:
1363              
1364             # DO NOT USE THIS!
1365             my $json = JSON::XS->new->utf8->encode ([chr 0x2028]);
1366             $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028
1367             $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029
1368             print $json;
1369              
1370             Note that I: the above only works for U+2028 and
1371             U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing
1372             javascript implementations, however, have issues with other characters as
1373             well - using C naively simply I cause problems.
1374              
1375             Another problem is that some javascript implementations reserve
1376             some property names for their own purposes (which probably makes
1377             them non-ECMAscript-compliant). For example, Iceweasel reserves the
1378             C<__proto__> property name for its own purposes.
1379              
1380             If that is a problem, you could parse try to filter the resulting JSON
1381             output for these property strings, e.g.:
1382              
1383             $json =~ s/"__proto__"\s*:/"__proto__renamed":/g;
1384              
1385             This works because C<__proto__> is not valid outside of strings, so every
1386             occurrence of C<"__proto__"\s*:> must be a string used as property name.
1387              
1388             If you know of other incompatibilities, please let me know.
1389              
1390              
1391             =head2 JSON and YAML
1392              
1393             You often hear that JSON is a subset of YAML. This is, however, a mass
1394             hysteria(*) and very far from the truth (as of the time of this writing),
1395             so let me state it clearly: I
1396             JSON::XS to output a data structure as valid YAML> that works in all
1397             cases.
1398              
1399             If you really must use JSON::XS to generate YAML, you should use this
1400             algorithm (subject to change in future versions):
1401              
1402             my $to_yaml = JSON::XS->new->utf8->space_after (1);
1403             my $yaml = $to_yaml->encode ($ref) . "\n";
1404              
1405             This will I generate JSON texts that also parse as valid
1406             YAML. Please note that YAML has hardcoded limits on (simple) object key
1407             lengths that JSON doesn't have and also has different and incompatible
1408             unicode character escape syntax, so you should make sure that your hash
1409             keys are noticeably shorter than the 1024 "stream characters" YAML allows
1410             and that you do not have characters with codepoint values outside the
1411             Unicode BMP (basic multilingual page). YAML also does not allow C<\/>
1412             sequences in strings (which JSON::XS does not I generate, but
1413             other JSON generators might).
1414              
1415             There might be other incompatibilities that I am not aware of (or the YAML
1416             specification has been changed yet again - it does so quite often). In
1417             general you should not try to generate YAML with a JSON generator or vice
1418             versa, or try to parse JSON with a YAML parser or vice versa: chances are
1419             high that you will run into severe interoperability problems when you
1420             least expect it.
1421              
1422             =over 4
1423              
1424             =item (*)
1425              
1426             I have been pressured multiple times by Brian Ingerson (one of the
1427             authors of the YAML specification) to remove this paragraph, despite him
1428             acknowledging that the actual incompatibilities exist. As I was personally
1429             bitten by this "JSON is YAML" lie, I refused and said I will continue to
1430             educate people about these issues, so others do not run into the same
1431             problem again and again. After this, Brian called me a (quote)I
1432             and worthless idiot>(unquote).
1433              
1434             In my opinion, instead of pressuring and insulting people who actually
1435             clarify issues with YAML and the wrong statements of some of its
1436             proponents, I would kindly suggest reading the JSON spec (which is not
1437             that difficult or long) and finally make YAML compatible to it, and
1438             educating users about the changes, instead of spreading lies about the
1439             real compatibility for many I and trying to silence people who
1440             point out that it isn't true.
1441              
1442             Addendum/2009: the YAML 1.2 spec is still incompatible with JSON, even
1443             though the incompatibilities have been documented (and are known to Brian)
1444             for many years and the spec makes explicit claims that YAML is a superset
1445             of JSON. It would be so easy to fix, but apparently, bullying people and
1446             corrupting userdata is so much easier.
1447              
1448             =back
1449              
1450              
1451             =head2 SPEED
1452              
1453             It seems that JSON::XS is surprisingly fast, as shown in the following
1454             tables. They have been generated with the help of the C program
1455             in the JSON::XS distribution, to make it easy to compare on your own
1456             system.
1457              
1458             First comes a comparison between various modules using
1459             a very short single-line JSON string (also available at
1460             L).
1461              
1462             {"method": "handleMessage", "params": ["user1",
1463             "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7,
1464             1, 0]}
1465              
1466             It shows the number of encodes/decodes per second (JSON::XS uses
1467             the functional interface, while JSON::XS/2 uses the OO interface
1468             with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables
1469             shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ
1470             uses the from_json method). Higher is better:
1471              
1472             module | encode | decode |
1473             --------------|------------|------------|
1474             JSON::DWIW/DS | 86302.551 | 102300.098 |
1475             JSON::DWIW/FJ | 86302.551 | 75983.768 |
1476             JSON::PP | 15827.562 | 6638.658 |
1477             JSON::Syck | 63358.066 | 47662.545 |
1478             JSON::XS | 511500.488 | 511500.488 |
1479             JSON::XS/2 | 291271.111 | 388361.481 |
1480             JSON::XS/3 | 361577.931 | 361577.931 |
1481             Storable | 66788.280 | 265462.278 |
1482             --------------+------------+------------+
1483              
1484             That is, JSON::XS is almost six times faster than JSON::DWIW on encoding,
1485             about five times faster on decoding, and over thirty to seventy times
1486             faster than JSON's pure perl implementation. It also compares favourably
1487             to Storable for small amounts of data.
1488              
1489             Using a longer test string (roughly 18KB, generated from Yahoo! Locals
1490             search API (L).
1491              
1492             module | encode | decode |
1493             --------------|------------|------------|
1494             JSON::DWIW/DS | 1647.927 | 2673.916 |
1495             JSON::DWIW/FJ | 1630.249 | 2596.128 |
1496             JSON::PP | 400.640 | 62.311 |
1497             JSON::Syck | 1481.040 | 1524.869 |
1498             JSON::XS | 20661.596 | 9541.183 |
1499             JSON::XS/2 | 10683.403 | 9416.938 |
1500             JSON::XS/3 | 20661.596 | 9400.054 |
1501             Storable | 19765.806 | 10000.725 |
1502             --------------+------------+------------+
1503              
1504             Again, JSON::XS leads by far (except for Storable which non-surprisingly
1505             decodes a bit faster).
1506              
1507             On large strings containing lots of high Unicode characters, some modules
1508             (such as JSON::PC) seem to decode faster than JSON::XS, but the result
1509             will be broken due to missing (or wrong) Unicode handling. Others refuse
1510             to decode or encode properly, so it was impossible to prepare a fair
1511             comparison table for that case.
1512              
1513              
1514             =head1 SECURITY CONSIDERATIONS
1515              
1516             When you are using JSON in a protocol, talking to untrusted potentially
1517             hostile creatures requires relatively few measures.
1518              
1519             First of all, your JSON decoder should be secure, that is, should not have
1520             any buffer overflows. Obviously, this module should ensure that and I am
1521             trying hard on making that true, but you never know.
1522              
1523             Second, you need to avoid resource-starving attacks. That means you should
1524             limit the size of JSON texts you accept, or make sure then when your
1525             resources run out, that's just fine (e.g. by using a separate process that
1526             can crash safely). The size of a JSON text in octets or characters is
1527             usually a good indication of the size of the resources required to decode
1528             it into a Perl structure. While JSON::XS can check the size of the JSON
1529             text, it might be too late when you already have it in memory, so you
1530             might want to check the size before you accept the string.
1531              
1532             Third, JSON::XS recurses using the C stack when decoding objects and
1533             arrays. The C stack is a limited resource: for instance, on my amd64
1534             machine with 8MB of stack size I can decode around 180k nested arrays but
1535             only 14k nested JSON objects (due to perl itself recursing deeply on croak
1536             to free the temporary). If that is exceeded, the program crashes. To be
1537             conservative, the default nesting limit is set to 512. If your process
1538             has a smaller stack, you should adjust this setting accordingly with the
1539             C method.
1540              
1541             Something else could bomb you, too, that I forgot to think of. In that
1542             case, you get to keep the pieces. I am always open for hints, though...
1543              
1544             Also keep in mind that JSON::XS might leak contents of your Perl data
1545             structures in its error messages, so when you serialise sensitive
1546             information you might want to make sure that exceptions thrown by JSON::XS
1547             will not end up in front of untrusted eyes.
1548              
1549             If you are using JSON::XS to return packets to consumption
1550             by JavaScript scripts in a browser you should have a look at
1551             L to
1552             see whether you are vulnerable to some common attack vectors (which really
1553             are browser design bugs, but it is still you who will have to deal with
1554             it, as major browser developers care only for features, not about getting
1555             security right).
1556              
1557              
1558             =head1 INTEROPERABILITY WITH OTHER MODULES
1559              
1560             C uses the L module to provide boolean
1561             constants. That means that the JSON true and false values will be
1562             comaptible to true and false values of iother modules that do the same,
1563             such as L and L.
1564              
1565              
1566             =head1 THREADS
1567              
1568             This module is I guaranteed to be thread safe and there are no
1569             plans to change this until Perl gets thread support (as opposed to the
1570             horribly slow so-called "threads" which are simply slow and bloated
1571             process simulations - use fork, it's I faster, cheaper, better).
1572              
1573             (It might actually work, but you have been warned).
1574              
1575              
1576             =head1 THE PERILS OF SETLOCALE
1577              
1578             Sometimes people avoid the Perl locale support and directly call the
1579             system's setlocale function with C.
1580              
1581             This breaks both perl and modules such as JSON::XS, as stringification of
1582             numbers no longer works correctly (e.g. C<$x = 0.1; print "$x"+1> might
1583             print C<1>, and JSON::XS might output illegal JSON as JSON::XS relies on
1584             perl to stringify numbers).
1585              
1586             The solution is simple: don't call C, or use it for only those
1587             categories you need, such as C or C.
1588              
1589             If you need C, you should enable it only around the code that
1590             actually needs it (avoiding stringification of numbers), and restore it
1591             afterwards.
1592              
1593              
1594             =head1 BUGS
1595              
1596             While the goal of this module is to be correct, that unfortunately does
1597             not mean it's bug-free, only that I think its design is bug-free. If you
1598             keep reporting bugs they will be fixed swiftly, though.
1599              
1600             Please refrain from using rt.cpan.org or any other bug reporting
1601             service. I put the contact address into my modules for a reason.
1602              
1603             =cut
1604              
1605             BEGIN {
1606             *true = \$Types::Serialiser::true;
1607             *true = \&Types::Serialiser::true;
1608             *false = \$Types::Serialiser::false;
1609             *false = \&Types::Serialiser::false;
1610             *is_bool = \&Types::Serialiser::is_bool;
1611              
1612             *JSON::XS::Boolean:: = *Types::Serialiser::Boolean::;
1613             }
1614              
1615             XSLoader::load "JSON::XS", $VERSION;
1616              
1617             =head1 SEE ALSO
1618              
1619             The F command line utility for quick experiments.
1620              
1621             =head1 AUTHOR
1622              
1623             Marc Lehmann
1624             http://home.schmorp.de/
1625              
1626             =cut
1627              
1628             1
1629