blib/lib/SPVM/Document/Language/Tokenization.pm untested

filter: Statementstmt n/a Branchbran n/a Conditioncond n/a MC/DCmcdc n/a Subroutinesub n/a Podpod n/a SCAR -
1 =encoding utf8
2  
3 =head1 Name
4  
5 SPVM::Document::Language::Tokenization - Tokenization in the SPVM Language
6  
7 =head1 Description
8  
9 This document describes the tokenization in the SPVM language.
10  
11 =head1 Tokenization
12  
13 This section describes L<lexical analysis|https://en.wikipedia.org/wiki/Lexical_analysis> in the SPVM Language.
14  
15 This is called tokenization.
16  
17 See L<SPVM::Document::Language::SyntaxParsing> about syntax parsing.
18  
19 =head2 Character Encoding
20  
21 The character encoding of SPVM source codes is UTF-8.
22  
23 If a character is an ASCII character, it must be an ASCII printable character or a L<space character|/"Space Characters">.
24  
25 Compilation Errors:
26  
27 The charactor encoding of SPVM source codes must be UTF-8. Otherwise a compilation error occurs.
28  
29 If a character is an ASCII character, it must be an L<ASCII printable character|https://en.wikipedia.org/wiki/ASCII#Printable_characters> or a L<space character|/"Space Characters">. Otherwise a compilation error occurs.
30  
31 =head2 Line Terminators
32  
33 The line terminator is ASCII C<LF>.
34  
35 When a line terminator appears, the current line number is incremented by 1.
36  
37 =head2 Space Characters
38  
39 The space characters are ASCII C<SP>, C<HT>, C<FF>, C<LF>.
40  
41 =head2 Word Characters
42  
43 The word characters are ASCII C<a-zA-Z>, C<0-9>, C<_>.
44  
45 =head2 Names
46  
47 This section describes names.
48  
49 =head3 Symbol Name
50  
51 A symbol name consists of L<word characters|/"Word Characters"> and C<::>.
52  
53 It dose not contains C<__>.
54  
55 It dose not begin with C<0-9>.
56  
57 It dose not begin with C<::>.
58  
59 It dose not end with C<::>.
60  
61 It dose not contains C<::::>.
62  
63 Compliation Errors:
64  
65 If a symbol name is invald, a compilation error occurs.
66  
67 Examples:
68  
69 # Symbol names
70 foo
71 foo_bar2
72 Foo::Bar
73
74 # Invalid symbol names
75 2foo
76 foo__bar
77 ::Foo
78 Foo::
79 Foo::::Bar
80  
81 =head3 Class Name
82  
83 A class name is a L<symbol name|/"Symbol Name">.
84  
85 Each partial name of a class name must begin with an uppercase letter.
86  
87 Partial names are individual names separated by C<::>. For example, the partial names of C<Foo::Bar::Baz> are C<Foo>, C<Bar>, and C<Baz>.
88  
89 Compilation Errors:
90  
91 If a class name is invalid, a compilation error occurs.
92  
93 Examples:
94
95 # Class names
96 Foo
97 Foo::Bar
98 Foo::Bar::Baz3
99 Foo::bar
100 Foo_Bar::Baz_Baz
101
102 # Invalid class names
103 Foo
104 Foo::::Bar
105 Foo::Bar::
106 Foo__Bar
107 Foo::bar
108  
109 =head3 Method Name
110  
111 A method name is a L<symbol name|/"Symbol Name"> without C<::> or an empty string C<"">.
112  
113 Method names with the same name as L<keywords|/"Keywords"> are allowed.
114  
115 Compilation Errors:
116  
117 If a method name is invalid, a compilation error occurs.
118  
119 Examples:
120  
121 # Method names
122 FOO
123 FOO_BAR3
124 foo
125 foo_bar
126 _foo
127 _foo_bar_
128
129 # Invalid method names
130 foo__bar
131 3foo
132  
133 =head3 Field Name
134  
135 A field name is a L<symbol name|/"Symbol Name"> without C<::>.
136  
137 Field names with the same name as L<keywords|/"Keywords"> are allowed.
138  
139 Compilation Errors:
140  
141 If a field names is invalid, a compilation error occurs.
142  
143 Examples:
144  
145 # Field names
146 FOO
147 FOO_BAR3
148 foo
149 foo_bar
150 _foo
151 _foo_bar_
152
153 # Invalid field names
154 foo__bar
155 3foo
156 Foo::Bar
157  
158 =head3 Variable Name
159  
160 A variable name begins with C<$> and is followed by a L<symbol name|/"Symbol Name">.
161  
162 The symbol name in a variable name can be surrounded by C<{> and C<}>.
163  
164 Compilation Errors:
165  
166 If a field names is invalid, a compilation error occurs.
167  
168 If an opening C<{> exists and the closing C<}> dose not exist, a compilation error occurs.
169  
170 Examples:
171  
172 # Variable names
173 $name
174 $my_name
175 ${name}
176 $Foo::name
177 $Foo::Bar::name
178 ${Foo::name}
179
180 # Invalid variable names
181 $::name
182 $name::
183 $Foo::::name
184 $my__name
185 ${name
186  
187 =head4 Class Variable Name
188  
189 A class variable name is a L<variable name|/"Variable Name">.
190  
191 Examples:
192  
193 # Class variable names
194 $NAME
195 $MY_NAME
196 ${NAME}
197 $FOO::NAME
198 $FOO::BAR::NAME
199 ${FOO::NAME_BRACE}
200 $FOO::name
201
202 # Invalid class variable names
203 $::NAME
204 $NAME::
205 $FOO::::NAME
206 $MY__NAME
207 $3FOO
208 ${NAME
209  
210 =head4 Local Variable Name
211  
212 A local variable name is a L<variable name|/"Variable Name"> without C<::>.
213  
214 Examples:
215  
216 # Local variable names
217 $name
218 $my_name
219 ${name_brace}
220 $_name
221 $NAME
222  
223 # Invalid local variable names
224 $::name
225 $name::
226 $Foo::name
227 $Foo::::name
228 $my__name
229 ${name
230 $3foo
231  
232 =head2 Keywords
233  
234 The List of Keywords:
235  
236 alias
237 allow
238 as
239 basic_type_id
240 break
241 byte
242 can
243 case
244 cmp
245 class
246 compile_type_name
247 copy
248 default
249 die
250 div_uint
251 div_ulong
252 double
253 dump
254 elsif
255 else
256 enum
257 eq
258 eval
259 eval_error_id
260 extends
261 for
262 float
263 false
264 gt
265 ge
266 has
267 if
268 interface
269 int
270 interface_t
271 isa
272 isa_error
273 isweak
274 is_compile_type
275 is_type
276 is_error
277 is_read_only
278 args_width
279 last
280 length
281 lt
282 le
283 long
284 make_read_only
285 my
286 mulnum_t
287 method
288 mod_uint
289 mod_ulong
290 mutable
291 native
292 ne
293 next
294 new
295 new_string_len
296 of
297 our
298 object
299 print
300 private
301 protected
302 public
303 precompile
304 pointer
305 return
306 require
307 required
308 rw
309 ro
310 say
311 static
312 switch
313 string
314 short
315 scalar
316 true
317 type_name
318 undef
319 unless
320 unweaken
321 use
322 version
323 void
324 warn
325 while
326 weaken
327 wo
328 INIT
329 __END__
330 __PACKAGE__
331 __FILE__
332 __LINE__
333  
334 =head2 Operator Tokens
335  
336 The List of Operator Tokens:
337  
338 !
339 !=
340 $
341 %
342 &
343 &&
344 &=
345 =
346 ==
347 ^
348 ^=
349 |
350 ||
351 |=
352 -
353 --
354 -=
355 ~
356 @
357 +
358 ++
359 +=
360 *
361 *=
362 <
363 <=
364 >
365 >=
366 <=>
367 %
368 %=
369 <<
370 <<=
371 >>=
372 >>
373 >>>
374 >>>=
375 .
376 .=
377 /
378 /=
379 \
380 (
381 )
382 {
383 }
384 [
385 ]
386 ;
387 :
388 ,
389 ->
390 =>
391  
392 =head2 Comment
393  
394 Comments have no meaning.
395  
396 #COMMENT
397  
398 A comment begins with C<#>.
399  
400 It is followed by any string I<COMMENT>.
401  
402 It ends with ASCII C<LF>.
403  
404 L<Line directives|/"Line Directive"> take precedence over comments.
405  
406 L<File directives|/"File Directive"> take precedence over comments.
407  
408 Examples:
409  
410 # This is a comment line
411  
412 =head2 line Directive
413  
414 A line directive set the current line number.
415  
416 #line NUMBER
417  
418 A line directive begins with C<#line> from the beggining of the line.
419  
420 It is followed by one or more ASCII C<SP>.
421  
422 It is followed by I<NUMBER>. I<NUMBER> is a positive 32bit integer.
423  
424 It ends with ASCII C<LF>.
425  
426 The current line number of the source code is set to I<NUMBER>.
427  
428 Line directives take precedence over L<comments|/"Comment">.
429  
430 Compilation Errors:
431  
432 A line directive must begin from the beggining of the line. Otherwise an compilation error occurs.
433  
434 A line directive must end with "\n". Otherwise an compilation error occurs.
435  
436 A line directive must have a line number. Otherwise an compilation error occurs.
437  
438 The line number given to a line directive must be a positive 32bit integer. Otherwise an compilation error occurs.
439  
440 Examples:
441  
442 class MyClass {
443
444 static method main : void () {
445
446 #line 39
447
448 }
449 }
450  
451 =head2 file Directive
452  
453 A file directive set the current file path.
454  
455 #file "FILE_PATH"
456
457 A file directive begins from the beggining of the source code excluding a shebang line.
458  
459 A shebang line before a file directive is allowed.
460  
461 #!command
462 #file "FILE_PATH"
463  
464 It is followed by one or more ASCII C<SP>.
465  
466 It is followed by C<">.
467  
468 It is followed by I<FILE_PATH>. I<FILE_PATH> is a string that represetns a file path.
469  
470 It is closed with C<">.
471  
472 It ends with ASCII C<LF>.
473  
474 The current file path is set to I<FILE_PATH>.
475  
476 File directives take precedence over L<comments|/"Comment">.
477  
478 Compilation Errors:
479  
480 A file directive must begin from the beggining of the source code. Otherwise an compilation error occurs.
481  
482 A file directive must end with "\n". Otherwise an compilation error occurs.
483  
484 A file directive must have a file path. Otherwise an compilation error occurs.
485  
486 A file directive must end with ". Otherwise an compilation error occurs.
487  
488 Examples:
489  
490 #file "/path/MyClass.spvm"
491 class MyClass {
492
493 }
494  
495 =head2 lib Directive
496  
497 A lib directive gives a hint for a class search directory to L<spvm> command and L<spvmcc> command.
498  
499 #lib "CLASS_SEARCH_DIRECTORY"
500
501 A lib directive begins from the beggining of a line.
502  
503 It is followed by one or more ASCII C<SP>.
504  
505 It is followed by C<">.
506  
507 It is followed by I<CLASS_SEARCH_DIRECTORY>. I<CLASS_SEARCH_DIRECTORY> is a string that represetns a L<class search directory|SPVM::Document::Language::Class/"Class Search Directories">.
508  
509 It is closed with C<">.
510  
511 It ends with ASCII C<LF>.
512  
513 The line directives take precedence over L<comments|/"Comment">.
514  
515 I<CLASS_SEARCH_DIRECTORY> can contains C<$FindBin::Bin>. This is expaned to the directory where the SPVM script is placed.
516  
517 #lib "$FindBin::Bin/lib/SPVM"
518  
519 Compilation Errors:
520  
521 A lib directive must begin from the beggining of a line. Otherwise an compilation error occurs.
522  
523 The directory specified by a lib directive end with "\n". Otherwise an compilation error occurs.
524  
525 The directory specified by a lib directive must not be an empty string. Otherwise an compilation error occurs.
526  
527 The directory specified by a lib directive must end with ". Otherwise an compilation error occurs.
528  
529 Examples:
530  
531 C<my_script.spvm>:
532  
533 #lib "$FindBin::Bin/lib/SPVM"
534
535 class {
536
537 }
538  
539 =head2 __END__
540  
541 If a line begins with C<__END__> and ends with ASCII C<LF>, the line with C<__END__> and the below lines are interpreted as L<comments|/"Comment">.
542  
543 Examples:
544
545 class MyClass {
546
547 }
548
549 __END__
550
551 foo
552 bar
553  
554 =head2 POD
555  
556 POD is a syntax to write multiline comment. POD has no meaning.
557  
558 The Beginning of a POD:
559  
560 =NAME
561  
562 The beginning of a POD begins with C<=> from the beggining of the line.
563  
564 It is followed by I<NAME>. I<NAME> is any string that begins with ASCII C<a-zA-Z>.
565  
566 It ends with ASCII C<LF>.
567  
568 The End of a POD:
569  
570 =cut
571  
572 The end of a POD begins with C<=> from the beggining of the line.
573  
574 It is followed by C<cut>.
575  
576 It ends with ASCII C<LF>.
577  
578 Examples:
579  
580
581 =pod
582
583 Comment1
584 Comment2
585
586 =cut
587
588 =head1
589
590 Comment1
591 Comment2
592
593 =cut
594  
595 =head2 Fat Comma
596  
597 A fat comma is
598  
599 =>
600  
601 The fat comma is an alias for a comma C<,>.
602  
603 # Comma
604 ["a", "b", "c", "d"]
605
606 # Fat Comma
607 ["a" => "b", "c" => "d"]
608  
609 If the left operand of a fat comma is a L<symbol name|/"Symbol Name"> without C<::>, it is wrraped by C<"> and is treated as a L<string literal|/"String Literal">.
610  
611 # foo_bar2 is treated as "foo_bar2"
612 [foo_bar2 => "Mark"]
613
614 ["foo_bar2" => "Mark"]
615  
616 =head1 Literals
617  
618 A literal represents a constant value.
619  
620 =head2 Numeric Literals
621  
622 A numeric literal represents a constant L<number|SPVM::Document::Language::Types/"Number">.
623  
624 =head2 Integer Literals
625  
626 A interger literal represents a constant number of an L<integer type|SPVM::Document::Language::Types/"Integer Types">.
627  
628 =head3 Integer Literal Decimal Notation
629  
630 The interger literal decimal notation represents a number of int type or long type using decimal numbers C<0-9>.
631
632 It can begin with a minus C<->.
633  
634 It is followed by one or more of C<0-9>.
635  
636 C<_> can be placed at the any positions after the first C<0-9> as a separator. C<_> has no meaning.
637  
638 It can end with the suffix C<L> or C<l>.
639  
640 If the suffix C<L> or C<l> exists, the return type is long type. Otherwise the return type is int type.
641  
642 Compilation Errors:
643  
644 If the return type is int type and the value is greater than the max value of int type or less than the minimal value of int type, a compilation error occurs.
645  
646 If the return type is long type and the value is greater than the max value of long type or less than the minimal value of long type, a compilation error occurs.
647  
648 Examples:
649  
650 123
651 -123
652 123L
653 123l
654 123_456_789
655 -123_456_789L
656  
657 =head3 Integer Literal Hexadecimal Notation
658  
659 The interger literal hexadecimal notation represents a number of int type or long type using hexadecimal numbers C<0-9a-zA-Z>.
660  
661 It can begin with a minus C<->.
662  
663 It is followed by C<0x> or C<0X>.
664  
665 It is followed by one or more C<0-9a-zA-Z>. This is called hexadecimal numbers part.
666  
667 C<_> can be placed at the any positions after C<0x> or C<0X> as a separator. C<_> has no meaning.
668  
669 It can end with the suffix C<L> or C<l>.
670  
671 If the suffix C<L> or C<l> exists, the return type is long type. Otherwise the return type is int type.
672  
673 If the return type is int type, the hexadecimal numbers part is interpreted as an unsigned 32 bit integer, and is converted to a signed 32-bit integer without changing the bits. For example, C<0xFFFFFFFF> is -1.
674  
675 If the return type is long type, the hexadecimal numbers part is interpreted as unsigned 64 bit integer, and is converted to a signed 64-bit integer without changing the bits. For example, C<0xFFFFFFFFFFFFFFFFL> is C<-1L>.
676  
677 Compilation Errors:
678  
679 If the return type is int type and the hexadecimal numbers part is greater than hexadecimal C<FFFFFFFF>, a compilation error occurs.
680  
681 If the return type is long type and the hexadecimal numbers part is greater than hexadecimal C<FFFFFFFFFFFFFFFF>, a compilation error occurs.
682  
683 Examples:
684  
685 0x3b4f
686 0X3b4f
687 -0x3F1A
688 0xDeL
689 0xFFFFFFFF
690 0xFF_FF_FF_FF
691 0xFFFFFFFFFFFFFFFFL
692  
693 =head3 Integer Literal Octal Notation
694  
695 The interger literal octal notation represents a number of int type or long type using octal numbers C<0-7>.
696  
697 It can begin with a minus C<->.
698  
699 It is followed by C<0>.
700  
701 It is followed by one or more C<0-7>. This is called octal numbers part.
702  
703 C<_> can be placed at the any positions after C<0> as a separator. C<_> has no meaning.
704  
705 It can end with the suffix C<L> or C<l>.
706  
707 If the suffix C<L> or C<l> exists, the return type is long type. Otherwise the return type is int type.
708  
709 If the return type is int type, the octal numbers part is interpreted as an unsigned 32 bit integer, and is converted to a signed 32-bit integer without changing the bits. For example, C<037777777777> is -1.
710  
711 If the return type is long type, the octal numbers part is interpreted as unsigned 64 bit integer, and is converted to a signed 64-bit integer without changing the bits. For example, C<01777777777777777777777L> is C<-1L>.
712  
713 If the return type is long type, the value that is except for C<-> is interpreted as unsigned 64 bit integer C<uint64_t> type in the C language, and the following conversion is performed.
714  
715 Compilation Errors:
716  
717 If the return type is int type and the octal numbers part is greater than octal 37777777777, a compilation error occurs.
718  
719 If the return type is long type and the octal numbers part is greater than octal 1777777777777777777777, a compilation error occurs.
720  
721 Examples:
722  
723 0755
724 -0644
725 0666L
726 0655_755
727  
728 =head3 Integer Literal Binary Notation
729  
730 The interger literal binary notation represents a number of int type or long type using binary numbers C<0> and C<1>.
731  
732 It can begin with a minus C<->.
733  
734 It is followed by C<0b> or C<0B>.
735  
736 It is followed by one or more C<0> and C<1>. This is called binary numbers part.
737  
738 C<_> can be placed at the any positions after C<0b> or C<0B> as a separator. C<_> has no meaning.
739  
740 It can end with the suffix C<L> or C<l>.
741  
742 If the suffix C<L> or C<l> exists, the return type is long type. Otherwise the return type is int type.
743  
744 If the return type is int type, the binary numbers part is interpreted as an unsigned 32 bit integer, and is converted to a signed 32-bit integer without changing the bits. For example, C<0b11111111111111111111111111111111> is -1.
745  
746 If the return type is long type, the binary numbers part is interpreted as unsigned 64 bit integer, and is converted to a signed 64-bit integer without changing the bits. For example, C<0b1111111111111111111111111111111111111111111111111111111111111111L> is C<-1L>.
747  
748 Compilation Errors:
749  
750 If the return type is int type and the value that is except for C<-> is greater than binary C<11111111111111111111111111111111>, a compilation error occurs.
751  
752 If the return type is long type and the value that is except for C<-> is greater than binary C<1111111111111111111111111111111111111111111111111111111111111111>, a compilation error occurs.
753  
754 Examples:
755  
756 0b0101
757 -0b1010
758 0b110000L
759 0b10101010_10101010
760  
761 =head2 Floating Point Literals
762  
763 The floating point litral represetns a floating point number.
764  
765 =head3 Floating Point Literal Decimal Notation
766  
767 The floating point litral decimal notation represents a number of float type and double type using decimal numbers C<0-9>.
768  
769 It can begin with a minus C<->.
770  
771 It is followed by one or more C<0-9>.
772  
773 C<_> can be placed at the any positions after the first C<0-9>.
774  
775 It can be followed by a floating point part, an exponent part, or a combination of a floating point part and an exponent part.
776  
777 [Floating Point Part Begin]
778  
779 A floating point part begins with C<.>.
780  
781 It is followed by one or more C<0-9>.
782  
783 [Floating Point Part End]
784  
785 [Exponent Part Begin]
786  
787 An exponent part begins with C<e> or C<E>.
788  
789 It can be followed by C<+> or C<->
790  
791 It is followed by one or more C<0-9>.
792  
793 [Exponent Part End]
794  
795 A floating point litral decimal notation can end with a suffix C<f>, C<F>, C<d>, or C<D>.
796  
797 If a suffix does not exists, a floating point litral decimal notation must have a floating point part or an exponent part.
798  
799 If the suffix C<f> or C<F> exists, the return type is float type. Otherwise the return type is double type.
800  
801 Compilation Errors:
802  
803 If the return type is float type, the floating point litral decimal notation without the suffix must be able to be parsed by the C<strtof> function in the C language. Otherwise, a compilation error occurs.
804  
805 If the return type is double type, the floating point litral decimal notation without the suffix must be able to be parsed by the C<strtod> function in the C language. Otherwise, a compilation error occurs.
806  
807 Examples:
808  
809 1.32
810 -1.32
811 1.32f
812 1.32F
813 1.32d
814 1.32D
815 1.32e3
816 1.32e-3
817 1.32E+3
818 1.32E-3
819 1.32e3f
820 12e7
821  
822 =head3 Floating Point Literal Hexadecimal Notation
823  
824 The floating point litral hexadecimal notation represents a number of float type and double type using hexadecimal numbers C<0-9a-zA-Z>.
825  
826 It can begin with a minus C<->.
827  
828 It is followed by C<0x> or C<0X>.
829  
830 It is followed by one or more C<0-9a-zA-Z>.
831  
832 C<_> can be placed at the any positions after C<0x> or C<0X>.
833  
834 It can be followed by a floating point part, an exponent part, or a combination of a floating point part and an exponent part.
835  
836 [Floating Point Part Begin]
837  
838 A floating point part begins with C<.>
839  
840 It is followed by one or more C<0-9a-zA-Z>.
841  
842 [Floating Point Part End]
843  
844 [Exponent Part Begin]
845  
846 An exponent part begins with C<p> or C<P>.
847  
848 It can be followed by C<+> or C<->.
849  
850 It is followed by one or more C<0-9>.
851  
852 [Exponent Part End]
853  
854 A floating point litral hexadecimal notation can end with a suffix C<f>, C<F>, C<d>, or C<D>.
855  
856 If a suffix does not exists, a floating point litral hexadecimal notation must have a floating point part or an exponent part.
857  
858 Compilation Errors:
859  
860 If the return type is float type, the floating point litral hexadecimal notation without the suffix must be able to be parsed by the C<strtof> function in the C language. Otherwise, a compilation error occurs.
861  
862 If the return type is double type, thefloating point litral hexadecimal notation without the suffix must be able to be parsed by the C<strtod> function in the C language. Otherwise, a compilation error occurs.
863  
864 Examples:
865  
866 0x3d3d.edp0
867 0x3d3d.edp3
868 0x3d3d.edP3
869 0x3d3d.edP+3
870 0x3d3d.edP-3f
871 0x3d3d.edP-3F
872 0x3d3d.edP-3d
873 0x3d3d.edP-3D
874 0x3d3dP+3
875  
876 =head2 Bool Literals
877  
878 The bool literal represents a bool object.
879  
880 =head3 true
881  
882 C<true> is the alias for L<Bool#TRUE|SPVM::Bool/"TRUE">.
883  
884 true
885  
886 Examples:
887  
888 # true
889 my $bool_object_true = true;
890  
891 =head3 false
892  
893 C<false> is the alias for L<Bool#FALSE|SPVM::Bool/"FALSE">.
894  
895 false
896  
897 Examples:
898  
899 # false
900 my $bool_object_false = false;
901  
902 =head2 Character Literal
903  
904 A character literal represents a number of L<byte type|SPVM::Document::Language::Types/"byte Type"> that normally represents an ASCII character.
905  
906 It begins with C<'>.
907  
908 It is followed by a printable ASCII character C<0x20-0x7e> or an L<character literal escape character|/"Character Literal Escape Characters">.
909  
910 It ends with C<'>.
911  
912 The return type is byte type.
913  
914 Compilation Errors:
915  
916 If the format of the character literal is invalid, a compilation error occurs.
917  
918 =head3 Character Literal Escape Characters
919  
920 The List of Character Literal Escape Characters:
921  
922 =begin html
923  
924 <table>
925 <tr>
926 <th>
927 Character Literal Escape Characters
928 </th>
929 <th>
930 Values
931 </th>
932 </tr>
933 <tr>
934 <td>
935 \a
936 </td>
937 <td>
938 <code>0x07</code> BEL
939 </td>
940 </tr>
941 <tr>
942 <td>
943 \t
944 </td>
945 <td>
946 <code>0x09</code> HT
947 </td>
948 </tr>
949 <tr>
950 <td>
951 \n
952 </td>
953 <td>
954 <code>0x0A</code> LF
955 </td>
956 </tr>
957 <tr>
958 <td>
959 \f
960 </td>
961 <td>
962 <code>0x0C</code> FF
963 </td>
964 </tr>
965 <tr>
966 <td>
967 \r
968 </td>
969 <td>
970 <code>0x0D</code> CR
971 </td>
972 </tr>
973 <tr>
974 <td>
975 \"
976 </td>
977 <td>
978 <code>0x22</code> "
979 </td>
980 </tr>
981 <tr>
982 <td>
983 \'
984 </td>
985 <td>
986 <code>0x27</code> '
987 </td>
988 </tr>
989 <tr>
990 <td>
991 \\
992 </td>
993 <td>
994 <code>0x5C</code> \
995 </td>
996 </tr>
997 <tr>
998 <td>
999 <a href="#Octal-Escape-Character">Octal Escape Character</a>
1000 </td>
1001 <td>
1002 A number represented by an octal escape character
1003 </td>
1004 </tr>
1005 <tr>
1006 <td>
1007 <a href="#Hexadecimal-Escape-Character">Hexadecimal Escape Character</a>
1008 </td>
1009 <td>
1010 A number represented by a hexadecimal escape character
1011 </td>
1012 </tr>
1013 </table>
1014  
1015 =end html
1016  
1017 The type of every character literal escape character is byte type.
1018  
1019 Examples:
1020  
1021 # Charater literals
1022 'a'
1023 'x'
1024 '\a'
1025 '\t'
1026 '\n'
1027 '\f'
1028 '\r'
1029 '\"'
1030 '\''
1031 '\\'
1032 ' '
1033 '\0'
1034 '\012'
1035 '\377'
1036 '\o{1}'
1037 '\xab'
1038 '\xAB'
1039 '\x0D'
1040 '\x0A'
1041 '\xD'
1042 '\xA'
1043 '\xFF'
1044 '\x{A}'
1045  
1046 =head2 Octal Escape Character
1047  
1048 The octal escape character represents an unsined 8-bit integer using octal numbers C<0-7>.
1049  
1050 The octal escape character is a part of a L<string literal|/"String Literal"> and a L<character literal|/"Character Literal">.
1051  
1052 It begins with C<\0>, C<\1>, C<\2>, C<\3>, C<\4>, C<\5>, C<\6>, C<\7>, or C<\o{>.
1053  
1054 If it begins with C<\0>, C<\1>, C<\2>, C<\3>, C<\4>, C<\5>, C<\6>, or C<\7>, it is followed by one to two C<0-7>.
1055  
1056 If it begins with C<\o{>, it is followed by one to three C<0-7>, and ends with C<}>.
1057  
1058 The octal numbers after C<\> or C<\o{> is called octal numbers part.
1059  
1060 Octal numbers part is interpreted as an unsined 8-bit integer, and is converted to a number of byte type without changing the bits.
1061  
1062 Compilation Errors:
1063  
1064 The octal numbers part must be less than or equal to C<377>. Otherwise a compilation error occurs.
1065  
1066 If an octal escape character begins with C<\o{>, the close C<}> must exist. Otherwise a compilation error occurs.
1067  
1068 Examples:
1069
1070 # Octal escape characters
1071 \0
1072 \01
1073 \03
1074 \012
1075 \001
1076 \077
1077 \377
1078 \o{1}
1079 \o{12}
1080  
1081 =head2 Hexadecimal Escape Character
1082  
1083 The hexadecimal escape character represents an unsined 8-bit integer using hexadecimal numbers C<0-9a-fA-F>.
1084  
1085 The hexadecimal escape character is a part of a L<string literal|/"String Literal"> and a L<character literal|/"Character Literal">.
1086  
1087 The hexadecimal escape character begins with C<\x>.
1088  
1089 It can be followed by C<{>.
1090  
1091 It is followed by one or two C<0-9a-fA-F>. This is called hexadecimal numbers part.
1092  
1093 If it contains C<{>, it must be followed by C<}>.
1094  
1095 Hexadecimal numbers part is interpreted as an unsined 8-bit integer, and is converted to a number of byte type without changing the bits.
1096  
1097 Compilation Errors:
1098  
1099 If the format of the hexadecimal escape character is invalid, a compilation error occurs.
1100  
1101 Examples:
1102
1103 # Hexadecimal escape characters
1104 \xab
1105 \xAB
1106 \x0D
1107 \x0A
1108 \xD
1109 \xA
1110 \xFF
1111 \x{A}
1112  
1113 =head2 String Literal
1114  
1115 A string literal represents a constant L<string|SPVM::Document::Language::Types/"String">.
1116  
1117 A string literal begins with C<">.
1118  
1119 It is followed by zero or more UTF-8 characters, L<string literal escape characters|/"String Literal Escape Characters">, or L<variable expansions|/"Variable Expansion">.
1120  
1121 It ends with C<">.
1122  
1123 The return type is L<string type|SPVM::Document::Language::Types/"string Type">.
1124  
1125 Compilation Errors:
1126  
1127 If the format of the string literal is invalid, a compilation error occurs.
1128  
1129 Examples:
1130  
1131 # String literals
1132 ""
1133 "abc";
1134 "あいう"
1135 "hello\tworld\n"
1136 "hello\x0D\x0A"
1137 "hello\xA"
1138 "hello\x{0A}"
1139 "hello\0"
1140 "hello\012"
1141 "hello\377"
1142 "AAA $foo BBB"
1143 "AAA $FOO BBB"
1144 "AAA $$foo BBB"
1145 "AAA $foo->{x} BBB"
1146 "AAA $foo->[3] BBB"
1147 "AAA $foo->{x}[3] BBB"
1148 "AAA $@ BBB"
1149 "\N{U+3042}\N{U+3044}\N{U+3046}"
1150  
1151 =head3 String Literal Escape Characters
1152  
1153 The List of String Literal Escape Characters:
1154  
1155 =begin html
1156  
1157 <table>
1158 <tr>
1159 <th>
1160 String Literal Escape Characters
1161 </th>
1162 <th>
1163 Values
1164 </th>
1165 </tr>
1166 <tr>
1167 <td>
1168 \a
1169 </td>
1170 <td>
1171 <code>0x07</code> BEL
1172 </td>
1173 </tr>
1174 <tr>
1175 <td>
1176 \t
1177 </td>
1178 <td>
1179 <code>0x09</code> HT
1180 </td>
1181 </tr>
1182 <tr>
1183 <td>
1184 \n
1185 </td>
1186 <td>
1187 <code>0x0A</code> LF
1188 </td>
1189 </tr>
1190 <tr>
1191 <td>
1192 \f
1193 </td>
1194 <td>
1195 <code>0x0C</code> FF
1196 </td>
1197 </tr>
1198 <tr>
1199 <td>
1200 \r
1201 </td>
1202 <td>
1203 <code>0x0D</code> CR
1204 </td>
1205 </tr>
1206 <tr>
1207 <td>
1208 \"
1209 </td>
1210 <td>
1211 <code>0x22</code> "
1212 </td>
1213 </tr>
1214 <tr>
1215 <td>
1216 \$
1217 </td>
1218 <td>
1219 <code>0x24</code> $
1220 </td>
1221 </tr>
1222 <tr>
1223 <td>
1224 \'
1225 </td>
1226 <td>
1227 <code>0x27</code> '
1228 </td>
1229 </tr>
1230 <tr>
1231 <td>
1232 \\
1233 </td>
1234 <td>
1235 <code>0x5C</code> \
1236 </td>
1237 </tr>
1238 <tr>
1239 <td>
1240 <a href="#Octal-Escape-Character">Octal Escape Character</a>
1241 </td>
1242 <td>
1243 A number represented by an octal escape character
1244 </td>
1245 </tr>
1246 <tr>
1247 <td>
1248 <a href="#Hexadecimal-Escape-Character">Hexadecimal Escape Character</a>
1249 </td>
1250 <td>
1251 A number represented by a hexadecimal escape character
1252 </td>
1253 </tr>
1254 <tr>
1255 <td>
1256 <a href="#Unicode-Escape-Character">A Unicode escape character</a>
1257 </td>
1258 <td>
1259 Numbers represented by an Unicode escape character
1260 </td>
1261 </tr>
1262 <tr>
1263 <td>
1264 <a href="#Raw-Escape-Characters">A raw escape character</a>
1265 </td>
1266 <td>
1267 Numbers represented by a hexadecimal escape character
1268 </td>
1269 </tr>
1270 </table>
1271  
1272 =end html
1273  
1274 The type of every string literal escape character ohter than the Unicode escape character and the raw escape character is byte type.
1275  
1276 The type of each number contained in the Unicode escape character and the raw escape character is byte type.
1277  
1278 =head3 Unicode Escape Character
1279  
1280 The Unicode escape character represents an UTF-8 character.
1281  
1282 An UTF-8 character is represented by an Unicode code point with hexadecimal numbers C<0-9a-fA-F>.
1283  
1284 This is one to four numbers of byte type.
1285  
1286 The Unicode escape character is a part of a L<string literal|/"String Literal">.
1287  
1288 It begins with C<\N{U+>.
1289  
1290 It is followed by one or more C<0-9a-fA-F>. This is called code point part.
1291  
1292 It ends with C<}>.
1293  
1294 Compilation Errors:
1295  
1296 If a code point part is not a Unicode scalar value, a compilation error occurs.
1297  
1298 Examples:
1299
1300 # Unicode escape characters
1301
1302 # あ
1303 \N{U+3042}
1304
1305 # い
1306 \N{U+3044}
1307
1308 # う
1309 \N{U+3046}"
1310  
1311 =head3 Raw Escape Characters
1312  
1313 A raw escape character is an escapa character that <\> is interpreted as ASCII C<\> and the following character is interpreted as itself.
1314  
1315 For example, a raw escape character C<\s> is ASCII chracters C<\s>.
1316  
1317 A raw escape character is a part of a L<string literal|/"String Literal">.
1318  
1319 The List of Raw Escape Characters:
1320  
1321 =begin html
1322  
1323 <table>
1324 <tr><th>Raw Escape Characters</th></tr>
1325 <tr><td>\!</td></tr>
1326 <tr><td>\#</td></tr>
1327 <tr><td>\%</td></tr>
1328 <tr><td>\&</td></tr>
1329 <tr><td>\(</td></tr>
1330 <tr><td>\)</td></tr>
1331 <tr><td>\*</td></tr>
1332 <tr><td>\+</td></tr>
1333 <tr><td>\,</td></tr>
1334 <tr><td>\-</td></tr>
1335 <tr><td>\.</td></tr>
1336 <tr><td>\/</td></tr>
1337 <tr><td>\:</td></tr>
1338 <tr><td>\;</td></tr>
1339 <tr><td>\<</td></tr>
1340 <tr><td>\=</td></tr>
1341 <tr><td>\></td></tr>
1342 <tr><td>\?</td></tr>
1343 <tr><td>\@</td></tr>
1344 <tr><td>\A</td></tr>
1345 <tr><td>\B</td></tr>
1346 <tr><td>\C</td></tr>
1347 <tr><td>\D</td></tr>
1348 <tr><td>\E</td></tr>
1349 <tr><td>\G</td></tr>
1350 <tr><td>\H</td></tr>
1351 <tr><td>\K</td></tr>
1352 <tr><td>\N</td></tr>
1353 <tr><td>\P</td></tr>
1354 <tr><td>\Q</td></tr>
1355 <tr><td>\R</td></tr>
1356 <tr><td>\S</td></tr>
1357 <tr><td>\V</td></tr>
1358 <tr><td>\W</td></tr>
1359 <tr><td>\X</td></tr>
1360 <tr><td>\Z</td></tr>
1361 <tr><td>\[</td></tr>
1362 <tr><td>\]</td></tr>
1363 <tr><td>\^</td></tr>
1364 <tr><td>\_</td></tr>
1365 <tr><td>\`</td></tr>
1366 <tr><td>\b</td></tr>
1367 <tr><td>\d</td></tr>
1368 <tr><td>\g</td></tr>
1369 <tr><td>\h</td></tr>
1370 <tr><td>\k</td></tr>
1371 <tr><td>\p</td></tr>
1372 <tr><td>\s</td></tr>
1373 <tr><td>\v</td></tr>
1374 <tr><td>\w</td></tr>
1375 <tr><td>\z</td></tr>
1376 <tr><td>\{</td></tr>
1377 <tr><td>\|</td></tr>
1378 <tr><td>\}</td></tr>
1379 <tr><td>\~</td></tr>
1380 </table>
1381  
1382 =end html
1383  
1384 =head3 Variable Expansion
1385  
1386 The variable expasion is a syntax to embed L<getting a local variable|SPVM::Document::Language::Operators/"Local Variable Get Operation">, L<getting a class variables|SPVM::Document::Language::Operators/"Class Variable Get Operation">, a L<dereference|SPVM::Document::Language::Operators/"Dereference Operator">, L<getting a field|SPVM::Document::Language::Operators/"Field Get Operation">, L<getting an array element|SPVM::Document::Language::Operators/"Array Element Get Operation">, L<getting the exception variable|SPVM::Document::Language::Operators/"Exception Variable Get Operation"> into a L<string literal|"String Literal">.
1387  
1388 "AAA $foo BBB"
1389 "AAA $FOO BBB"
1390 "AAA $$foo BBB"
1391 "AAA $foo->{x} BBB"
1392 "AAA $foo->[3] BBB"
1393 "AAA $foo->{x}[3] BBB"
1394 "AAA $foo->{x}->[3] BBB"
1395 "AAA $@ BBB"
1396 "AAA ${foo}BBB"
1397  
1398 The above codes are expanded to the following codes.
1399  
1400 "AAA " . $foo . " BBB"
1401 "AAA " . $FOO . " BBB"
1402 "AAA " . $$foo . " BBB"
1403 "AAA " . $foo->{x} . " BBB"
1404 "AAA " . $foo->[3] . " BBB"
1405 "AAA " . $foo->{x}[3] . " BBB"
1406 "AAA " . $foo->{x}->[3] . " BBB"
1407 "AAA " . $@ . "BBB"
1408 "AAA " . ${foo} . "BBB"
1409  
1410 The operation of getting field does not contain L<space characters|/"Space Characters"> between C<{> and C<}>.
1411  
1412 The index of getting array element must be a constant interger.
1413  
1414 The getting array dose not contain L<space characters|/"Space Characters"> between C<[> and C<]>.
1415  
1416 The end C<$> is interpreted by C<$>, not interpreted as a variable expansion.
1417
1418 # AAA$
1419 "AAA$"
1420  
1421 =head2 Single-Quoted String Literal
1422  
1423 A single-quoted string literal represents a constant string without variable expansions with a few escape characters.
1424  
1425 It begins with C<q'>.
1426  
1427 It is followed by zero or more UTF-8 characters, or L<single-quoted string literal escape characters|/"Single-Quoted String Literal Escape Characters">.
1428  
1429 It ends with C<'>.
1430  
1431 The return type is L<string type|SPVM::Document::Language::Types/"string Type">.
1432  
1433 Compilation Errors:
1434  
1435 A single-quoted string literal must be end with C<'>. Otherwise a compilation error occurs.
1436  
1437 If the escape character in a single-quoted string literal is invalid, a compilation error occurs.
1438  
1439 Examples:
1440  
1441 # Single-quoted string literals
1442 q'abc';
1443 q'abc\'\\';
1444  
1445 =head3 Single-Quoted String Literal Escape Characters
1446  
1447 The List of Single-Quoted String Literal Escape Characters:
1448  
1449 =begin html
1450  
1451 <table>
1452 <tr>
1453 <th>
1454 Single-Quoted String Literal Escape Characters
1455 </th>
1456 <th>
1457 Values
1458 </th>
1459 </tr>
1460 <tr>
1461 <td>
1462 \'
1463 </td>
1464 <td>
1465 <code>0x27</code> '
1466 </td>
1467 </tr>
1468 <tr>
1469 <td>
1470 \\
1471 </td>
1472 <td>
1473 <code>0x5C</code> \
1474 </td>
1475 </tr>
1476 </table>
1477  
1478 =end html
1479  
1480 The type of every single-quoted string literal escape character is byte type.
1481  
1482 =head2 Here Document
1483  
1484 A here document represents a constant string in multiple lines without escape characters and L<variable expansions|/"Variable Expansion">.
1485  
1486 <<'HERE_DOCUMENT_NAME';
1487 LINE1
1488 LINE2
1489 LINEn
1490 HERE_DOCUMENT_NAME
1491  
1492 A here document begins with C<<<'HERE_DOCUMENT_NAME';> and ASCII C<LF>.
1493  
1494 I<HERE_DOCUMENT_NAME> is a L<here document name|/"Here Document Name">.
1495  
1496 It is followed by a string in multiple lines.
1497  
1498 It ends with I<HERE_DOCUMENT_NAME> from the beginning of a line and ASCII C<LF>.
1499  
1500 Compilation Errors:
1501  
1502 C<<<'HERE_DOCUMENT_NAME';> must not contain L<space characters|/"Space Characters">. Otherwise a compilation error occurs.
1503  
1504 Examples:
1505
1506 # Here document
1507 my $string = <<'EOS';
1508 Hello
1509 World
1510 EOS
1511  
1512 =head3 Here Document Name
1513  
1514 A here document name consist of C<a-z>, C<A-Z>, C<_>, C<0-9>.
1515  
1516 The length of a here document name is greater than or equal to 0.
1517  
1518 A here document name cannot begin with C<0-9>.
1519  
1520 A here document name cannot contain C<__>.
1521  
1522 Compilaition Errors:
1523  
1524 If the format of a here document name is invalid, a compilatio error occurs.
1525  
1526 =head1 See Also
1527  
1528 =over 2
1529  
1530 =item * L<SPVM::Document::Language::SyntaxParsing>
1531  
1532 =item * L<SPVM::Document::Language::Statements>
1533  
1534 =item * L<SPVM::Document::Language::Operators>
1535  
1536 =item * L<SPVM::Document::Language::Class>
1537  
1538 =item * L<SPVM::Document::Language>
1539  
1540 =item * L<SPVM::Document>
1541  
1542 =back
1543  
1544 =head1 Copyright & License
1545  
1546 Copyright (c) 2023 Yuki Kimoto
1547  
1548 MIT License