File Coverage

blib/lib/Lingua/EN/CommonMistakes.pm

Criterion	Covered	Total	%
statement	63	66	95.4
branch	19	22	86.3
condition	2	3	66.6
subroutine	9	9	100.0
pod			n/a
total	93	100	93.0

package Lingua::EN::CommonMistakes;

use 5.006;
use strict;
use warnings FATAL => 'all';
use warnings::register;
use Carp;

our $VERSION = 20130425;

my %MISTAKES;

# reads data from __DATA__ section into %MISTAKES
sub _read_mistakes {
  my $in_tag = ':common';

  while ( my $line =  ) {
    chomp $line;
    $line =~ s{#.*\z}{};
    $line =~ s{\s+\z}{};
    $line =~ s{\A\s+}{};
    $line =~ s{ {2,}}{ };
    next unless $line;

    if ( $line =~ m{\A:[^\s]+\z} ) {
      $in_tag = $line;
      next;
    }

    my ( $word, $correction ) = split( /\s/, $line, 2 );
    $MISTAKES{$in_tag}{$word} = $correction;
  }
  close(Lingua::EN::CommonMistakes::DATA);

  return;
}

sub import {
  my ( $package, @args ) = @_;
  my @out_name;
  my %tags = map { $_ => 1 } qw(:common :punct);
  foreach my $arg (@args) {
    if ( substr( $arg, 0, 1 ) eq '%' ) {
      push @out_name, substr( $arg, 1 );
    }
    elsif ( substr( $arg, 0, 1 ) eq ':' ) {
      if ($arg eq ':no-defaults') {
        %tags = ();
      } elsif ($arg =~ m{\A:no-(.+)\z}) {
        delete $tags{ ":$1" };
      } else {
        $tags{$arg}++;
      }
    }
    else {
      croak __PACKAGE__ . ": import argument $arg is not understood";
    }
  }

  if ( !@out_name ) {
    push @out_name, 'MISTAKES';
  }

  if ( $tags{':american'} && $tags{':british'} ) {
    croak __PACKAGE__ .  ": can't use both :american and :british";
  }

  if ( !%MISTAKES ) {
    _read_mistakes();
  }

  my %out;
  foreach my $tag ( keys %tags ) {
    if ( !$MISTAKES{$tag} ) {
      if (warnings::enabled( __PACKAGE__ )) {
        carp __PACKAGE__ . ": import argument $tag is not understood";
      }
    }
    else {
      (%out) = ( %out, %{ $MISTAKES{$tag} } );
    }
  }

  my ($caller_package) = caller();
  foreach my $out_name (@out_name) {
    no strict 'refs';
    no warnings 'once';
    *{ $caller_package . '::' . $out_name } = \%out;
  }
  return;
}

=head1 NAME

Lingua::EN::CommonMistakes - map of common English spelling errors

=head1 SYNOPSIS

    use Lingua::EN::CommonMistakes qw(%MISTAKES);

    foreach my $word (split /\b/, $text) {
        if (my $correction = $MISTAKES{lc $word}) {
            warn "Likely spelling error: $word (-> $correction)\n";
        }
    }

    # or use a different flavor of English
    use Lingua::EN::CommonMistakes qw(:no-punct :british %MISTAKES);
    ...

Provides a customizable map of common English spelling errors with their
respective corrections.

=head1 USAGE

The behavior of this package is customized at import time.

By default, importing this package will create a hash named
C<%MISTAKES> in the calling package, containing most corrections, but
not containing either American English or British English corrections.

This behavior may be customized by providing the following parameters
when importing:

=over

=item %I [default: C<%MISTAKES>]

The map will be imported with the given name.

=item C<:common>, C<:no-common> [default: C<:common>]

If enabled, include the base set of corrections common among all
English variants. This is the largest set of corrections.

=item C<:american>, C<:no-american> [default: C<:no-american>]

If enabled, American English is desirable; include corrections from
British English to American English. For example, "colour" should be
replaced with "color".

=item C<:british>, C<:no-british> [default: C<:no-british>]

If enabled, British English is desirable; include corrections from
American English to British English. For example, "recognized" should
be replaced with "recognised".

=item C<:punct>, C<:no-punct> [default: C<:punct>]

If enabled, include corrections which introduce punctuation characters;
for example, "dont" should be replaced with "don't".

C<:no-punct> is often useful when scanning input text where
punctuation characters have special meaning, such as in most
programming languages.

=item C<:no-defaults>

If set, the corrections map only includes sets which have been
explicitly enabled.

=back

It's possible to C the package several times if multiple mappings are
needed, as in the following example:

  # one map for common mistakes, another for british->american only
  use Lingua::EN::CommonMistakes qw(%MISTAKES_COMMON);
  use Lingua::EN::CommonMistakes qw(:no-defaults :american %MISTAKES_GB_TO_US);

=head1 WHY?

One might justifiably wonder why it would make sense to use a list of
mistakes rather than a full dictionary when spell checking.

Spell checking typically uses a whitelist approach: all words are
considered incorrect unless they can be found in the whitelist
(dictionary). This module instead facilitates a blacklist approach:
words are considered correct unless they can be found in the blacklist
(map of mistakes).

A blacklist approach to spell-checking is often more suitable than a
whitelist approach when scanning text which is partly but not entirely
English.

Computer programs are a prime example of semi-English documents;
comments and identifiers may be written in English, with additional
restrictions (such as no punctuation characters permitted in
identifiers) and often contain words which are intentionally not
spelled correctly (abbreviations or corruptions of valid English
words, e.g. "int" for "integer").

Other examples include mixed language documents or documents which are
ostensibly English but contain a lot of domain-specific jargon
unlikely to be found in an English dictionary.

Despite the fact that such bodies of text are only partly English, any
occurrences of words in the blacklist are likely to be genuine errors.

A blacklist approach also makes sense when it is more important to
have a low rate of false positives than it is to find every error (for
example, an automated system which risks being ignored if it generates
too many reports of dubious value).

=head1 AUTHOR

Rohan McGovern, C

=head1 BUGS

Please view and report any bugs here:
L

=head1 ACKNOWLEDGEMENTS

Most of the word list has been sourced from other projects, including:

=over

=item *

I code checker tool, written for KDE:
L

=item *

I package checker tool, written for Debian:
L

=back

=head1 LICENSE AND COPYRIGHT

Copyright 2012 Rohan McGovern.

Incorporated word lists may be Copyright their respective authors.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 dated June, 1991 or at your option
any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

A copy of the GNU General Public License is available in the source tree;
if not, write to the Free Software Foundation, Inc.,
59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.


=cut

1;

__DATA__