File Coverage

blib/lib/Sys/Mmap.pm

Criterion	Covered	Total	%
statement	45	55	81.8
branch	15	24	62.5
condition			n/a
subroutine	9	9	100.0
pod	1	1	100.0
total	70	89	78.6

package Sys::Mmap;

=head1 NAME

Sys::Mmap - uses mmap to map in a file as a Perl variable

=head1 SYNOPSIS

    use Sys::Mmap;

    Sys::Mmap->new( my $str, 8192, 'structtest2.pl' ) or die $!;
    Sys::Mmap->new( $var, 8192 ) or die $!;

    mmap( $foo, 0, PROT_READ, MAP_SHARED, FILEHANDLE ) or die "mmap: $!";
    @tags = $foo =~ /<(.*?)>/g;
    munmap($foo) or die "munmap: $!";

    mmap( $bar, 8192, PROT_READ | PROT_WRITE, MAP_SHARED, FILEHANDLE );
    substr( $bar, 1024, 11 ) = "Hello world";

    mmap( $baz, 8192, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, STDOUT );

    $addr = mmap( $baz, 8192, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANON, STDOUT );
    Sys::Mmap::hardwire( $qux, $addr, 8192 );

=head1 DESCRIPTION

The Sys::Mmap module uses the POSIX L call
to map in a file as a Perl variable.  Memory access by mmap may be shared
between threads or forked processes, and may be a disc file that has been
mapped into memory.  L depends on your operating system supporting
UNIX or POSIX.1b mmap, of course.

B that L now defines a C<:mmap> tag and presents mmap'd files as
regular files, if that is your cup of joe.

Several processes may share one copy of the file or string, saving memory, and
concurrently making changes to portions of the file or string. When not used
with a file, it is an alternative to SysV shared memory. Unlike SysV shared
memory, there are no arbitrary size limits on the shared memory area, and
sparse memory usage is handled optimally on most modern UNIX implementations.

Using the C method provides a C'd interface to C that
allows you to use the variable as a normal variable. If a filename is provided,
the file is opened and mapped in. If the file is smaller than the length
provided, the file is grown to that length.  If no filename is provided,
anonymous shared inheritable memory is used. Assigning to the variable will
replace a section in the file corresponding to the length of the variable,
leaving the remainder of the file intact and unmodified. Using C
allows you to access the file at an offset, and does not place any requirements
on the length argument to C or the length of the variable being
inserted, provided it does not exceed the length of the memory region. This
protects you from the pathological cases involved in using C directly,
documented below.

When calling C or C directly, you need to be careful how
you use the variable. Some programming constructs may create copies of a string
which, while unimportant for smallish strings, are far less welcome if you're
mapping in a file which is a few gigabytes big. If you use C and
attempt to write to the file via the variable you need to be even more careful.
One of the few ways in which you can safely write to the string in-place is by
using C as an lvalue and ensuring that the part of the string that
you replace is exactly the same length.  Other functions will allocate other
storage for the variable, and it will no longer overlay the mapped in file.

=over 4

=item Sys::Mmap->new( C, C, C )

Maps C bytes of (the contents of) C if
C is provided, otherwise uses anonymous, shared inheritable
memory. This memory region is inherited by any Ced children.
C will now refer to the contents of that file.  Any change to
C will make an identical change to the file.  If C is zero
and a file is specified, the current length of the file will be used.  (Note:
the tied interface does not support C, so this always maps the full
file.)  If
C is larger then the file, and C is provided, the
file is grown to that length before being mapped.  This is the preferred
interface, as it requires much less caution in handling the variable.
C will be tied into the "Sys::Mmap" package, and C will be
called for you.

Assigning to C will overwrite the beginning of the file for a length
of the value being assigned in. The rest of the file or memory region after
that point will be left intact.  You may use C to assign at a given
position:

    substr(VARIABLE, POSITION, LENGTH) = NEWVALUE

=item mmap(VARIABLE, LENGTH, PROTECTION, FLAGS, FILEHANDLE, OFFSET)

Maps C bytes of (the underlying contents of) C into your
address space, starting at offset C and makes C refer to that
memory. The C argument can be omitted in which case it defaults to
zero. The C argument can be zero in which case a stat is done on
C and the file size is used to infer the mapping length.  If
C is also specified, the inferred length is C
(i.e. the remainder of the file from C onward).  An exception is
thrown if C is at or beyond the end of the file.

B: the underlying C system call requires the offset to be a
multiple of the system page size (typically 4096 bytes).  Sys::Mmap handles
non-page-aligned offsets internally by adjusting the mapping, but very large
non-aligned offsets may map slightly more memory than the requested region.

The C argument should be some ORed combination of the constants
C, C and C, or else C. The
constants C and C are unlikely to be useful here but are
included for completeness.

The C argument must include either C or C (the
latter is unlikely to be useful here).  If your platform supports it, you may
also use C or C.  If your platform supplies
C as a non-zero constant (necessarily non-POSIX) then you should also
include that in C.  POSIX.1b does not specify C as a C
argument and most if not all versions of Unix have C as zero.

mmap returns C on failure, and the address in memory where the variable
was mapped to on success.

=item munmap(VARIABLE)

Unmaps the part of your address space which was previously mapped in with a
call to C and makes VARIABLE become undefined.

munmap returns 1 on success and undef on failure.

=item hardwire(VARIABLE, ADDRESS, LENGTH)

Specifies the address in memory of a variable, possibly within a region you've
Ced another variable to. You must use the same precautions to keep the
variable from being reallocated, and use C with an exact length. If
you C a region that a Ced variable lives in, the
Ced variable will not automatically be Ced. You must do this
manually.

=item Constants

The Sys::Mmap module exports the following constants into your namespace:

    MAP_SHARED MAP_PRIVATE MAP_ANON MAP_ANONYMOUS MAP_FILE
    MAP_NORESERVE MAP_POPULATE
    MAP_HUGETLB MAP_HUGE_2MB MAP_HUGE_1GB
    PROT_EXEC PROT_NONE PROT_READ PROT_WRITE

Of the constants beginning with C, only C and C
are defined in POSIX.1b and only C is likely to be useful.

=back

=head1 BUGS

Scott Walters doesn't know XS, and is just winging it. There must be a better
way to tell Perl not to reallocate a variable in memory...

The C interface makes writing to a substring of the variable much less
efficient.  One user cited his application running 10-20 times slower when C<<
Sys::Mmap->new() >> is used than when C is called directly.

Malcolm Beattie has not reviewed Scott's work and is not responsible for any
bugs, errors, omissions, stylistic failings, importabilities, or design flaws
in this version of the code.

There should be a tied interface to C as well.

Scott Walter's spelling is awful.

C will segfault Perl if the C area it was referring to is
C'd out from under it.

C will segfault Perl if the variable was not successfully C'd
previously, or if it has since been reallocated by Perl.

=head1 AUTHOR

CowboyTim added support for MAP_NORESERVE, MAP_HUGETLB, MAP_HUGE_2MB, and MAP_HUGE_1GB.
Thanks CowboyTim!

Todd Rinaldo cleaned up code, modernized again, and merged in many fixes,
2010-2011.

Scott Walters updated for Perl 5.6.x, additions, 2002.

Malcolm Beattie, 21 June 1996.

=cut

use strict;
use warnings;

our $VERSION = '0.21';

our $AUTOLOAD; # For sub AUTOLOAD

require Exporter;
our @ISA = qw(Exporter);

our @EXPORT = qw(mmap munmap
	     MAP_ANON MAP_ANONYMOUS MAP_FILE MAP_LOCKED MAP_PRIVATE MAP_SHARED MAP_NORESERVE
    MAP_POPULATE
    MAP_HUGETLB
    MAP_HUGE_2MB
    MAP_HUGE_1GB
	     PROT_EXEC PROT_NONE PROT_READ PROT_WRITE);


sub new {

  if(scalar @_ < 3) {
    warn 'Usage: Sys::Mmap->new( $var, $desiredSize, $optFile );';
    return undef;
  }

  my $type = $_[0];
  my $leng = $_[2];

  tie $_[1], $_[0], @_[2 .. scalar(@_)-1 ];
  # tie $_[1], $type, $leng;

}

sub TIESCALAR {

  if(scalar @_ < 2) {
    # print "debug: got args: ", join ', ', @_, "\n";
    warn 'Usage: tie $var, "Sys::Mmap", $desiredSize, $optionalFile;';
    return undef;
  }

  my $me;
  my $fh;

  my $type = shift;
  my $leng = shift;
  my $file = shift;

  my $flags = constant('MAP_INHERIT',0)|
              constant('MAP_SHARED',0);

  if($file) {
    open $fh, '+>>', $file or do {
      warn "mmap: could not open file '$file' for append r/w";
      return undef;
    };
    # if we dont pad the file out to the specified length, we coredump
    my $fhsize = -s $fh;
    if($leng > $fhsize) {
      $fhsize = $leng - $fhsize;
      print $fh ("\000" x $fhsize) or die $!;
      # print $fh pack("a$fhsize", '') or die $!;
      # while($fhsize) { print $fh "\000"; $fhsize--; }
    }
    $flags |= constant('MAP_FILE',0);
  } else {
    $flags |= constant('MAP_ANON',0);
  }

  my $addr = mmap(
      $me,
      $leng,
      constant('PROT_READ',0)|constant('PROT_WRITE',0),
      $flags,
      $file ? $fh : *main::STDOUT
  ) or die $!;

  bless \$me, $type;

  # XXX return $addr somehow...
}

sub STORE {
  my $me = shift;
  my $newval = shift;
  substr($$me, 0, length($newval), $newval);
  $$me;
}

sub FETCH {
  my $me = shift;
  $$me;
}

sub AUTOLOAD {
    # This AUTOLOAD is used to 'autoload' constants from the constant()
    # XS function.  If a constant is not found then control is passed
    # to the AUTOLOAD in AutoLoader.

    if ($AUTOLOAD =~ /::(_?[a-z])/) {
        $AutoLoader::AUTOLOAD = $AUTOLOAD;
        goto &AutoLoader::AUTOLOAD;
    }

    local $! = 0;
    my $constname = $AUTOLOAD;
    $constname =~ s/.*:://;
    return if $constname eq 'DESTROY';
    my $val = constant($constname, @_ ? $_[0] : 0);
    if ($! == 0) {
        no strict 'refs';
        *$AUTOLOAD = sub { $val };
    }
    else {
        require Carp;
        Carp::croak("Your vendor has not defined Sys::Mmap macro $constname");
    }

    goto &$AUTOLOAD;
}

eval {
       require XSLoader;
       XSLoader::load( 'Sys::Mmap', $VERSION );
} or do {
    require DynaLoader;
    push @ISA, 'DynaLoader';
    bootstrap Sys::Mmap $VERSION;
};

1;

__END__