File Coverage

blib/lib/Dancer/Plugin/DetectRobots.pm

Criterion	Covered	Total	%
statement	31	57	54.3
branch	5	30	16.6
condition			n/a
subroutine	7	8	87.5
pod			n/a
total	43	95	45.2

package Dancer::Plugin::DetectRobots;
{
  $Dancer::Plugin::DetectRobots::VERSION = '0.6';
}

use strict;
use warnings;

use Regexp::Assemble qw();

use Dancer ':syntax';
use Dancer::Plugin;

my $conf = plugin_setting;
my $key  = $conf->{session_key} || 'robot_client';
my $type = $conf->{type} || 'BASIC';

my $reList = _read_list();
my $basic = _assemble( $reList, 'basic' );
my $extended = _assemble( $reList, 'extended' );
my $generic = _assemble( $reList, 'generic' );

register is_robot => sub {
	my $ua = request->user_agent;
	my $value = "";
	if( session( $key ) ) {
		$value = session $key;
	}
	if( $value eq "NO") {
		return 0;
	}
	elsif( $value eq "BASIC") {
		return 1;
	}
	elsif( $value eq "EXTENDED") {
		return 1;
	}
	elsif( $value eq "GENERIC") {
		return 1;
	}

	my $rv = 0;

	if ( $type eq "BASIC" ) {
		if ( $ua =~ $basic ) {
			session $key => $type;
			$rv = 1;
		}
		else {
			session $key => "NO";
		}
	}
	elsif ( $type eq "EXTENDED" ) {
		if ( $ua =~ $basic ) {
			session $key => $type;
			$rv = 1;
		}
		elsif ( $ua =~ $extended ) {
			session $key => $type;
			$rv = 1;
		}
		else {
			session $key => "NO";
		}
	}
	elsif ( $type eq "GENERIC" ) {
		if ( $ua =~ $generic ) {
			session $key => $type;
			$rv = 1;
		}
		else {
			session $key => "NO";
		}
	}
	return $rv;
};

sub _assemble {
	my ( $list, $use_type ) = @_;

	my $ra = Regexp::Assemble->new( flags => 'i' );
	foreach my $r ( @{ $list->{$use_type} } ) {
		$ra->add($r);
	}

	return $ra->re;
}

sub _read_list {
	my $bots = { basic => [], extended => [], generic => [], };
	my $currentType = 'basic';

	while () {
		chomp;
		next unless $_;
		$currentType = 'extended' if /\A##\s+EXTENDED/;
		$currentType = 'generic'  if /\A##\s+GENERIC/;

		push @{ $bots->{$currentType} }, $_;
	}

	return $bots;
}

register_plugin;

1;

=pod

=head1 NAME

Dancer::Plugin::DetectRobots - Dancer plugin to determine if the user is a robot

=head1 VERSION

version 0.6

=head1 DESCRIPTION

A plugin for Dancer applications providing a keyword, is_robot,
which tests request->user_agent and returns 1 if the user_agent
appears to be a robot.

To use, simply call is_robot whenever/wherever you would like to
know if the user is a bot or a human.  For example, if you would
like to skip logging for bots

	if( ! is_robot ) {
		log_message("your log message");
	}

The plugin has been written to be as efficient as possible.  The
list of Robot UserAgent strings is only matched against request->user_agent
once per session.

This is done by storing its results in a session variable so a session
engine must be enabled.  Session::Cookie would be a poor choice since
the optimization will be lost when dealing with a search engine or robot.

The first call to is_robot in a session checks to see if the session
variable has been set, if if it has, it returns 0 or 1 based upon the
session variable.

By default the session variable key is "robot_client"

The check is done against the list of UserAgent strings used
by AWStats.  There are three levels of testing, BASIC which matches
AWStats LevelForRobotsDetection=1, EXTENDED which matches
LevelForRobotsDetection=2 and GENERIC which is a very lax test.

By default the level is set to BASIC

You can change these settings. See L

=head1 NAME

Dancer::Plugin::DetectRobots - A plugin to detect if the HTTP_USER_AGENT
matches a known search engine or robot string.

=head1 SYNOPSYS

In your configuration, make sure you have session configured. Of course you can
use any session engine.

  session: "simple"

In your Dancer App

  use Dancer;
  use Dancer::Plugin::DetectRobots;

  if( is_robot ) {
	...
  }
  else {
	processing goes here
	...
  }

=head1 METHODS

=head2 is_robot

  # returns 1 if the HTTP_USER_AGENT as returned by request->user_agent
  # matches one of the strings used by AWStats to detect search engines and
  # bots

  if ( is_robot ) {
	..
  }

=head1 CONFIGURATION

With no configuration whatsoever, the plugin will work fine, thus contributing
to the I motto of Dancer.

=head2 configuration default values

These are the default values. See below for a description of the keys

  plugins:
    DetectRobots:
      session_key: robot_client
      type: BASIC

=head2 configuration description

=over

=item session_key

The name of the session key which is used to store the results of the
robot test lookup

B : C

=item type

This value determinse which of 3 lists the search tests against.
  BASIC - this is the same as AWStats LevelForRobotsDetection=1
  It tests for major search engines and know bots
  EXTENDED - this is the same as AWStats LevelForRobotsDetection=2
  It tests for major search engines and know bots as in BASIC plus
  about 800 minor bots and search engines.
  GENERIC - this is a very simple test that only looks for a couple of
  dozen generic bot strings, e.g. robot, crawl, hunter, spider ...

B : C

=back

=head1 COPYRIGHT

This software is copyright (c) 2014 by Dan Busarow .

=head1 LICENCE

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.

=head1 AUTHORS

This module has been written by Dan Busarow
based upon Plack::Middleware::DetectRobots by Heiko Jansen

=head1 SEE ALSO

L

=head1 AUTHOR

Dan Busarow

=head1 COPYRIGHT AND LICENSE

This software is copyright (c) 2014 by Dan Busarow

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.

=cut


__DATA__