File Coverage

blib/lib/Amazon/API.pm

Criterion	Covered	Total	%
statement	189	590	32.0
branch	31	212	14.6
condition	26	163	15.9
subroutine	44	95	46.3
pod	7	21	33.3
total	297	1081	27.4

package Amazon::API;

# Generic interface to Amazon APIs

use strict;
use warnings;

use 5.010;

our %LOG4PERL_LOG_LEVELS;

BEGIN {
  %LOG4PERL_LOG_LEVELS = eval {
    require Log::Log4perl::Level;

    Log::Log4perl::Level->import();

    use vars qw($ERROR $WARN $INFO $DEBUG $TRACE $FATAL);

    return (
      error => $ERROR,
      warn  => $WARN,
      info  => $INFO,
      debug => $DEBUG,
      trace => $TRACE,
      fatal => $FATAL,
    );
  };
}

use parent qw( Exporter Class::Accessor::Fast);

use Amazon::API::Error;
use Amazon::API::Signature4;
use Amazon::API::Constants qw( :all );
use Amazon::Credentials;
use Amazon::API::Botocore qw(paginator);
use Amazon::API::Botocore::Shape::Serializer;
use Amazon::API::Botocore::Shape::Utils qw(
  require_class
  require_shape
  create_module_name
  get_service_from_class
);

use Carp;
use Carp::Always;
use Data::Dumper;
use Date::Format;
use English qw( -no_match_vars);
use HTTP::Request;
use JSON qw( encode_json decode_json );
use LWP::UserAgent;
use List::Util qw(any all pairs none);
use ReadonlyX;
use Scalar::Util qw( reftype blessed );
use Time::Local;
use Time::HiRes qw( gettimeofday tv_interval );
use XML::Simple;

Readonly::Scalar my $DEFAULT_REGION   => 'us-east-1';
Readonly::Scalar my $REGIONAL_URL_FMT => '%s://%s.%s.amazonaws.com';
Readonly::Scalar my $GLOBAL_URL_FMT   => '%s://%s.amazonaws.com';

Readonly::Scalar my $DEFAULT_LAYOUT_PATTERN => '[%d] %p (%R/%r) %M:%L - %m%n';

local $Data::Dumper::Pair  = $COLON;
local $Data::Dumper::Useqq = $TRUE;
local $Data::Dumper::Terse = $TRUE;

__PACKAGE__->follow_best_practice;

__PACKAGE__->mk_accessors(
  qw(
    action
    api
    api_methods
    botocore_metadata
    botocore_operations
    botocore_shapes
    content_type
    credentials
    debug
    decode_always
    endpoint_prefix
    error
    force_array
    http_method
    last_action
    _log_level
    logger
    log_file
    log_layout
    paginators
    print_error
    protocol
    raise_error
    raise_serialization_errors
    region
    request_uri
    response
    service
    serializer
    service_url_base
    target
    target_prefix
    url
    user_agent
    use_paginator
    version
  )
);

our $VERSION = '2.0.8'; ## no critic (RequireInterpolationOfMetachars)

our @EXPORT_OK = qw(
  create_urlencoded_content
  get_api_service
  param_n
  paginator
);

our $START_TIME    = [gettimeofday];
our $LAST_LOG_TIME = [gettimeofday];

=begin 'ignore'

The service categories were extracted from Botocore project:

grep -ri '"protocol":"' botocore/botocore/data/* | grep 'xml' | \
  cut -f 8 -d '/' | sort -u > xml.services

grep -ri '"protocol":"' botocore/botocore/data/* | grep 'json' | \
  cut -f 8 -d '/' | sort -u > json.services

grep -ri '"protocol":"' botocore/botocore/data/* | grep 'query' | \
  cut -f 8 -d '/' | sort -u > query.services

These service categories are used to deduce content type for
parameters sent to methods for these APIs when Botocore metadata was
not used to create an API class. Content-Type can however be
overridden when invoking APIs if we guess wrong.

 rest-json => application/x-amz-1.1
 rest-xml  => application/xml
 query     => application/x-www-form-urlencoded

=end 'ignore'

=cut

our %API_TYPES = (
  query => [
    qw(
      autoscaling
      cloudformation
      cloudsearch
      cloudwatch
      docdb
      ec2
      elasticache
      elasticbeanstalk
      elb
      elbv2
      forecastquery
      iam
      importexport
      neptune
      rds
      redshift
      sdb
      ses
      sns
      sqs
      sts
      timestream-query
    )
  ],
  xml => [
    qw(
      cloudfront
      route53
      s3
      s3control
    )
  ],

  json => [
    qw(
      accessanalyzer
      account
      acm
      acm-pca
      alexaforbusiness
      amp
      amplify
      amplifyuibuilder
      apigateway
      appconfig
      appconfigdata
      appflow
      appintegrations
      application-autoscaling
      applicationcostprofiler
      application-insights
      appmesh
      apprunner
      appstream
      appsync
      athena
      auditmanager
      autoscaling-plans
      backup
      backup-gateway
      batch
      billingconductor
      braket
      budgets
      ce
      chime
      chime-sdk-identity
      chime-sdk-media-pipelines
      chime-sdk-meetings
      chime-sdk-messaging
      cloud9
      cloudcontrol
      clouddirectory
      cloudhsm
      cloudhsmv2
      cloudsearchdomain
      cloudtrail
      codeartifact
      codebuild
      codecommit
      codedeploy
      codeguruprofiler
      codeguru-reviewer
      codepipeline
      codestar
      codestar-connections
      codestar-notifications
      cognito-identity
      cognito-idp
      cognito-sync
      comprehend
      comprehendmedical
      compute-optimizer
      config
      connect
      connectcampaigns
      connect-contact-lens
      connectparticipant
      cur
      customer-profiles
      databrew
      datapipeline
      datasync
      dax
      detective
      devicefarm
      devops-guru
      directconnect
      discovery
      dlm
      dms
      drs
      ds
      dynamodb
      dynamodbstreams
      ebs
      ec2-instance-connect
      ecr
      ecr-public
      ecs
      efs
      eks
      elastic-inference
      elastictranscoder
      emr
      emr-containers
      emr-serverless
      es
      events
      evidently
      finspace
      finspace-data
      firehose
      fis
      fms
      forecast
      forecastquery
      frauddetector
      fsx
      gamelift
      gamesparks
      glacier
      globalaccelerator
      glue
      grafana
      greengrassv2
      groundstation
      guardduty
      health
      healthlake
      honeycode
      identitystore
      imagebuilder
      inspector
      inspector2
      iot
      iot1click-projects
      iotanalytics
      iot-data
      iotdeviceadvisor
      iotevents
      iotevents-data
      iotfleethub
      iot-jobs-data
      iotsecuretunneling
      iotsitewise
      iotthingsgraph
      iottwinmaker
      iotwireless
      ivs
      ivschat
      kafkaconnect
      kendra
      keyspaces
      kinesis
      kinesisanalytics
      kinesisanalyticsv2
      kinesisvideo
      kinesis-video-archived-media
      kinesis-video-media
      kinesis-video-signaling
      kms
      lakeformation
      lambda
      lex-models
      lex-runtime
      lexv2-models
      lexv2-runtime
      license-manager
      license-manager-user-subscriptions
      lightsail
      location
      logs
      lookoutequipment
      lookoutmetrics
      lookoutvision
      m2
      machinelearning
      macie
      managedblockchain
      marketplace-catalog
      marketplacecommerceanalytics
      marketplace-entitlement
      mediastore
      mediastore-data
      memorydb
      meteringmarketplace
      mgh
      mgn
      migrationhub-config
      migration-hub-refactor-spaces
      migrationhubstrategy
      mobile
      mturk
      mwaa
      network-firewall
      networkmanager
      nimble
      opensearch
      opsworks
      opsworkscm
      organizations
      outposts
      panorama
      personalize
      personalize-events
      personalize-runtime
      pi
      pinpoint-email
      pinpoint-sms-voice-v2
      polly
      pricing
      proton
      qldb
      qldb-session
      quicksight
      ram
      rbin
      rds-data
      redshift-data
      redshift-serverless
      rekognition
      resiliencehub
      resource-groups
      resourcegroupstaggingapi
      robomaker
      rolesanywhere
      route53domains
      route53-recovery-cluster
      route53resolver
      rum
      s3outposts
      sagemaker
      sagemaker-a2i-runtime
      sagemaker-edge
      sagemaker-featurestore-runtime
      sagemaker-runtime
      savingsplans
      secretsmanager
      securityhub
      servicecatalog
      servicecatalog-appregistry
      servicediscovery
      service-quotas
      sesv2
      shield
      signer
      sms
      snowball
      snow-device-management
      ssm
      ssm-contacts
      ssm-incidents
      sso
      sso-admin
      sso-oidc
      stepfunctions
      storagegateway
      support
      swf
      synthetics
      textract
      timestream-query
      timestream-write
      transcribe
      transfer
      translate
      voice-id
      waf
      waf-regional
      wafv2
      wellarchitected
      wisdom
      workdocs
      worklink
      workmail
      workmailmessageflow
      workspaces
      workspaces-web
      xray
    )
  ],
);

our @GLOBAL_SERVICES = qw(
  cloudfront
  iam
  importexport
  route53
  s3
  savingsplans
  sts
);

our @REQUIRED_KEYS = qw( aws_access_key_id aws_secret_access_key );

########################################################################
sub new {
########################################################################
  my ( $class, @options ) = @_;

  $class = ref $class || $class;

  my %options = ref $options[0] ? %{ $options[0] } : @options;

  my $self = $class->SUPER::new( \%options );

  if ( $self->get_service_url_base ) {
    $self->set_service( $self->get_service_url_base );
  }

  croak 'service is required'
    if !$self->get_service;

  $self->_set_default_logger;

  $self->_set_defaults(%options);

  $self->_create_methods;

  if ( $self->is_botocore_api ) {
    $self->set_serializer(
      Amazon::API::Botocore::Shape::Serializer->new(
        service => get_service_from_class( ref $self )
      )
    );
  }

  return $self;
}

########################################################################
sub set_log_level {
########################################################################
  my ( $self, $log_level ) = @_;

  $self->set__log_level($log_level);

  if ( Log::Log4perl->initialized() ) {
    Log::Log4perl->get_logger->level( $LOG4PERL_LOG_LEVELS{$log_level} );
  }
  else {
    my $logger = $self->get_logger;

    if ( $logger->can('level') ) {
      $logger->level($log_level);
    }
  }

  return $self;
}

########################################################################
sub get_log_level {
########################################################################
  my ($self) = @_;

  return $self->get__log_level();
}

########################################################################
sub get_api_service {
########################################################################
  my ( $service, %args ) = @_;

  $service = create_module_name lc $service;

  my $class = sprintf 'Amazon::API::%s', $service;

  require_class $class;

  return $class->new(%args);
}

########################################################################
sub decode_response {
########################################################################
  my ( $self, $response ) = @_;

  $response = $response || $self->get_response;

  # could be Furl or HTTP?
  if ( !ref $response && $response->can('content') ) {
    croak q{can't decode response - not a response object: } . ref $response;
  }

  my $content      = $response->content;
  my $content_type = $response->content_type;

  my $decoded_content;

  if ($content) {

    $decoded_content = eval {
      if ( $content_type =~ /json/xmsi ) {
        decode_json($content);
      }
      elsif ( $content_type =~ /xml/xmsi ) {

        XMLin(
          $content,
          ForceContent => $self->is_botocore_api,
          ForceArray   => ['item'],
          #          $self->get_force_array ? ( ForceArray => ['item'] ) : ()
        );
      }
    };

    # disregard content_type (it might be misleading)
    if ( !$decoded_content || $EVAL_ERROR ) {
      $decoded_content = eval { return decode_json($content); };

      if ( !$decoded_content || $EVAL_ERROR ) {
        $decoded_content = eval {
          return XMLin(
            $content,
            SuppressEmpty => undef,
            $self->get_force_array ? ( ForceArray => ['item'] ) : ()
          );
        };
      }
    }
  }

  $content = $decoded_content || $content;

  DEBUG( sub { return Dumper( [ 'content', $content ] ) } );

  # we'll only have a "serializer" if this is a Botocore generated API
  my $serializer = $self->get_serializer;

  if ( ref $content && $serializer ) {

    my $botocore_action = $self->get_botocore_action;

    my $output = $botocore_action->{'output'};

    my $orig_content = $content;

    if ( $output->{resultWrapper} ) {
      $content = $content->{ $output->{resultWrapper} };
    }

    DEBUG( sub { return Dumper( [ 'content', $content, $output ] ) } );

    $serializer->set_logger( $self->get_logger );

    $content = eval {
      $serializer->serialize(
        service => get_service_from_class( ref $self ),
        shape   => $output->{shape},
        data    => $content
      );
    };

    # ...but this isn't necessarily where things STB
    if ( !$content || $EVAL_ERROR ) {
      if ( $self->get_raise_serialization_errors ) {
        die $EVAL_ERROR;
      }
      else {
        warn
          "error serializing content: please report this error\n$EVAL_ERROR";
        $content = $orig_content;
      }
    }
  }

  return $content;
}

########################################################################
sub get_botocore_action {
########################################################################
  my ($self) = @_;

  return $self->get_botocore_operations->{ $self->get_action };
}

########################################################################
sub is_botocore_shape {
########################################################################
  my ($request) = @_;

  my $shape_name = ref $request;

  if ( $shape_name =~ /Botocore::Shape::([^:]+)::([^:]+)$/xsm ) {
    $shape_name = $2;
  }
  else {
    $shape_name = undef;
  }

  return $shape_name;
}

########################################################################
sub is_param_type {
########################################################################
  my ( $self, $shape_name, $param, $type ) = @_;

  my $members = $self->get_botocore_shapes->{$shape_name}->{members};
  my $member  = $members->{$param};

  my $location = $member->{location};

  TRACE(
    sub {
      return Dumper( $members, $member, $location, $param, $type,
        $shape_name );
    }
  );

  return ( $location && $location eq $type )
    ? $member->{locationName}
    : $EMPTY;
}

########################################################################
sub is_query_param {
########################################################################
  my ( $self, $shape_name, $param ) = @_;

  return $self->is_param_type( $shape_name, $param, 'querystring' );
}

########################################################################
sub is_uri_param {
########################################################################
  my ( $self, $shape_name, $param ) = @_;

  return $self->is_param_type( $shape_name, $param, 'uri' );
}

########################################################################
sub create_botocore_request {
########################################################################
  my ( $self, %args ) = @_;

  my ( $parameters, $action ) = @args{qw(parameters action)};

  $action //= $self->get_action;

  croak 'no action'
    if !$action;

  croak 'no parameters'
    if !$parameters;

  croak 'not a botocore API'
    if !$self->is_botocore_api($action);

  my $botocore_operations = $self->get_botocore_operations->{$action};

  my $input = $botocore_operations->{'input'};
  my $shape = $input->{'shape'};

  my $class = require_shape( $shape, get_service_from_class($self) );

  croak "could not create request shape: $shape\n$EVAL_ERROR"
    if !$class;

  my $request = $class->new($parameters); # request class

  return $request;
}

########################################################################
# init_botocore_request( $self, $request)
########################################################################

# This function will accept either an object which is a sub-class of
# Amazon::API::Botocore::Shape, or a hash if the parameters have been
# constructed "by-hand",
#
# The parameters are used to populate both the URL if some parameters
# are passed in the URL and either a JSON or XML payload depending on
# the API type (rest-json, rest-xml).

########################################################################
sub init_botocore_request {
########################################################################
  my ( $self, $request ) = @_;

  $request //= {};

  my $action = $self->get_action;

  my $botocore_operations = $self->get_botocore_operations->{$action};

  TRACE(
    sub {
      return Dumper(
        [ 'init_botocore_request:', $botocore_operations,
          $request,                 $self->get_botocore_metadata
        ]
      );
    }
  );

  my $protocol = $self->get_botocore_metadata->{'protocol'};

  my $input = $botocore_operations->{'input'};

  my $shape = $input->{'shape'};

  my $request_shape_name = is_botocore_shape($request);

  # if a shape object is passed, it must be the correct type
  croak "$action requires a $shape object, not a $request_shape_name object"
    if $request_shape_name && $request_shape_name ne $shape;

  # try to create a Botocore request shape
  my $boto_request;

  if ( !$request_shape_name && $self->is_botocore_api ) {
    $boto_request = $self->create_botocore_request( parameters => $request );
    if ( !$boto_request ) {
      carp "could not create a botocore request object\n$EVAL_ERROR\n";
    }
    else {
      $request_shape_name = is_botocore_shape($boto_request);
      $request            = $boto_request;
    }
  }

  my $http   = $botocore_operations->{'http'};
  my $method = $http->{'method'};

  my %parameters;

  # is the request a Botocore::Shape object? if so we can use metadata
  # to create URI and payload, otherwise it's up to the caller to make
  # sure the URI and the payload are correct...good luck!

  if ( !$request_shape_name ) {
    %parameters = %{$request};
  }
  else {
    my $finalized_request = $request->finalize;

    if ( $protocol =~ /rest\-(xml|json)/xsm ) {
      $finalized_request = { $request_shape_name => $finalized_request };
    }

    DEBUG(
      sub { return Dumper( [ 'finalized request:', $finalized_request ] ) } );

    %parameters = %{$finalized_request};

    if ( defined $input->{'xmlNamespace'} ) {
      $parameters{$request_shape_name}->{'xmlns'}
        = $input->{'xmlNamespace'}->{uri};
    }

    DEBUG( sub { return Dumper( [ 'parameters:', \%parameters ] ) } );
  }

  my $json_version = $self->get_botocore_metadata->{'jsonVersion'} // '1.0';

  my %content_types = (
    'ec2'       => 'application/x-www-form-urlencoded',
    'query'     => 'application/x-www-form-urlencoded',
    'rest-json' => 'application/json',
    'json'      => 'application/x-amz-json-' . $json_version,
    'rest-xml'  => 'application/xml',
  );

  $self->set_http_method($method);
  $self->set_content_type( $content_types{$protocol} );

  DEBUG("protocol: $protocol");
  DEBUG("method: $method");
  DEBUG( 'content-type: ' . $self->get_content_type );

  my $uri;

  if ( $protocol =~ /^rest\-(json|xml)/xsm ) {
    my @args = @{ $http->{'parsed_request_uri'}->{'parameters'} };

    my $request_uri_tpl = $http->{'parsed_request_uri'}->{'request_uri_tpl'};

    # if the request is a shape, we've already checked for required parameters
    if ( !$request_shape_name ) {
      foreach my $p (@args) {
        croak 'required parameter ' . $p . ' not found.'
          if !exists $parameters{$p};
      }

      $uri = sprintf $request_uri_tpl, @parameters{@args};
      $self->set_request_uri($uri);

      delete @parameters{@args};
    }
    else {

      if ($request_shape_name) {

        $uri = $http->{requestUri}; # use the Botocore template

        my $shape_parameters = $parameters{$shape};

        DEBUG(
          sub {
            return sprintf "requestUri: %s\nshape parameter: %s", $uri,
              Dumper($shape_parameters);
          }

        );

        foreach my $p ( keys %{$shape_parameters} ) {
          DEBUG("parameter: $p");

          if ( my $var = $self->is_uri_param( $request_shape_name, $p ) ) {

            my $val = $shape_parameters->{$p};

            DEBUG("var: $var val: $val");

            $uri =~ s/[{]$var[}]/$val/xsm;

            delete $shape_parameters->{$p};
          }
        }
      }
      else {
        $uri = sprintf $request_uri_tpl, @{ $parameters{$shape} }{@args};
        delete @{ $parameters{$shape} }{@args};
      }

      # we're not done yet...just to make things interesting, some
      # APIs embed request parameters in the uri, payload AND query
      # string!

      if ($request_shape_name) {
        my %query_parameters;

        my $shape_parameters = $parameters{$shape};

        foreach my $p ( keys %{$shape_parameters} ) {
          if ( my $var = $self->is_query_param( $request_shape_name, $p ) ) {
            $query_parameters{$var} = $shape_parameters->{$p};
            delete $shape_parameters->{$p};
          }
        }

        DEBUG(
          sub {
            return Dumper( [ 'query parameters: ', \%query_parameters ] );
          }
        );

        if ( keys %query_parameters ) {
          # $self->set_content_type(undef);

          $uri = sprintf '%s?%s', $uri,
            create_urlencoded_content( \%query_parameters );
        }

        if ( !keys %{$shape_parameters} ) {
          %parameters = ();
        }
      }
    }

    $self->set_request_uri($uri);

    TRACE(
      sub {
        return Dumper [ 'rest API', \%parameters, $self->get_request_uri ];
      }
    );
  }

  my $content = \%parameters;

  my $content_type = $self->get_content_type // $EMPTY;

  if ( $method ne 'POST' && !keys %parameters ) {
    $content = undef;
  }

  DEBUG( sub { return 'content ' . Dumper( [$content] ) } );

  return $content;
}

########################################################################
sub is_botocore_api {
########################################################################
  my ($self) = @_;

  return defined $self->get_botocore_metadata;
}

########################################################################
sub serialize_content {
########################################################################
  my ( $self, $parameters ) = @_;

  my $content      = $parameters;
  my $action       = $self->get_action;
  my $version      = $self->get_version;
  my $content_type = $self->get_content_type;

  DEBUG(
    sub {
      return Dumper(
        [ 'serialize_content', $parameters, 'service', $self->get_service ] );
    }
  );

  # if the API is a query API, url encode parameters
  if ( any { $_ eq lc $self->get_service } @{ $API_TYPES{query} } ) {
    $content = create_urlencoded_content( $parameters, $action, $version );
  }
  elsif ( ref $parameters && reftype($parameters) eq 'HASH' ) {
    if ( $content_type =~ /json/xsm ) {
      $content = encode_json($parameters);
    }
    elsif ( $content_type =~ /xml/xms ) {
      return
        if !ref $content || !keys %{$content};

      $content = XMLout(
        $parameters,
        XMLDecl  => $TRUE,
        KeepRoot => $TRUE,
        KeyAttr  => ['xmlns'],
      );
    }
  }

  return $content;
}

########################################################################
# invoke_api( action, parameters, content-type, headers)
########################################################################
sub invoke_api {
########################################################################
  my ( $self, @args ) = @_;

  my ( $action, $parameters, $content_type, $headers );

  if ( ref $args[0] && reftype( $args[0] ) eq 'HASH' ) {
    ( $action, $parameters, $content_type, $headers )
      = @{ $args[0] }{qw( action parameters content_type headers)};
  }
  else {
    ( $action, $parameters, $content_type, $headers ) = @args;
  }

  DEBUG(
    sub {
      return Dumper(
        [ 'parameters', $parameters, 'content-type', $content_type ] );
    }
  );

  $self->set_action($action);
  $self->set_last_action($action);
  $self->set_error(undef);

  my $decode_response = $self->get_decode_always;

  croak sprintf qq{"%s" was not generated with Botocore support.\n}
    . qq{Parameters should be simple objects, not blessed.\n}, ref $self
    if blessed($parameters) && !$self->is_botocore_api;

  my @paged_results;

  my $paginator = $self->get_paginators && $self->get_paginators->{$action};
  my $use_paginator = $paginator        && $self->get_use_paginator;

  my $limit;

  # See if this was generated w/Botocore
  if ( $self->is_botocore_api ) {
    $parameters = $self->init_botocore_request($parameters);

    $content_type = $self->get_content_type;

    if ($use_paginator) {

      $paginator->{more_results} //= $paginator->{output_token};

      $limit = $parameters->{ $paginator->{limit_key} };

      DEBUG(
        sub { return Dumper( [ 'paginator', $paginator, 'limit', $limit ] ) }
      );
    }
  }
  elsif ( !$content_type ) {
    $content_type = $self->set_content_type( $self->_set_content_type );
  }

  my $serialized_content = $self->serialize_content($parameters);

  DEBUG(
    sub {
      Dumper [ $content_type, $parameters, $serialized_content ];
    }
  );

  my $page_count = 0;

  PAGINATE:

  DEBUG( sub { return Dumper( [ 'page', ++$page_count ] ) } );

  my $rsp = $self->submit(
    content      => $serialized_content,
    content_type => $content_type,
    headers      => $headers,
  );

  $self->set_response($rsp);

  if ( !$rsp->is_success ) {

    $self->set_error(
      Amazon::API::Error->new(
        { error        => $rsp->code,
          message_raw  => $rsp->content,
          content_type => scalar $rsp->content_type,
          api          => ref $self,
          response     => $rsp,
        }
      )
    );

    if ( $self->get_print_error ) {
      print {*STDERR} $self->print_error;
    }

    if ( $self->get_raise_error ) {
      die $self->get_error; ## no critic (RequireCarping)
    }
  }

  DEBUG(
    sub {
      return Dumper(
        [ 'content',       $rsp->content,
          'decode always', $self->get_decode_always
        ]
      );
    }
  );

  return $rsp->content
    if !$self->get_decode_always;

  eval {
    if ($use_paginator) {
      my $result = $self->decode_response;

      push @paged_results, @{ $result->{ $paginator->{result_key} } };

      DEBUG(
        sub {
          return Dumper(
            [ 'paginating...',
              $result,
              'paged results',
              \@paged_results,
              $paginator->{more_results},
              !$result->{ $paginator->{more_results} }
            ]
          );
        }
      );

      return \@paged_results
        if !$result->{ $paginator->{more_results} };

      $limit = $limit // $result->{ $paginator->{limit_key} };

      my $parameters = $self->init_botocore_request(
        { $limit ? ( $paginator->{limit_key} => $limit ) : (),
          $paginator->{input_token} => $result->{ $paginator->{output_token} }
        }
      );

      $serialized_content = $self->serialize_content($parameters);

      goto PAGINATE;
    }
  };

  if ($EVAL_ERROR) {
    if ( $self->get_raise_serialization_errors ) {
      die $EVAL_ERROR;
    }
    else {
      warn "error serializing content: please report this error\n$EVAL_ERROR";
      return $rsp->content;
    }
  }

  return \@paged_results
    if $use_paginator;

  my $results = $self->decode_response;

  return $results
    if !$paginator || !$use_paginator;

  # you must have a paginator, but told me not to use it, delete
  # blank/null markers
  if ( !$results->{ $paginator->{more_results} } ) {
    delete $results->{ $paginator->{more_results} };
    delete $results->{ $paginator->{input_token} };
  }

  return $results;
}

########################################################################
sub print_error {
########################################################################
  my ($self) = @_;

  my $error = $self->get_error;

  my $err_str = 'API ERROR (' . $self->get_last_action . '): ';

  if ( $error && ref($error) =~ /Amazon::API::Error/xms ) {
    my $response = $error->get_response;

    local $Data::Dumper::Terse = $TRUE;
    local $Data::Dumper::Pair  = $COLON;
    local $Data::Dumper::Useqq = $TRUE;

    $err_str .= sprintf q{[%s], %s},
      $error->get_error, Dumper( ref $response ? $response : [$response] );
  }
  else {
    $err_str .= '[' . $error . ']';
  }

  return $err_str;
}

########################################################################
sub get_valid_token {
########################################################################
  my ($self) = @_;

  my $credentials = $self->get_credentials;

  if ( $credentials->can('is_token_expired') ) {
    if ( $credentials->is_token_expired ) {

      if ( !$self->can('refresh_token') || !$credentials->refresh_token ) {
        croak 'token expired';
      }
    }
  }

  TRACE( sub { return Dumper( [ 'valid token:', $credentials->get_token ] ) }
  );

  return $credentials->get_token;
}

########################################################################
sub submit {
########################################################################
  my ( $self, %options ) = @_;

  DEBUG( sub { return Dumper [ 'submit:', \%options ] } );

  my $method  = $self->get_http_method || 'POST';
  my $headers = $options{'headers'}    || [];

  my $url = $self->get_url;

  my $botocore_protocol
    = eval { $self->get_botocore_metadata->{'protocol'} } // $EMPTY;

  if ( $botocore_protocol =~ /^rest\-(json|xml)/xsm ) {
    croak 'no request URI provided for rest-json call'
      if !$self->get_request_uri;

    $url .= $self->get_request_uri;
  }

  DEBUG( sub { return Dumper [ $method, $url, $headers ] } );

  my $request = HTTP::Request->new( $method, $url, $headers );

  # 1. set the header
  # 2. set the content
  # 3. sign the request
  # 4. send the request & return result

  # see IMPLEMENTATION NOTES for an explanation
  if ( $self->get_api || $self->get_target_prefix ) {
    $request = $self->_set_x_amz_target($request);
  }

  $self->_set_request_content( request => $request, %options );
  my $credentials = $self->get_credentials;

  if ( my $token = $self->get_valid_token ) {
    $request->header( 'X-Amz-Security-Token', $token );
  }

  # TODO: global end-points
  my $region = $self->get_region;

  # sign the request
  Amazon::API::Signature4->new(
    -access_key     => $credentials->get_aws_access_key_id,
    -secret_key     => $credentials->get_aws_secret_access_key,
    -security_token => $credentials->get_token || undef,
    service         => $self->get_service,
    region          => $region,
  )->sign( $request, $self->get_region );

  DEBUG( sub { return Dumper( [$request] ) } );

  # make the request, return response object
  my $ua  = $self->get_user_agent;
  my $rsp = $ua->request($request);

  DEBUG( sub { return Dumper [$rsp] } );

  return $rsp;
}

# +------------------+
# | EXPORTED METHODS |
# +------------------+

########################################################################
sub param_n {
########################################################################
  my (@args) = @_;

  return Amazon::API::Botocore::Shape::Utils::param_n(@args);
}

########################################################################
# create_urlencoded_content(parameters, action, version)
# input:
#       parameters:
#                  SCALAR - query string to encode (x=y&w=z...)
#                  ARRAY  - either an array of hashes or...
#                           key/value pairs of the form x=y or...
#                           key/values pairs
#                  HASH   - key/value pairs, if the value is an array then
#                           it is assumed to be a list of hashes
#       action:    API method
#       version:   wsdl version for API
#
# output:
#       URL encodode query string
#
########################################################################
sub create_urlencoded_content {
########################################################################
  my ( $parameters, $action, $version ) = @_;

  my @args;

  if ( $parameters && !ref $parameters ) {
    @args = map { split /=/xsm } split /&/xsm, $parameters;
  }
  elsif ( $parameters && reftype($parameters) eq 'HASH' ) {
    foreach my $key ( keys %{$parameters} ) {
      if ( ref $parameters->{$key}
        && reftype( $parameters->{$key} ) eq 'ARRAY' ) {
        push @args, map { %{$_} } @{ $parameters->{$key} };
      }
      else {
        push @args, $key, $parameters->{$key};
      }
    }
  }
  elsif ( $parameters && reftype($parameters) eq 'ARRAY' ) {

    # if any are refs then they should be hashes...
    if ( any {ref} @{$parameters} ) {

      @args = map { %{$_} } @{$parameters}; # list of hashes
    }
    elsif ( any {/=/xsm} @{$parameters} ) {
      @args = map { split /=/xsm } @{$parameters}; # formatted list
    }
    else {
      @args = @{$parameters};                      # simple list
    }
  }

  my $content;

  if ( $action && !any {/Action/xsm} @args ) {
    push @args, 'Action', $action;
  }

  if ( $version && !any {/Version/xsm} @args ) {
    push @args, 'Version', $version;
  }

  return join $AMPERSAND, map { sprintf '%s=%s', @{$_} } pairs @args;
}

########################################################################
sub has_keys {
########################################################################
  my ( $self, %options ) = @_;

  my %creds = keys %options ? %options : @{$self}{@REQUIRED_KEYS};

  return $creds{'aws_secret_access_key'} && $creds{'aws_access_key_id'};
}

# +-----------------+
# | PRIVATE METHODS |
# +-----------------+

# should not be called if we have botocore definition
########################################################################
sub _set_content_type {
########################################################################
  my ($self) = @_;

  my $service = $self->get_service;

  # default content-type
  my $content_type = $self->get_content_type;

  return 'application/x-www-form-urlencoded'
    if any { $_ eq $service } @{ $API_TYPES{query} };

  return 'application/x-amz-json-1.1'
    if any { $_ eq $service } @{ $API_TYPES{json} };

  return 'application/xml'
    if any { $_ eq $service } @{ $API_TYPES{xml} };

  return $content_type;
}

########################################################################
sub _create_methods {
########################################################################
  my ($self) = @_;

  my $class = ref $self || $self;

  if ( $self->get_api_methods ) {
    no strict 'refs';       ## no critic (TestingAndDebugging::ProhibitNoStrict)
    no warnings 'redefine'; ## no critic (TestingAndDebugging::ProhibitNoWarnings)

    my $stash = \%{ __PACKAGE__ . $DOUBLE_COLON };

    foreach my $api ( @{ $self->get_api_methods } ) {

      my $method = lcfirst $api;

      $method =~ s/([[:lower:]])([[:upper:]])/$1_$2/xmsg;
      $method = lc $method;

      my $snake_case_method = $class . $DOUBLE_COLON . $method;
      my $camel_case_method = $class . $DOUBLE_COLON . $api;

      # snake case rules the day

      if ( !$stash->{$method} ) {
        *{$snake_case_method} = sub {
          my $self = shift;

          $self->invoke_api( $api, @_ );
        };
      }

      # ...but some prefer camels
      if ( !$stash->{$api} ) {
        *{$camel_case_method} = sub {
          my $self = shift;

          $self->$method(@_);
        };
      }
    }

  }

  return $self;
}

########################################################################
sub _create_stealth_logger {
########################################################################
  my ( $self, $name, $level ) = @_;

  my $sub_name = sprintf '%s::%s', $name, uc $level;

  no strict 'refs'; ## no critic ProhibitNoStrict

  return
    if defined *{$sub_name}{CODE};

  *{$sub_name} = sub {
    if ( $self->get_logger && $self->get_logger->can($level) ) {
      $self->get_logger->$level(@_);
    }
    else {
      return;
    }
  };

  return;
}

########################################################################
sub _set_default_logger {
########################################################################
  my ($self) = @_;

  if ( !$self->get_logger && !Log::Log4perl->initialized ) {

    eval {
      require Log::Log4perl;

      Log::Log4perl->import(':easy');

      Log::Log4perl->easy_init(
        { level  => $LOG4PERL_LOG_LEVELS{'info'},
          layout => $self->get_log_layout // $DEFAULT_LAYOUT_PATTERN,
          ( $self->get_log_file ? ( file => $self->get_log_file ) : () ),

        }
      );
    };
  }
  else {
    for my $level (qw(debug trace info warn error )) {
      $self->_create_stealth_logger( __PACKAGE__, $level );
      $self->_create_stealth_logger( ref($self),  $level );
    }
  }

  return $self;
}

########################################################################
sub _set_defaults {
########################################################################
  my ( $self, %options ) = @_;

  $self->set_raise_error( $self->get_raise_error // $TRUE );
  $self->set_print_error( $self->get_print_error // $TRUE );

  $self->set_use_paginator( $self->get_use_paginator // $TRUE );

  if ( !$self->get_user_agent ) {
    $self->set_user_agent( LWP::UserAgent->new );
  }

  if ( !defined $self->get_decode_always ) {
    $self->set_decode_always($TRUE);

    if ( !$self->get_decode_always && !defined $self->get_force_array ) {
      $self->set_force_array($FALSE);
    }
  }

  # most API services are POST, but using the Botocore metadata is best
  $self->set_http_method( $self->get_http_method // 'POST' );

  $self->set_protocol( $self->get_protocol() // 'https' );

  # note some APIs are global, hence an API may send '' to indicate global
  if ( !defined $self->get_region ) {
    $self->set_region( $self->get_region
        || $ENV{'AWS_REGION'}
        || $ENV{'AWS_DEFAULT_REGION'}
        || $DEFAULT_REGION );
  }

  my $debug //= ( $options{debug} || $ENV{DEBUG} );

  $options{log_level} = $debug ? 'debug' : $options{log_level} // 'info';

  $self->set_log_level( $options{log_level} );

  if ( !$self->get_credentials ) {
    if ( $self->has_keys(%options) ) {
      $self->set_credentials(
        Amazon::Credentials->new(
          { aws_secret_access_key => $options{'aws_secret_access_key'},
            aws_access_key_id     => $options{'aws_access_key_id'},
            token                 => $options{'token'},
            region                => $self->get_region,
          }
        )
      );
    }
    else {
      $self->set_credentials(
        Amazon::Credentials->new(
          order  => $options{'order'},
          region => $options{'region'},
        )
      );

      if ( !defined $self->get_region ) {
        $self->set_region( $self->get_credentials->get_region );
      }
    }
  }

  # set URL last since it contains region
  $self->_set_url;

  return $self;
}

########################################################################
sub _create_service_url {
########################################################################
  my ($self) = @_;

  my $url;

  my $botocore_metadata = $self->get_botocore_metadata;
  my $service           = $self->get_service;

  if ( $botocore_metadata && $botocore_metadata->{globalEndpoint} ) {
    $url = sprintf '%s://%s', $self->get_protocol,
      $botocore_metadata->{globalEndpoint};
  }
  else {
    my $endpoint = $self->get_endpoint_prefix || $service;

    if ( any { $_ eq $service } @GLOBAL_SERVICES ) {
      $url = sprintf $GLOBAL_URL_FMT, $self->get_protocol, $endpoint;
    }
    elsif ( $self->get_region ) {
      $url = sprintf $REGIONAL_URL_FMT, $self->get_protocol, $endpoint,
        $self->get_region;
    }
    else {
      $url = sprintf $REGIONAL_URL_FMT, $self->get_protocol, $endpoint,
        'us-east-1';
    }
  }

  return $url;
}

########################################################################
sub _set_url {
########################################################################
  my ($self) = @_;

  my $url = $self->get_url;

  if ( !$url ) {
    $url = $self->_create_service_url;
  }
  else {
    if ( $url !~ /^https?/xmsi ) {
      $url =~ s/^\///xms; # just remove leading slash...
      $url = $self->get_protocol . '://' . $url;
    }

  }

  $self->set_url($url);

  return $self;
}

########################################################################
sub _set_x_amz_target {
########################################################################
  my ( $self, $request ) = @_;

  my $target  = $self->get_target_prefix;
  my $version = $self->get_version;
  my $api     = $self->get_api;
  my $action  = $self->get_action;

  if ( !$target ) {
    $target = $version ? $api . $UNDERSCORE . $version : $api;
  }

  $target = $target . $DOT . $action;

  $self->set_target($target);

  $request->header( 'X-Amz-Target', $target );

  return $request;
}

########################################################################
sub _set_request_content {
########################################################################
  my ( $self, %args ) = @_;

  my $request      = $args{'request'};
  my $content      = $args{'content'};
  my $content_type = $args{'content_type'} || $self->get_content_type;

  DEBUG(
    sub {
      return Dumper(
        [ '_set_request_content', $self->get_http_method, \%args ] );
    }
  );

  if ( $self->get_http_method ne 'GET' || !defined $content ) {
    if ($content_type) {
      $request->content_type( $content_type . '; charset=utf-8' );
    }
    $request->content($content);
  }
  else {
    $request->uri( $request->uri . $QUESTION_MARK . $content );
  }

  return $request;
}

1;

__END__

=pod

=head1 NAME

Amazon::API - A generic base class for AWS Services

=head1 SYNOPSIS

 package Amazon::CloudWatchEvents;

 use parent qw( Amazon::API );

 # subset of methods I need
 our @API_METHODS = qw(
  ListRuleNamesByTarget
  ListRules
  ListTargetsByRule
  PutEvents
  PutRule
 );

 sub new {
   my $class = shift;

   $class->SUPER::new(
     service       => 'events',
     api           => 'AWSEvents',
     api_methods   => \@API_METHODS,
     decode_always => 1
   );
 }

 1;

Then...

 my $rules = Amazon::CloudWatchEvents->new->ListRules({});

...or

  my $rules = Amazon::API->new(
    { service => 'events',
      api     => 'AWSEvents',
    }
  )->invoke_api( 'ListRules', {} );

=head1 DESCRIPTION

Generic class for constructing AWS API interfaces. Typically used as a
parent class, but can be used directly. This package can also
generates stubs for Amazon APIs using the Botocore project's
metadata. (See L</BOTOCORE SUPPORT>).

=over 5

=item * See L</IMPLEMENTATION NOTES> for using C<Amazon::API>
directly to call AWS services.

=item * See
L<Amazon::CloudWatchEvents|https://github.com/rlauer6/perl-Amazon-CloudWatchEvents/blob/master/src/main/perl/lib/Amazon/CloudWatchEvents.pm.in>
for an example of how to use this module as a parent class.

=item * See L<Amazon::API::Botocore::Pod> for information regarding
how to automatically create Perl classes for AWS services using
Botocore metadata.

=back

=head1 BACKGROUND AND MOTIVATION

A comprehensive Perl interface to AWS services similar to the I<Botocore>
library for Python has been a long time in coming. The PAWS project
has been attempting to create an always up-to-date AWS interface with
community support. Some however may find that project a little heavy
in the dependency department. If you are looking for an extensible
(albeit spartan) method of invoking a subset of services with a lower
dependency count, you might want to consider C<Amazon::API>.

Think of this class as a DIY kit for invoking B<only> the methods you
need for your AWS project.  Using the included C<amazon-api> utility
however, you can also roll your own complete Amazon API classes that
include support for serializing requests and responses based on
metadata provided by the Botocore project. The classes you create with
C<amazon-api> include full documentation as pod. (See L</BOTOCORE
SUPPORT> for more details).

=over 5

I<NOTE:> The original L<Amazon::API> was written in 2017 as a I<very>
lightweight way to call a handfull of APIs. The evolution of the
module was based on discovering, without much documentation or help,
the nature of Amazon APIs. In retrospect, even back then, it would
have been easier to consult the Botocore project and decipher how that
project managed to create a library from the metadata. Fast forward
to 2022 and L<Amazon::API> is now capable of using the Botocore
metadata in order to, in most cases, correctly call any AWS service.
The L<Amazon::API> module can still be used without the assistance of
Botocore metadata, but it works a heckuva lot better with it.

=back

You can use L<Amazon::API> in 3 different ways:

=over 5

=item Take the Luddite approach

  my $queues = Amazon::API->new(
   {
    service     => 'sqs',
    http_method => 'GET'
   })->invoke_api('ListQueues');

=item Build your own API classes with just what you need

 package Amazon::API::SQS;

 use strict;
 use warnings;

 use parent qw( Amazon::API );

 our @API_METHODS = qw(
   ListQueues
   PurgeQueue
   ReceiveMessage
   SendMessage
 );

 sub new {
   my ( $class, @options ) = @_;
   $class = ref($class) || $class;

   my %options = ref( $options[0] ) ? %{ $options[0] } : @options;

   return $class->SUPER::new(
     { service       => 'sqs',
       http_method   => 'GET',
       api_methods   => \@API_METHODS,
       decode_always => 1,
       %options
     }
   );
 }

 1;

 use Amazon::API::SQS;
 use Data::Dumper;

 my $sqs = Amazon::API::SQS->new;
 print Dumper($sqs->ListQueues);

=item Use the Botocore metadata to build classes for you

 amazon-api -s sqs create-stubs
 amazon-api -s sqs create-shapes

 perl -I . -MData::Dumper -MAmazon::API:SQS -e 'print Dumper(Amazon::API::SQS->new->ListQueues);'

=over 5

I<NOTE:> In order to use Botocore metadata you must clone the Botocore
repository and point the utility to the repo.

Clone the Botocore project from GitHub:

 mkdir ~/git
 cd git
 git clone https:://github.com/boto/botocore.git

Generate stub classes for the API and shapes:

 amazon-api -b ~/git/botocore -s sqs -o ~/lib/perl5 create-stubs
 amazon-api -b ~/git/botocore -s sqs -o ~/lib/perl5 create-shapes

 perldoc Amazon::API::SQS

See L<Amazon::API::Botocore::Pod> for more details regarding building
stubs and shapes.

=back

=back

=head1 THE APPROACH

Essentially, most AWS APIs are RESTful services that adhere to a
common protocol, but differences in services make a single solution
difficult. All services more or less adhere to this framework:

=over 5

=item 1. Set HTTP headers (or query string) to indicate the API and
method to be invoked

=item 2. Set credentials in the header

=item 3. Set API specific headers

=item 4. Sign the request and set the signature in the header

=item 5. Optionally send a payload of parameters for the method being invoked

=back

Specific details of the more recent AWS services are well documented,
however early services were usually implemented as simple HTTP
services that accepted a query string. This module attempts to account
for most of the nuances involved in invoking AWS services and
provide a fairly generic way of invoking these APIs in the most
lightweight way possible.

Using L<Amazon::API> as a generic, lightweight module, naturally does
not provide nuanced support for individual AWS services. To use this
class in that manner for invoking the AWS APIs, you need to be very
familiar with the specific API requirements and responses and be
willng to invest time reading the documentation on Amazon's website.
The payoff is that you can probably use this class to call I<any> AWS
API without installing a large number of dependencies.

If you don't mind a few extra dependencies and overhead, you should
generate the stub APIs and support classes using the C<amazon-api>
utility. The stubs and shapes produced by the utility will serialize
and deserialize requests and responses correctly by using the Botocore
metadata. Botocore metadata provides the necessary information to
create classes that can successfully invoke all of the Amazon APIs.

A good example of creating a quick and dirty interface to CloudWatch
Events can be found here:

L<Amazon::CloudWatchEvents|https://github.com/rlauer6/perl-Amazon-CloudWatchEvents/blob/master/src/main/perl/lib/Amazon/CloudWatchEvents.pm.in>

And invoking some of the APIs can be as easy as:

  Amazon::API->new(
    service     => 'sqs',
    http_method => 'GET'
  }
  )->invoke_api('ListQueues');

=head1 BOTOCORE SUPPORT

A new experimental module ( L<Amazon::API::Botocore>) is now included
in this project.

B<!!CAUTION!!>

I<Support for API calls using the Botocore metadata may be buggy and
is subject to change. It may not be suitable for production
environments at this time.>

Using Botocore metadata and the utilities in this project, you can
create Perl classes that simplify calling AWS services.  After
creating service classes and shape objects from the Botocore metadata
calling AWS APIs will look something like this:

 use Amazon::API::SQS;

 my $sqs = Amazon::API::SQS->new;
 my $rsp = $sqs->ListQueues();

The L<Amazon::API::Botocore> module augments L<Amazon::API> by using
Botocore metadata for determining how to call individual services and
serialize parameters passed to its API methods. A utility (C<amazon-api>)
is provided that can generate Perl classes for all AWS services using
the Botocore metadata.

Perl classes that represent AWS data structures (aka shapes) that are
passed to or returned from services can also be generated. These
classes allow you to call all of the API methods for a given service
using simple Perl objects that are serialized correctly for a specific
method.

Service classes are subclassed from C<Amazon::API> so their C<new()>
constructor takes the same arguments as C<Amazon::API::new()>.

 my $credentials = Amazon::Credential->new();

 my $sqs = Amazon::API::SQS->new( credentials => $credentials );

If you are going to use the Botocore support and automatically
generate API classes you I<must also> create the data structure classes
that are used by each service. The Botocore based APIs will use these
classes to serialize requests and responses.

For more information on generating API classes, see
L<Amazon::API::Botocore::Pod>.

=head2 Serialization Errors

With little documentation to go on, Interpretting the Botocore
metadata and deducing how to serialize shapes from Perl objects has
been a difficult task. It's likely that there are still some edge
cases and bugs lurking in the serialization methods. Accordingly,
starting with version 1.4.5, serialization exceptions or exceptions
that occur while attempting to decode a response, will result in the
raw response being returned to the caller. The idea being that getting
something back that allows you figure out what to do with the response
might be better than receiving an error.

OTOH, you might want to see the error, report it, or possibly
contribute to its resolution.  You can prevent errors from being
surpressed by setting the C<raise_serializtion_errors> to a true
value. The default is I<false>.

I<Throughout the rest of this documentation a request made using one
of the classes created by the Botocore support scripts will be
referred to as a B<Botocore request> or B<Botocore API>.>

=over 5

=item *

=back

=head1 ERRORS

When an error is returned from an API request, an exception class
(C<Amazon::API::Error>) will be raised if C<raise_error> has been set
to a true value. Additionally, a detailed error message will be
displayed if C<print_error> is set to true.

See L<Amazon::API::Error> for more details.

=head1 METHODS AND SUBROUTINES

I<Reminder: You can mostly ignore this part of the documentation when
you are leveraging Botocore to generate your API classes.>

=head2 new

 new(options)

All options are described below. C<options> can be a list of
key/values or hash reference.

=over 5

=item action

The API method. Normally, you would not set C<action> when you
construct your object. It is set when you call the C<invoke_api>
method or automatically set when you call one of the API stubs created
for you.

Example: 'PutEvents'

=item api

The name of the AWS service. See L</IMPLEMENTATION NOTES> for a
detailed explanation of when to set this value.

Example: 'AWSEvents'

=item api_methods

A reference to an array of method names for the API.  The new
constructor will automatically create methods for each of the method
names listed in the array.

The methods that are created for you are nothing more than stubs that
call C<invoke_api>. The stub is a convenience for calling the
C<invoke_api> method as shown below.


 my $api = Amazon::CloudWatch->new;

 $api->PutEvents($events);

...is equivalent to:

 $api->invoke_api->('PutEvents', $events);

Consult the Amazon API documentation for the service to determine what
parameters each action requires.

=item aws_access_key_id

Your AWS access key. Both the access key and secret access key are
required if either is passed. If no credentials are passed, an attempt
will be made to find credentials using L<Amazon::Credentials>. Note
that you may need to pass C<token> as well if you are using temporary
credentials.

=item aws_secret_access_key

Your AWS secret access key.

=item content_type

Default content for parameters passed to the C<invoke_api()>
method. If you do not provide this value, a default content type will
be selected based on the service's protocol.

 query     => application/x-www-form-urlencoded
 rest-json => application/x-amz-json-1.1
 json      => application/json
 rest-xml  => application/xml

=item credentials (optional)

Accessing AWS services requires credentials with sufficient privileges
to make programmatic calls to the APIs that support a service.  This
module supports three ways that you can provide those credentials.

=over 10

=item 1. Pass the credentials directly.

Pass the values for the credentials (C<aws_access_key_id>,
C<aws_secaret_access_key>, C<token>) when you call the C<new> method.
A session token is typically required when you have assumed
a role, you are using the EC2's instance role or a container's role.

=item 2. Pass a class that will provide the credential keys.

Pass a reference to a class that has I<getters> for the credential
keys. The class should supply I<getters> for all three credential keys.

Pass the reference to the class as C<credentials> in the constructor
as shown here:

 my $api = Amazon::API->new(credentials => $credentials_class, ... );

=item 3. Use the default C<Amazon::Credentials> class.

If you do not explicitly pass credentials or do not pass a class that
will supply credentials, the module will use the
C<Amazon::Credentials> class that attempts to find credentials in the
I<environment>, your I<credentials file(s)>, or the I<container or
instance role>.  See L<Amazon::Credentials> for more details.

I<NOTE: The latter method of obtaining credentials is probably the
easiest to use and provides the most succinct and secure way of
obtaining credentials.>

=back

=item debug

Set debug to a true value to enable debug messages. Debug mode will
dump the request and response from all API calls. You can also set the
environment variable DEBUG to enable debugging output. Set the debug
value to '2' to increase the logging level.

default: false

=item decode_always

Set C<decode_always> to a true value to return Perl objects from API
method calls. The default is to return the raw output from the call.
Typically, API calls will return either XML or JSON encoded objects.
Setting C<decode_always> will attempt to decode the content based on
the returned content type.

default: false

=item error

The most recent result of an API call. C<undef> indicates no error was
encountered the last time C<invoke_api> was called.

=item http_method

Sets the HTTP method used to invoke the API. Consult the AWS
documentation for each service to determine the method utilized. Most
of the more recent services utilize the POST method, however older
services like SQS or S3 utilize GET or a combination of methods
depending on the specific method being invoked.

default: POST

=item last_action

The last method call invoked.

=item print_error

Setting this value to true enables a detailed error message containing
the error code and any messages returned by the API when errors occur.

default: true

=item protocol

One of 'http' or 'https'.  Some Amazon services do not support https
(yet).

default: https

=item raise_error

Setting this value to true will raise an exception when errors
occur. If you set this value to false you can inspect the C<error>
attribute to determine the success or failure of the last method call.

 $api->invoke_api('ListQueues');

 if ( $api->get_error ) {
   ...
 }

default: true

=item region

The AWS region. Pass an empty string if the service is a global
service that does not require or want a region.

default: $ENV{'AWS_REGION'}, $ENV{'AWS_DEFAULT_REGION'}, 'us-east-1'

=item response

The HTTP response from the last API call.

=item service

The AWS service name. Example: C<sqs>. This value is used as a prefix
when constructing the the service URL (if not C<url> attribute is set).

=item service_url_base

Deprecated, use C<service>

=item token

Session token for assumed roles.

=item url

The service url.  Example: https://events.us-east-1.amazonaws.com

Typically this will be constructed for you based on the region and the
service being invoked. However, you may want to set this manually if
for example you are using a service like
<LocalStack|https://localstack.cloud/> that mocks AWS API calls.

 my $api = Amazon::API->new(service => 's3', url => 'http://localhost:4566/');

=item user_agent

Your own user agent object.  Using
C<Furl>, if you have it avaiable may result in faster response.

default: C<LWP::UserAgent>

=item version

Sets the API version.  Some APIs require a version. Consult the
documentation for individual services.

=back

=head2 invoke_api

 invoke_api(action, [parameters], [content-type], [headers]);

or using named parameters...

 invoke_api({ action => args, ... } )

Invokes the API with the provided parameters.

=over 5

=item action

API name.

=item parameters

Parameters to send to the API. C<parameters> can be a scalar, a hash
reference or an array reference. See the discussion below regarding
C<content-type> and how C<invoke_api()> formats parameters before
sending them as a payload to the API.

You can use the C<param_n()> method to format query string arguments
that are required to be in the I<param.n> notation. This is about the
best documentation I have seen for that format. From the AWS
documentation...

=over 10

Some actions take lists of parameters. These lists are specified using
the I<param.n> notation. Values of n are integers starting from 1. For
example, a parameter list with two elements looks like this:

&AttributeName.1=first

&AttributeName.2=second

=back

An example of using this notation is to set queue attributes when
creating an SQS queue.

 my $attributes = { Attributes => [ { Name => 'VisibilityTimeout', Value => '100' } ] };
 my @sqs_attributes= Amazon::API::param_n($attributes);

 eval {
   $sqs->CreateQueue([ 'QueueName=foo', @sqs_attributes ]);
 };

See L</param_n> for more details.

=item content-type

If you pass the C<content-type> parameter, it is assumed that the parameters are
the actual payload to be sent in the request (unless the parameter is a reference).

The C<parameters> will be converted to a JSON string if the
C<parameters> value is a hash reference.  If the C<parameters> value
is an array reference it will be converted to a query string (Name=Value&...).

To pass a query string, you should send an array of key/value
pairs, or an array of scalars of the form C<Name=Value>.

 [ { Action => 'DescribeInstances' } ]
 [ 'Action=DescribeInstances' ]

=item headers

Array reference of key/value pairs representing additional headers to
send with the request.

=back

=head2 decode_response

Boolean that indicates whether or not to deserialize the most recent
response from an invoked API based on the I<Content-Type> header
returned.  If there is no I<Content-Type> header, then the method will
try to decode it first as a JSON string and then as an XML string. If
both of those fail, the raw content is returned.

You can enable or disable deserializing responses globally by setting
the C<decode_always> attribute when you call the C<new> constructor.

default: true

By default, `Amazon::API` will retrieve all results for Botocore based
API calls that require pagination. To turn this behavior off, set
C<use_paginator> to a false value when you instantiate the API
service.

 my $ec2 = Amazon::API->new(use_paginator => 0);

You can also use the L</paginator> method to retrieve all results from Botocore requests that implement pagination.

=head2 submit

 submit(options)

I<This method is used internally by C<invoke_api> and normally should
not be called by your applications.>

C<options> is a reference to a hash of options:

=over 5

=item content

Payload to send.

=item content_type

Content types we have seen used to send values to AWS APIs:

 application/json
 application/x-amz-json-1.0
 application/x-amz-json-1.1
 application/x-www-form-urlencoded

Check the documentation for the individual APIs for the correct
content type.

=item headers

Array reference of key/value pairs that represent additional headers
to send with the request.

=back

=head1 EXPORTED METHODS

=head2 get_api_service

 get_api_service(api, options)

Convenience routine that will return an API instance.

 my $sqs = get_api_service 'sqs';

Equivalent to:

 require Amazon::API::SQS;

 my $sqs = Amazon::API::SQS->new(%options);

=over 5

=item api

The service name. Example: route53, sqs, sns

=item options

list of key/value pairs passed to the new constructor as options

=back

=head2 create_url_encoded_content

 create_urlencoded_content(parameters, action, version)

Returns a URL encoded query string. C<parameters> can be any of SCALAR, ARRAY, or HASH. See below.

=over 5

=item parameters

=over 5

=item SCALAR

Query string to encode (x=y&w=z..)

=item ARRAY

Can be one of:

=over 5

=item * Array of hashes where the keys are the query string variable and the value is the value of that variable

=item * Array of strings of the form "x=y"

=item * An array of key/value pairs - qw( x y w z )

=back

=item HASH

Key/value pairs. If value is an array it is assumed to be a list of hashes

=back

=item action

The method being called. For some query type APIs an Action query variable is required.

=item version

The WSDL version for the API. Some query type APIs require a Version query variable.

=back

=head2 paginator

 paginator(service, api, request)

Returns an array containing the results of an API call that requires
pagination,


 my $result = paginator($ec2, 'DescribeInstances', { MaxResults => 10 });

=head2 param_n

 param_n(parameters)

Format parameters in the "param.n" notation.

C<parameters> should be a hash or array reference.

A good example of a service that uses this notation is the
I<SendMessageBatch> SQS API call.

The sample request can be found here:

L<SendMessageBatch|https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html>


 https://sqs.us-east-2.amazonaws.com/123456789012/MyQueue/
 ?Action=SendMessageBatch
 &SendMessageBatchRequestEntry.1.Id=test_msg_001
 &SendMessageBatchRequestEntry.1.MessageBody=test%20message%20body%201
 &SendMessageBatchRequestEntry.2.Id=test_msg_002
 &SendMessageBatchRequestEntry.2.MessageBody=test%20message%20body%202
 &SendMessageBatchRequestEntry.2.DelaySeconds=60
 &SendMessageBatchRequestEntry.2.MessageAttribute.1.Name=test_attribute_name_1
 &SendMessageBatchRequestEntry.2.MessageAttribute.1.Value.StringValue=test_attribute_value_1
 &SendMessageBatchRequestEntry.2.MessageAttribute.1.Value.DataType=String
 &Expires=2020-05-05T22%3A52%3A43PST
 &Version=2012-11-05
 &AUTHPARAMS

To produce this message you would pass the Perl object below to C<param_n()>:

 my $message = {
   SendMessageBatchRequestEntry => [
     { Id          => 'test_msg_001',
       MessageBody => 'test message body 1'
     },
     { Id               => 'test_msg_002',
       MessageBody      => 'test message body 2',
       DelaySeconds     => 60,
       MessageAttribute => [
         { Name  => 'test_attribute_name_1',
           Value =>
             { StringValue => 'test_attribute_value_1', DataType => 'String' }
         }
       ]
     }
   ]
 };

=head1 CAVEATS

=over 5

=item If you are calling an API that does not expect parameters (or all of
them are optional and you do not pass a parameter) the default is to
pass an empty hash..

  $cwe->ListRules();

would be equivalent to...

  $cwe->ListRules({});

I<CAUTION! This may not be what the API expects! Always consult
the AWS API for the service you are are calling.>

=back

=head1 GORY DETAILS

If you using the Botocore APIs you can probably ignore this section.

=head2 X-Amz-Target

Most of the newer AWS APIs are invoked as HTTP POST operations and
accept a header C<X-Amz-Target> in lieu of the CGI parameter C<Action>
to specify the specific API action. Some APIs also want the version in
the target, some don't. There is sparse documentation about the
nuances of using the REST interface I<directly> to call AWS APIs.

When invoking an API, the class uses the C<api> value to indicate
that the action should be set in the C<X-Amz-Target> header.  We also
check to see if the version needs to be attached to the action value
as required by some APIs.

  if ( $self->get_api ) {
    if ( $self->get_version) {
      $self->set_target(sprintf('%s_%s.%s', $self->get_api, $self->get_version, $self->get_action));
    }
    else {
      $self->set_target(sprintf('%s.%s', $self->get_api, $self->get_action));
    }

    $request->header('X-Amz-Target', $self->get_target());
  }

DynamoDB and KMS seem to be able to use this in lieu of query
variables C<Action> and C<Version>, although again, there seems to be
a lot of inconsistency (and sometimes flexibility) in the APIs.
DynamoDB uses DynamoDB_YYYYMMDD.Action while KMS does not require the
version that way and prefers TrentService.Action (with no version).
There is no explanation in any of the documentations I have been able
to find as to what "TrentService" might actually mean.  Again, your
best approach is to read Amazon's documentation and look at their
sample requests for guidance.  You can also look to the L<Botocore
project|https://github.com/boto/botocore> for information regarding
the service.  Checkout the F<service-2.json> file within the
sub-directory F<botocore/botocore/data/{api-version}/{service-name}>
which contains details for each service.

In general, the AWS API ecosystem is very organic. Each service seems
to have its own rules and protocol regarding what the content of the
headers should be.

As noted, this generic API interface tries to make it possible to use
one class C<Amazon::API> as a sort of gateway to the APIs. The most
generic interface is simply sending query variables and not much else
in the header.  Services like EC2 conform to that protocol and can be
invoked with relatively little fanfare.

 use Amazon::API;
 use Data::Dumper;

 print Dumper(
   Amazon::API->new(
     service => 'ec2',
     version => '2016-11-15'
   )->invoke_api('DescribeInstances')
 );

Note that invoking the API in this fashion, C<version> is
required.

For more hints regarding how to call a particular service, you can use
the AWS CLI with the --debug option.  Invoke the service using the CLI
and examine the payloads sent by the botocore library.

=head2 Rolling a New API

The L<Amazon::API> class will stub out methods for the API if you pass
an array of API method names.  The stub is equivalent to:

 sub some_api {
   my $self = shift;

   $self->invoke_api('SomeApi', @_);
 }

Some will also be happy to know that the class will create an
equivalent I<CamelCase> version of the method.

As an example, here is a possible implementation of
C<Amazon::CloudWatchEvents> that implements one of the API calls.

 package Amazon::CloudWatchEvents;

 use parent qw/Amazon::API/;

 sub new {
   my ($class, $options) = @_;

   my $self = $class->SUPER::new(
     { %{$options},
       api         => 'AWSEvents',
       service     => 'events',
       api_methods => [qw( ListRules )],
     }
   );

   return $self;
 }

Then...

 use Data::Dumper;

 print Dumper(Amazon::CloudWatchEvents->new->ListRules({}));

Of course, creating a class for the service is optional. It may be
desirable however to create higher level and more convenient methods
that aid the developer in utilizing a particular API.

=head2 Overriding Methods

Because the class does some symbol table munging, you cannot easily
override the methods in the usual way.

 sub ListRules {
   my $self = shift;
   ...
   $self->SUPER::ListRules(@_)
 }

Instead, you should re-implement the method as implemented by this
class.

 sub ListRules {
   my $self = shift;
   ...
   $self->invoke_api('ListRules', @_);
 }

=head2 Content-Type

Yet another piece of evidence that suggests the I<organic> nature of
the Amazon API ecosystem is their use of different C<Content-Type>
headers.  Some of the variations include:

 application/json
 application/x-amz-json-1.0
 application/x-amz-json-1.1
 application/x-www-form-urlencoded

Accordingly, the C<invoke_api()> method can be passed the
C<Content-Type> or will try to make its I<best guess> based on the
service protocol. It guesses using the following decision tree:

You can also set the default content type used for the calling service
by passing the C<content_type> option to the constructor.

  $class->SUPER::new(
    content_type => 'application/x-amz-json-1.1',
    api          => 'AWSEvents',
    service      => 'events'
  );

=head2 ADDITIONAL HINTS

=over 5

=item * Bad Request

If you send the wrong headers or payload you're liable to get a 400
Bad Request. You may also get other errors that can be misleading when
you send incorrect parameters. When in doubt compare your requests to
requests from the AWS CLI using the C<--debug> option.

=over 10

=item 1. Set the C<debug> option to true to see the request object and
the response object from C<Amazon::API>.

=item 2. Excecute the AWS CLI with the --debug option and compare the
request and response with that of your calls.

=back

=item * Payloads

Pay attention to the payloads that are required by each service.  B<Do
not> assume that sending nothing when you have no parameters to pass
is correct. For example, the C<ListSecrets> API of SecretsManager
requires at least an empty JSON object.

 $api->invoke_api('ListSecrets', {});

Failure to send at least an empty JSON object will result in a 400
response.

=back

=head1 VERSION

This documentation refers to version 2.0.8 of C<Amazon::API>.

=head1 DIAGNOSTICS

To enable diagnostic output set C<debug> to a true value when calling
the constructor. You can also set the C<DEBUG> environment variable to a
true value to enable diagnostics.

=head2 Logging

By default L<Amazon::API> uses L<Log::Log4perl>'s Stealth loggers to
log at the DEBUG and TRACE levels. Setting the environment variable
DEBUG to some value or passing a true value for C<debug> in the
constructor will trigger verbose logging.

If you pass a logger to the constructor, C<Amazon::API> will attempt
to use that if it has the appropriate logging level methods (error,
warn, info, debug, trace). If L<Log::Log4perl> is unavailable and you
do not pass a logger, logging is essentially disabled at any level.

If, for some reason you set the enviroment variable DEBUG to a true
value but do not want C<Amazon::API> to log messages you can turn off
logging as shown below:

 my $ec2 = Amazon::API::EC2->new();

 $ec2->set_log_level('fatal');

=head1 DEPENDENCIES

=over 5

=item * L<Amazon::Signature4>

=item * L<Amazon::Credentials>

=item * L<Class::Accessor::Fast>

=item * L<Date::Format>

=item * L<HTTP::Request>

=item * L<JSON>

=item * L<LWP::UserAgent>

=item * L<List::Util>

=item * L<ReadonlyX>

=item * L<Scalar::Util>

=item * L<Time::Local>

=item * L<XML::Simple>

=back

...and possibly others.

=head1 INCOMPATIBILITIES

=head1 BUGS AND LIMITATIONS

This module has not been tested on Windows OS. Please report any
issues found by opening an issue here:

L<https://github.com/rlauer6/perl-Amazon-API/issues>

=head1 FAQs

=head2 Why should I use this module instead of PAWS?

Maybe you shouldn't. PAWS is a community supported project and
may be a better choice for most people. The programmers who created
PAWS are luminaries in the pantheon of Perl programming (alliteration
intended). If you want to use something a little lighter in the
dependency department however, and perhaps only need to invoke a
single service, L<Amazon:API> may be the right choice.

=head2 Does it perform better than PAWS?

Probably. But individual API calls to Amazon services have their own
performance characteristics and idiosyncracies.  The overhead
introduced by this module and PAWS may be insignificant compared to
the API performance itself. YMMV.

=head2 Does this work for all APIs?

I don't know. Probably not. Would love to hear your
feedback. L<Amazon::API> has been developed based on my needs.
Although I have tested it on many APIs, there may still be some cases
that are not handled properly and I am still deciphering the nuances
of flattening, boxing and serializing objects to send to Amazon APIs. The newer versions of this module using Botocore metadata have become increasingly reliable.

Amazon APIs are not created equal, homogenous or invoked in the the
same manner. Some accept parameters as a query strings, some
parameters are embedded in the URI, some are sent as JSON payloads and
others as XML. Content types for payloads are all over the map.
Likewise with return values.

Luckily, the Botocore metadata describes the protocols, parameters and
return values for all APIs. The Botocore metadata is quite amazing
actually. It is used to provide information to the Botocore library
for calling any of the AWS services and even for creating
documentation!

L<Amazon::API> can use that information for creating the Perl classes
that invoke each API but may not interpret the metadata correctly in
all circumstances. Bugs almost certainly exist. :-( Did I mention help
is welcome?

If you want to use this to invoke S3 APIs, don't. I
haven't tried it and I'm pretty sure it would not work anyway. There are
modules designed specifically for S3; L<Amazon::S3>,
L<Net::Amazon::S3>. Use them.

=head2 Do I have to create the shape classes when I generate stubs for
a service?

Possibly. If you create stubs manually, then you do not need the shape
classes. If you use the scripts provide to create the API stubs using
Botocore metadata, then yes, you must create the shapes so that the
Botocore API methods know how to serialize requests. Note that you can
create the shape stubs using the Botocore metadata while not creating
the API services. You might want to do that if you want a
lean stub but want the benefits of using the shape stubs for
serialization of the parameters.

If you produce your stubs manually and do not create the shape stubs,
then you must pass parameters to your API methods that are ready to be
serialized by L<Amazon::API>.  Creating data structures that will be
serialized correctly however is done for you if you use the shape
classes.  For example, to create an SQS queue using the shape stubs,
you can call the C<CreateQueue> API method as describe in the Botocore
documentation.

 $sqs->CreateQueue(
  { QueueName => $queue_name,
    Tag       => [ { Name => 'my-new-queue' }, { Env => 'dev' } ],
    Attribute => [ { VisibilityTimeout => 40 }, { DelaySeconds => 60 } ]
  });

If you do not use the shape classes, then you must pass the arguments
in the form that will eventually be serialized in the correct manner
as a query string.

 $sqs->CreateQueue([
  'QueueName=foo',
  'Attributes.1.Value=100',
  'Attributes.1.Name=VisibilityTimeout',
  'Tag.1.Key=Name',
  'Tag.1.Value=foo',
  'Tag.2.Key=Env',
  'Tag.2.Value=dev'
 ]);

=head2 This code is wonky. Why?

This code has evolved over the years from being I<ONLY> a way to make
RESTful calls to Amazon APIs to incorporating the use of the Botocore
metadata. It I<was> one person's effort to create a somewhat
lightweight interface to a few AWS APIs. As my own needs have changed
and my knowledge of AWS services has increased it became obvious that
using the Botocore metadata would yield far superior results. Still,
I'm grokking the Botocore metadata intuitively without any help. My
interpretations may be off. Help? Pull requests welcomed.

=head2 How do I pass AWS credentials to the API?

There is a bit of magic here as L<Amazon::API> will use
L<Amazon::Credentials> transparently if you do not explicitly pass the
credentials object. I've taken great pains to try to make the
aforementioned module somewhat useful and I<secure>.

See L<Amazon::Credentials>.

=head2 Can I use more than one set of credentials to invoke different APIs?

Yes. See L<Amazon::Credentials>.

=head2 OMG! You update this too frequently!

Yeah, there are a lot of bugs. Should I stop fixing them?

=head2 Why are you using XML::Simple when it clearly says "DO NOT"?

It's simple. And it seems easier to build than other modules that
almost do the same thing.

=head2 I tried to use this with XYZ service and it barfed. What do I do?

That's not too surprising. There are several reasons why your call
might not have worked.

Did you enable debugging? Tracing?

=over 5

=item * You passed bad data

Take a look at the data you passed, how was it serialized and
ultimately passed to the API?

=item * You didn't read the docs and passed bad data

 amazon-api -s sqs CreateQueue

=item * The serialization of Amazon::API::Botocore::Shape is busted

I have not tested every class generated for every API. You may find
that some API methods return C<Bad Request> or do not serialize the
results in the manner expected. Requests are serialized based on the
metadata found in the Botocore project. There lie the clues for each
API (protocol, end points, etc) and the models (shapes) for requests
and response elements.

Some requests require a query string, some an XML or JSON payload. The
Botocore based API classes use the metadata to determine how to send a
request and how to interpret the results. This module uses
L<XML::Simple> or L<JSON> to parse the results. It then uses the
C<Amazon::API::Bottocore::Serializer> to turn the parsed results into
a Perl object that respresents the response shape. It's likely that
this module has bugs and the shapes returned might not exactly match the
return response from the C<aws> command line interface version.

What should be returned by an API request method is documented in
these modules and Amazon's CLI.

 perldoc Amazon::API::Botocore::Shape::EC2:DescribeInstancesRequest

If you find this project's serializer deficient, please log an issue
and I will attempt to address it.

=back

=head1 LICENSE AND COPYRIGHT

This module is free software. It may be used, redistributed and/or
modified under the same terms as Perl itself.

=head1 SEE OTHER

L<Amazon::Credentials>, L<Amazon::API::Error>, L<AWS::Signature4>, L<Amazon::API::Botocore>

=head1 AUTHOR

Rob Lauer - <[e-mail]>

=cut