JobScheduler.pm

filter: Statementstmt 76.4% 68 / 89 Branchbran 50.0% 14 / 28 Conditioncond 33.3% 2 / 6 MC/DCmcdc 0.0% 0 / 4 Subroutinesub 100% 14 / 14 Podpod 50.0% 1 / 2 totaltotal 69.2% 99 / 143 CC 18 SCAR 17.7
CC
18
Cov
68%
_verify_configuration
16.8
_build__clients
8.7
submit_job
3.0
CRAP 5.9
↑ Summary JobScheduler/Client/TheSchwartz.pm »
1 package Dancer2::Plugin::JobScheduler;
2 6 ▶use strict;
Subroutine BEGIN: called CC=1 SCAR=0.0
3 6 ▶use warnings;
Subroutine BEGIN: called CC=1 SCAR=0.0
4  
5 # ABSTRACT: Plugin for Dancer2 web app to send and query jobs in different job schedulers
6  
7 6 our $VERSION = '0.007';
8  
9 =pod
10  
11 =encoding utf8
12  
13 =cut
14  
15 #Lots of subs not covered by Pod::Coverage
16 #because they are inherited from Dancer2::Plugin.
17  
18 =for Pod::Coverage ClassHooks PluginKeyword dancer_app execute_plugin_hook hook
19 =for Pod::Coverage list_jobs on_plugin_import plugin_args plugin_setting
20 =for Pod::Coverage register register_hook register_plugin request submit_job var
21  
22 =head1 NAME
23  
24 Dancer2::Plugin::JobScheduler - Plugin for Dancer2 web app to send and query jobs in different job schedulers
25  
26 =head1 STATUS
27  
28 This module is currently being developed so changes in the API are possible,
29 though not likely.
30  
31 =head1 SYNOPSIS
32  
33 use Dancer2;
34 BEGIN {
35 my %plugin_config = (
36 default => 'theschwartz',
37 schedulers => {
38 theschwartz => {
39 client => 'TheSchwartz',
40 parameters => {
41 handle_uniqkey => 'acknowledge',
42 dbh_callback => 'Database::ManagedHandle->instance',
43 databases => {
44 theschwartz_db1 => {
45 prefix => q{schema_name.},
46 },
47 }
48 }
49 }
50 }
51 );
52 set log => 'debug';
53 set plugins => {
54 JobScheduler => \%plugin_config,
55 };
56 }
57 use Dancer2::Plugin::JobScheduler;
58  
59 set serializer => 'JSON';
60  
61 get q{/submit_job} => sub {
62 my %r = submit_job(
63 client => 'theschwartz',
64 job => {
65 task => 'task1',
66 args => { name => 'My Name', age => 123 },
67 opts => {},
68 },
69 );
70 return to_json(\%r);
71 };
72  
73 get q{/list_jobs} => sub {
74 my %r = list_jobs(
75 client => 'theschwartz',
76 search_params => {
77 task => 'task1',
78 },
79 );
80 return to_json(\%r);
81 };
82  
83  
84 =head1 DESCRIPTION
85  
86 =for stopwords Gearman
87  
88 Dancer2::Plugin::JobScheduler is an interface to access different
89 L<job schedulers|https://en.wikipedia.org/wiki/Job_scheduler> in L<Dancer2>
90 web app.
91  
92 Dancer2::Plugin::JobScheduler provides an interface to submit jobs
93 and query jobs currently in queue. As a L<Dancer2> plugin, it creates two
94 new commands in the web app: C<submit> and C<list_jobs>.
95  
96 These commands abstract away the complexity of interfacing with a job scheduler.
97 User does not need to even know which job scheduler the website is using,
98 unless there are several in use, in which case they can be identified
99 by a short id.
100  
101 A job scheduler is used to off-load CPU power or time consuming tasks from the web app
102 so that it can answer user's web requests as quickly as possible. One example
103 of tasks like these is sending a confirmation email. The email can be sent
104 after a delay, so the sending is scheduled off to a worker server somewhere else
105 where it will not burden the web app.
106  
107 There are many job schedulers, and since their operation is separated from
108 Dancer2 web app, they can be implemented in any language, not just Perl,
109 the language of Dancer2.
110  
111 Perl has several job schedulers, too. Most notable ones are
112 L<TheSchwartz> and L<Minion>. Also L<Gearman|http://gearman.org/>
113 is often mentioned among job schedulers because Gearman's
114 L<original version|https://en.wikipedia.org/wiki/Gearman>
115 was written in Perl though later it was rewritten in C.
116  
117 Dancer2::Plugin::JobScheduler supports the following job schedulers:
118  
119 =over 8
120  
121 =item L<TheSchwartz>
122  
123 =back
124  
125 =head2 Using Dancer2::Plugin::JobScheduler with Dancer2::Plugin::Database
126  
127 If you are doing database operations in a Dancer2 web app, you are probably
128 using L<Dancer2::Plugin::Database> to get the database handle you need.
129 You can use the same database handle with Dancer2::Plugin::JobScheduler.
130 This can be especially useful if you are doing database transactions.
131 If a transaction fails, you would probably want the scheduled job to
132 be removed as well.
133  
134 You need to configure the databases just like you would normally
135 but without dbh_callback. You would provide the handle callback at the point
136 of calling C<submit_job()> or C<list_jobs()>.
137  
138 use Dancer2;
139 use HTTP::Status qw( :constants status_message );
140 BEGIN {
141 set log => 'debug';
142 set plugins => {
143 JobScheduler => {
144 default => 'theschwartz',
145 schedulers => {
146 theschwartz => {
147 client => 'TheSchwartz',
148 parameters => {
149 databases => {
150 dancer_app_db => { },
151 },
152 dbh_callback => 'replaced-when-calling',
153 }
154 }
155 },
156 },
157 Database => {
158 connections => {
159 dancer_app_db => {
160 driver => SQLite,
161 database => '/tmp/dancer.sqlite'
162 },
163 },
164 },
165 };
166 }
167 use Dancer2::Plugin::JobScheduler;
168 use Dancer2::Plugin::Database;
169 set serializer => 'JSON';
170 get q{/submit_job} => sub {
171 my %r = submit_job(
172 client => 'theschwartz',
173 job => {
174 task => 'task1',
175 args => { name => 'Mikko', age => 123 },
176 opts => {},
177 },
178 opts => {
179 # database is the keyword and command from
180 # Dancer2::Plugin::Database. It takes one argument:
181 # the database name, similar to our dbh_callback.
182 dbh_callback => \&database,
183 },
184 );
185 status HTTP_OK;
186 return \%r;
187 };
188  
189  
190 =head1 METHODS
191  
192 =head2 submit
193  
194 Submit a job with arguments to a job scheduler.
195 This can be as simple as following:
196  
197 submit_job( job => { task => 'task_name' });
198  
199 In the example above, C<submit_job> uses the default scheduler.
200 This is enough when there is only one job scheduler.
201  
202 Parameter B<job> can also have sub parameters:
203  
204 =over 8
205  
206 =item B<args> can be used to provide a hash of arguments to the task. These are task specific.
207  
208 =item B<opts> can be used to provide a hash of options for the job scheduler. These are job scheduler specific and rarely used. They can be used, for example, to submit the job to a particular queue if there is priority queues in the system.
209  
210 =back
211  
212 submit_job(
213 job => {
214 task => 'task_name',
215 args => { name => 'Average Joe', age => 67 },
216 opts => { run_after => time + (60*60) },
217 },
218 );
219  
220 In the example above, the task is created with a delay of 60 minutes,
221 i.e. the job scheduler TheSchwartz will not attempt to run the task
222 before one hour is passed.
223  
224 If you have several different job schedulers you can submit jobs to,
225 then use parameter B<client> to identify the one you want to use.
226 The client names are specified in the configuration.
227 You can also specify a default client.
228  
229 submit_job(
230 client => 'theschwartz',
231 job => {
232 task => 'task_name',
233 },
234 );
235  
236 C<submit_job> will return a hash which contains at least the following items:
237  
238 =over 8
239  
240 =item success, boolean. Was the operation successful?
241  
242 =item status, string. Contains the status of the submit. In the case of success, this will be "OK".
243  
244 =item error, string. Contains an error message if a message is available. Can also be undef.
245  
246 =back
247  
248 It can also contain other items depending on the job scheduler.
249 In the case of TheSchwartz, after a successful submit there will be item B<id>
250 which contains the id of the new job in the queue.
251  
252 The following example showcases a very trivial way on how to integrate C<submit_job> into
253 a route:
254  
255 post q{/send_email} => sub {
256 my $email = body_parameters->{email};
257 # Remember to untaint input:
258 ($email) = $email =~ m/ ( [a-zA-Z0-9]{1,} @ [a-zA-Z0-9]{1,} ) /msx;
259 submit_job(
260 job => {
261 task => 'send_email',
262 args => { email => $email },
263 },
264 );
265 };
266  
267  
268 =head2 list_jobs
269  
270 Return a list of all active jobs in the job scheduler.
271  
272 Parameters:
273  
274 =over 8
275  
276 =item client, string. The scheduler name. Default specified in the configuration.
277  
278 =item search_params, hash. These are job scheduler specific.
279  
280 =back
281  
282 set serializer => q{JSON};
283 get q{/list_jobs} => sub {
284 my %r = list_jobs(
285 client => 'theschwartz',
286 search_params => {
287 task => 'task1',
288 },
289 );
290 return $r{'jobs'};
291 };
292  
293  
294 =head1 CONFIGURATION
295  
296 Dancer2::Plugin::JobScheduler uses Dancer2's configuration system.
297 You can either write your configuration in the config files
298 or specify it in the module.
299  
300 The different job schedulers have their own configuration needs.
301 As an example we will cover here only TheSchwartz.
302  
303  
304 =head1 SEE ALSO
305  
306 There is a Dancer2 plugin for Minion: L<Dancer2::Plugin::Minion>.
307  
308 =cut
309  
310 6 ▶use Carp qw( croak );
Subroutine BEGIN: called CC=1 SCAR=0.0
311 6 ▶use English '-no_match_vars';
Subroutine BEGIN: called CC=1 SCAR=0.0
312 6 ▶use Module::Load;
Subroutine BEGIN: called CC=1 SCAR=0.0
313  
314 6 ▶use Log::Any qw( $log );
Subroutine BEGIN: called CC=1 SCAR=0.0
315  
316 6 ▶use Dancer2::Plugin 0.200002;
Subroutine BEGIN: called CC=1 SCAR=0.0
317  
318 6 plugin_keywords qw(
319 submit_job
320 list_jobs
321 );
322  
323 has default => (
324 is => 'ro',
325 3 ▶ isa => sub {
Subroutine __ANON__: called CC=2 SCAR=7.6
326 3 ▶ croak if(! ($_[0] =~ m/^[[:word:]]{1,}$/msx) );
Branch: 1 branch
BranchTrueFalseTotal
if not $_[0] =~ /^[[:word:]]{1,}$/msx 0 3 3
327 },
328 6 from_config => 'default',
329 );
330  
331 has schedulers => (
332 is => 'ro',
333 3 ▶ isa => sub {
Subroutine __ANON__: called CC=2 SCAR=7.6
334 3 ▶ croak if(ref $_[0] ne 'HASH');
Branch: 1 branch
BranchTrueFalseTotal
if ref $_[0] ne "HASH" 0 3 3
335 },
336 6 from_config => 'schedulers',
337 );
338  
339 3 ▶sub _verify_configuration {
Subroutine _verify_configuration: called CC=5 SCAR=28.2
340 3 my ($self) = @_;
341 3 ▶ if( $self->default ) {
Branch: 1 branch
BranchTrueFalseTotal
if ($self->default) { } 3 0 3
342 3 ▶ if( exists $self->schedulers->{ $self->default } ) {
Branch: 1 branch
BranchTrueFalseTotal
if (exists $self->schedulers->{$self->default}) { } 3 0 3
343 3 return $self->default;
344 } else {
345 0 my $e = 'Invalid value in config: plugins->JobScheduler->default: '
346 . '\'%s\', no matching job scheduler';
347 0 $log->errorf( $e, $self->default);
348 0 croak sprintf $e, $self->default;
349 }
350 } else {
351 0 ▶ if( scalar keys %{ $self->schedulers } > 1 ) {
Branch: 2 branches
BranchTrueFalseTotal
if (scalar keys %{$self->schedulers;} > 1) { } 0 0 0
elsif (scalar keys %{$self->schedulers;} <= 0) { } 0 0 0
352 0 my $e = 'Default job queue missing in config. Must define '
353 . 'default job queue when there is more than one job queue';
354 0 $log->errorf( $e );
355 0 croak $e;
356 0 } elsif( scalar keys %{ $self->schedulers } <= 0 ) {
357 0 my $e = 'Invalid config. Must have at least one job queue';
358 0 $log->errorf( $e );
359 0 croak $e;
360 } else {
361 0 return (keys %{ $self->schedulers })[0];
362 }
363 }
364 0 return 1; # All good. Otherwise we would have croaked.
365 }
366  
367  
368 # A client (of a client) of the job queue
369 # or other object via which the jobs are submitted
370 has _clients => (
371 is => 'lazy',
372 3 ▶ isa => sub {
Subroutine __ANON__: called CC=2 SCAR=7.6
373 3 ▶ croak if(ref $_[0] ne 'HASH');
Branch: 1 branch
BranchTrueFalseTotal
if ref $_[0] ne "HASH" 0 3 3
374 },
375 6 );
376 3 ▶sub _build__clients { ## no critic (Subroutines::ProhibitUnusedPrivateSubroutines)
Subroutine _build__clients: called CC=7 SCAR=21.6
377 3 my $self = shift;
378 3 $self->_verify_configuration();
379 3 my %h;
380 3 foreach my $key ( keys %{ $self->schedulers } ) {
381 3 my $s = $self->schedulers->{$key};
382 my $client_name = $s->{'client'} =~ m/::/msx
383 3 ▶ ? $s->{'client'}
Branch: 1 branch
BranchTrueFalseTotal
$s->{'client'} =~ /::/msx ? : 0 3 3
384 : "Dancer2::Plugin::JobScheduler::Client::$s->{'client'}";
385  
386 # Does it exist?
387 {
388 3 local $EVAL_ERROR = $EVAL_ERROR;
389 3 my $r = eval { load "$client_name"; 1; };
390 3 ▶ if( ! $r || $EVAL_ERROR ) {
Branch: 1 branch
BranchTrueFalseTotal
if (not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR) 0 3 3
Condition33%1/3
not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR
l!l&&r!l&&!r
003
MC/DC0%0/2
not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR
not $r $Dancer2::Plugin::JobScheduler::EVAL_ERROR
Truth table33%1/3
not(A) or B
A= not $rB= $Dancer2::Plugin::JobScheduler::EVAL_ERROR
ABresult
1X1
011
000
391 0 my $e = 'Failed to load job queue client \'%s\', error: %s';
392 0 $log->errorf( $e, $client_name, $EVAL_ERROR);
393 0 croak sprintf $e, $client_name, $EVAL_ERROR;
394 }
395 }
396  
397 # Can we instantiate it?
398 3 my $scheduler;
399 {
400 3 local $EVAL_ERROR = $EVAL_ERROR;
401 3 my $r = eval {
402 $scheduler = $client_name->new(
403 3 config => $s->{'parameters'},
404 );
405 3 1;
406 };
407 3 ▶ if( ! $r || $EVAL_ERROR ) {
Branch: 1 branch
BranchTrueFalseTotal
if (not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR) 0 3 3
Condition33%1/3
not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR
l!l&&r!l&&!r
003
MC/DC0%0/2
not $r or $Dancer2::Plugin::JobScheduler::EVAL_ERROR
not $r $Dancer2::Plugin::JobScheduler::EVAL_ERROR
Truth table33%1/3
not(A) or B
A= not $rB= $Dancer2::Plugin::JobScheduler::EVAL_ERROR
ABresult
1X1
011
000
408 0 my $e = 'Failed to instantiate job queue client \'%s\', error: %s';
409 0 $log->errorf( $e, $client_name, $EVAL_ERROR);
410 0 croak sprintf $e, $client_name, $EVAL_ERROR;
411 }
412 }
413 3 $h{ $key } = $scheduler;
414 }
415 3 return \%h;
416 }
417  
418 9 ▶sub submit_job {
Subroutine submit_job: called CC=3 SCAR=11.0
Pod undocumented
419 9 my ($self, %args) = @_;
420  
421 9 ▶ my $client_key = $args{client} ? delete $args{client} : $self->default;
Branch: 1 branch
BranchTrueFalseTotal
$args{'client'} ? : 9 0 9
422 9 my $job = $args{job};
423 9 ▶ my $opts = $args{opts} ? delete $args{opts} : {};
Branch: 1 branch
BranchTrueFalseTotal
$args{'opts'} ? : 4 5 9
424 9 my $client = $self->_clients->{$client_key};
425  
426 9 return $client->submit_job( $job, $opts );
427 }
428  
429 6 ▶sub list_jobs {
Subroutine list_jobs: called CC=3 SCAR=11.0
Pod documented
430 6 my ($self, %args) = @_;
431 6 ▶ my $client_key = $args{client} ? delete $args{client} : $self->default;
Branch: 1 branch
BranchTrueFalseTotal
$args{'client'} ? : 6 0 6
432 6 my $search_params = $args{search_params};
433 6 ▶ my $opts = $args{opts} ? delete $args{opts} : {};
Branch: 1 branch
BranchTrueFalseTotal
$args{'opts'} ? : 2 4 6
434 6 my $client = $self->_clients->{$client_key};
435 6 return $client->list_jobs( $search_params, $opts );
436 }
437  
438 =pod
439  
440 =head1 ADOPTION
441  
442 If you're interested in adopting this module, and the author/maintainer
443 appears to be no longer active, please consult the PAUSE module
444 adoption process documented at L<https://github.com/Perl-Toolchain-Gang/pause/blob/master/doc/takeover-policy.md>.
445  
446 The PAUSE admins (modules@perl.org) may grant co-maintainer or
447 primary-maintainer permissions to a suitable adopter if:
448  
449 =over 4
450  
451 =item *
452  
453 There has been no release for a year or more, AND
454  
455 =item *
456  
457 There are outstanding issues, pull requests, or bug reports that would
458 benefit from attention, AND
459  
460 =item *
461  
462 Reasonable attempts to contact me have failed (CPAN email address,
463 GitHub issues on the project repository, and any other channels listed
464 in this distribution) over a period of at least one month, AND
465  
466 =item *
467  
468 The prospective adopter intends to make changes that benefit users of
469 the module.
470  
471 =back
472  
473 =for stopwords maintainership
474  
475 In the event of my death or permanent incapacity, my heirs are not
476 obligated to maintain these modules, and I explicitly authorize the
477 PAUSE admins to transfer maintainership without further consultation
478 once the conditions above are met.
479  
480 =head1 AUTHOR
481  
482 Mikko Koivunalho <mikkoi@cpan.org>
483  
484 =head1 COPYRIGHT AND LICENSE
485  
486 This software is copyright (c) 2026 by Mikko Koivunalho.
487  
488 This is free software; you can redistribute it and/or modify it under
489 the same terms as the Perl 5 programming language system itself.
490  
491 =cut
492  
493 6 1;
↑ Summary JobScheduler/Client/TheSchwartz.pm »