line |
stmt |
bran |
cond |
sub |
pod |
time |
code |
1
|
|
|
|
|
|
|
=head1 NAME |
2
|
|
|
|
|
|
|
|
3
|
|
|
|
|
|
|
Synapse::Monitor::Listener |
4
|
|
|
|
|
|
|
|
5
|
|
|
|
|
|
|
|
6
|
|
|
|
|
|
|
=head1 About Synapse's Open Telephony Toolkit |
7
|
|
|
|
|
|
|
|
8
|
|
|
|
|
|
|
L is a part of Synapse's Wholesale Open Telephony Toolkit. |
9
|
|
|
|
|
|
|
|
10
|
|
|
|
|
|
|
As we are refactoring our codebase, we at Synapse have decided to release a |
11
|
|
|
|
|
|
|
substantial portion of our code under the GPL. We hope that as a developer, you |
12
|
|
|
|
|
|
|
will find it as useful as we have found open source and CPAN to be of use to |
13
|
|
|
|
|
|
|
us. |
14
|
|
|
|
|
|
|
|
15
|
|
|
|
|
|
|
|
16
|
|
|
|
|
|
|
=head1 What is L all about |
17
|
|
|
|
|
|
|
|
18
|
|
|
|
|
|
|
In the wholesale telecom business, you need to monitor a lot of stuff. For |
19
|
|
|
|
|
|
|
example, for each egress and ingress route (and there's hundreds of them and |
20
|
|
|
|
|
|
|
thousands of dialcodes), you need to permaently check answer seizure ration, |
21
|
|
|
|
|
|
|
average length of call, MOS, customer credit limit usage, make sure your |
22
|
|
|
|
|
|
|
systems are up and ping-eable, CPU load, etc. etc. |
23
|
|
|
|
|
|
|
|
24
|
|
|
|
|
|
|
The idea is that you have a collection of scripts / daemons / cron jobs / etc |
25
|
|
|
|
|
|
|
which product Ip. Events have an associated type (e.g. 'ping'), |
26
|
|
|
|
|
|
|
I (e.g. UNKNOWN, OK, WARNING, DOWN), and indentifier (e.g. ping |
27
|
|
|
|
|
|
|
example-dot-com). |
28
|
|
|
|
|
|
|
|
29
|
|
|
|
|
|
|
What L does is that it picks up newly created |
30
|
|
|
|
|
|
|
events and can be configured to perform a set of predefined actions when it |
31
|
|
|
|
|
|
|
detects an I, such as firing an email, suspending a service, |
32
|
|
|
|
|
|
|
blocking an IP address, etc. |
33
|
|
|
|
|
|
|
|
34
|
|
|
|
|
|
|
|
35
|
|
|
|
|
|
|
=head1 L overview and installation |
36
|
|
|
|
|
|
|
|
37
|
|
|
|
|
|
|
The library is split as follows: |
38
|
|
|
|
|
|
|
|
39
|
|
|
|
|
|
|
=over 4 |
40
|
|
|
|
|
|
|
|
41
|
|
|
|
|
|
|
=item script synapse-monitor-listener-cli, used to create and set configuration objects. |
42
|
|
|
|
|
|
|
|
43
|
|
|
|
|
|
|
=item script synapse-monitor-listener-service, which is a daemon desgined to be |
44
|
|
|
|
|
|
|
running in the background |
45
|
|
|
|
|
|
|
|
46
|
|
|
|
|
|
|
=item L, listener object, which is designed to |
47
|
|
|
|
|
|
|
choose what action(s) to do with .evt.yml objects / files. |
48
|
|
|
|
|
|
|
|
49
|
|
|
|
|
|
|
=item L, listener action object, which is |
50
|
|
|
|
|
|
|
designed to define and execute arbitrary actions. |
51
|
|
|
|
|
|
|
|
52
|
|
|
|
|
|
|
=back |
53
|
|
|
|
|
|
|
|
54
|
|
|
|
|
|
|
You install the package as follows: |
55
|
|
|
|
|
|
|
|
56
|
|
|
|
|
|
|
perl Makefile.PL |
57
|
|
|
|
|
|
|
make |
58
|
|
|
|
|
|
|
make test |
59
|
|
|
|
|
|
|
make install |
60
|
|
|
|
|
|
|
synapse-monitor-listener-cli --create-configdir |
61
|
|
|
|
|
|
|
synapse-monitor-listener-service install |
62
|
|
|
|
|
|
|
synapse-monitor-listener-service start |
63
|
|
|
|
|
|
|
|
64
|
|
|
|
|
|
|
synapse-monitor-listener-service should then start on each reboot, runlevels |
65
|
|
|
|
|
|
|
3-5. You can check that this is the case and that the daemon has been correctly |
66
|
|
|
|
|
|
|
installed using the excellent chkconfig linux tool. |
67
|
|
|
|
|
|
|
|
68
|
|
|
|
|
|
|
|
69
|
|
|
|
|
|
|
=head1 L configuration |
70
|
|
|
|
|
|
|
|
71
|
|
|
|
|
|
|
Note: L does NOT need restarting once you have |
72
|
|
|
|
|
|
|
changed the configuration. |
73
|
|
|
|
|
|
|
|
74
|
|
|
|
|
|
|
For the sake of the example - and because this package is designed to be a |
75
|
|
|
|
|
|
|
telephony package after all - let's say we have a bunch of call detail record |
76
|
|
|
|
|
|
|
(or "cdr") files on the filesystem. Each file contains a list of calls, wether |
77
|
|
|
|
|
|
|
they were answered or not, the call duration, etc. |
78
|
|
|
|
|
|
|
|
79
|
|
|
|
|
|
|
Say we want to check and monitor ASR values, i.e. how many calls were answered |
80
|
|
|
|
|
|
|
on the last 100 calls. We have written a script, called asrcheck.pl, which runs |
81
|
|
|
|
|
|
|
in the background, and which produces notification files looking like this: |
82
|
|
|
|
|
|
|
|
83
|
|
|
|
|
|
|
--- |
84
|
|
|
|
|
|
|
id: asr-myvoipsupplier-moroccomobilemeditel |
85
|
|
|
|
|
|
|
state: OK |
86
|
|
|
|
|
|
|
listener: asr |
87
|
|
|
|
|
|
|
destination: Morocco-Mobile-Meditel |
88
|
|
|
|
|
|
|
vendor: myvoipsupplier |
89
|
|
|
|
|
|
|
notification-email: customer.service@example.com |
90
|
|
|
|
|
|
|
head-wc: 100 |
91
|
|
|
|
|
|
|
asr: 62 |
92
|
|
|
|
|
|
|
|
93
|
|
|
|
|
|
|
Notification files should be placed in /tmp/ and end with .evt.yml. |
94
|
|
|
|
|
|
|
synapse-monitor-listener-service will process these files as they appear and |
95
|
|
|
|
|
|
|
delete them after they have been processed. |
96
|
|
|
|
|
|
|
|
97
|
|
|
|
|
|
|
The only required fields are "listener", "state", and "id". All the rest is |
98
|
|
|
|
|
|
|
optional but your underlying "action" scpripts could use the extra information. |
99
|
|
|
|
|
|
|
A copy of the file will be passed to them as $ENV{YAML_FILE}. |
100
|
|
|
|
|
|
|
|
101
|
|
|
|
|
|
|
=over 4 |
102
|
|
|
|
|
|
|
|
103
|
|
|
|
|
|
|
=item id - should be unique across all listeners. i.e. there shouldn't be an |
104
|
|
|
|
|
|
|
"asr" listener called "foo" and an "acd" listener called "foo". Set up your |
105
|
|
|
|
|
|
|
scripts to call your checks "asr-foo" and "acd-foo" instead. |
106
|
|
|
|
|
|
|
|
107
|
|
|
|
|
|
|
=item state - can be any string without spaces. The number of states should be |
108
|
|
|
|
|
|
|
discrete and finite. |
109
|
|
|
|
|
|
|
|
110
|
|
|
|
|
|
|
=item listener - which listener configuration to use on this type of check. |
111
|
|
|
|
|
|
|
|
112
|
|
|
|
|
|
|
=back |
113
|
|
|
|
|
|
|
|
114
|
|
|
|
|
|
|
For the sake of the example, say we have 3 possible states: |
115
|
|
|
|
|
|
|
|
116
|
|
|
|
|
|
|
|
117
|
|
|
|
|
|
|
=over 4 |
118
|
|
|
|
|
|
|
|
119
|
|
|
|
|
|
|
=item OK : when the ASR is > 30 |
120
|
|
|
|
|
|
|
|
121
|
|
|
|
|
|
|
=item WARNING : when the ASR is < 30 but > 10 |
122
|
|
|
|
|
|
|
|
123
|
|
|
|
|
|
|
=item DOWN : when the ASR is < 10 |
124
|
|
|
|
|
|
|
|
125
|
|
|
|
|
|
|
=back |
126
|
|
|
|
|
|
|
|
127
|
|
|
|
|
|
|
|
128
|
|
|
|
|
|
|
We need to let L know what to do when there is a |
129
|
|
|
|
|
|
|
I on this type of notification. |
130
|
|
|
|
|
|
|
|
131
|
|
|
|
|
|
|
|
132
|
|
|
|
|
|
|
First, let's configure an "asr" listener: |
133
|
|
|
|
|
|
|
|
134
|
|
|
|
|
|
|
# first of all, create our "ASR listener" object |
135
|
|
|
|
|
|
|
synapse-monitor-listener-cli type listener create asr "ASR listener" |
136
|
|
|
|
|
|
|
|
137
|
|
|
|
|
|
|
# DOWN -> OK, or WARN -> OK : cool... |
138
|
|
|
|
|
|
|
synapse-monitor-listener-cli listener asr action WARN.DOWN OK asr-email-goodjob |
139
|
|
|
|
|
|
|
|
140
|
|
|
|
|
|
|
# OK -> WARN : send warning email |
141
|
|
|
|
|
|
|
synapse-monitor-listener-cli listener asr action OK WARN asr-email-warning |
142
|
|
|
|
|
|
|
|
143
|
|
|
|
|
|
|
# OK -> DOWN or WARN -> DOWN = send "down" email + suspend route |
144
|
|
|
|
|
|
|
synapse-monitor-listener-cli listener asr action OK.WARN DOWN asr-email-down suspend-route |
145
|
|
|
|
|
|
|
|
146
|
|
|
|
|
|
|
|
147
|
|
|
|
|
|
|
Now that's done, let's configure matching actions: |
148
|
|
|
|
|
|
|
|
149
|
|
|
|
|
|
|
# a copy of the YAML notification file will be passed as $ENV{YAML_FILE}, set up your scripts accordingly... |
150
|
|
|
|
|
|
|
# format is |
151
|
|
|
|
|
|
|
# synapse-monitor-listener-cli type action create useless-action echo I_AM_USELESS >/dev/null |
152
|
|
|
|
|
|
|
|
153
|
|
|
|
|
|
|
# to delete an action: |
154
|
|
|
|
|
|
|
# synapse-monitor-listener-cli action useless-action remove |
155
|
|
|
|
|
|
|
|
156
|
|
|
|
|
|
|
synapse-monitor-listener-cli type action create asr-email-goodjob synapse-email-notification /etc/synapse-monitor/emails/goodjob.xml |
157
|
|
|
|
|
|
|
synapse-monitor-listener-cli type action create asr-email-warning synapse-email-notification /etc/synapse-monitor/emails/warning.xml |
158
|
|
|
|
|
|
|
synapse-monitor-listener-cli type action create asr-email-down synapse-email-notification /etc/synapse-monitor/emails/down.xml |
159
|
|
|
|
|
|
|
synapse-monitor-listener-cli type action create email-restored synapse-email-notification /etc/synapse-monitor/emails/restored.xml |
160
|
|
|
|
|
|
|
synapse-monitor-listener-cli type action create suspend-route synapse-suspend-route |
161
|
|
|
|
|
|
|
|
162
|
|
|
|
|
|
|
|
163
|
|
|
|
|
|
|
That's it. You don't need to restart the daemon: the configuration changes are |
164
|
|
|
|
|
|
|
picked up immediately. |
165
|
|
|
|
|
|
|
|
166
|
|
|
|
|
|
|
=cut |
167
|
|
|
|
|
|
|
package Synapse::Monitor::Listener; |
168
|
1
|
|
|
1
|
|
933
|
use base qw /Synapse::CLI::Config::Object/; |
|
1
|
|
|
|
|
2
|
|
|
1
|
|
|
|
|
1216
|
|
169
|
1
|
|
|
1
|
|
91483
|
use Synapse::Logger; |
|
1
|
|
|
|
|
622
|
|
|
1
|
|
|
|
|
68
|
|
170
|
1
|
|
|
1
|
|
7
|
use YAML::XS; |
|
1
|
|
|
|
|
14
|
|
|
1
|
|
|
|
|
54
|
|
171
|
1
|
|
|
1
|
|
5
|
use warnings; |
|
1
|
|
|
|
|
1
|
|
|
1
|
|
|
|
|
30
|
|
172
|
1
|
|
|
1
|
|
5
|
use strict; |
|
1
|
|
|
|
|
2
|
|
|
1
|
|
|
|
|
1136
|
|
173
|
|
|
|
|
|
|
|
174
|
|
|
|
|
|
|
|
175
|
|
|
|
|
|
|
our $VERSION = 0.3; |
176
|
|
|
|
|
|
|
our $EVTDIR = '/tmp'; |
177
|
|
|
|
|
|
|
our $EVTEXT = '.evt.yml'; |
178
|
|
|
|
|
|
|
|
179
|
|
|
|
|
|
|
|
180
|
|
|
|
|
|
|
sub action { |
181
|
42
|
|
|
42
|
1
|
30565
|
my $self = shift; |
182
|
42
|
|
|
|
|
68
|
my $before = shift; |
183
|
42
|
|
|
|
|
53
|
my $after = shift; |
184
|
42
|
|
|
|
|
84
|
my @actions = @_; |
185
|
42
|
100
|
|
|
|
123
|
if ($before =~ /\./) { |
186
|
12
|
|
|
|
|
38
|
for my $before (split /\./, $before) { |
187
|
24
|
|
|
|
|
67
|
$self->action ($before, $after, @actions); |
188
|
|
|
|
|
|
|
} |
189
|
12
|
|
|
|
|
81
|
return $self; |
190
|
|
|
|
|
|
|
} |
191
|
30
|
50
|
|
|
|
74
|
if ($after =~ /\./) { |
192
|
0
|
|
|
|
|
0
|
for my $after (split /\./, $after) { |
193
|
0
|
|
|
|
|
0
|
$self->action ($before, $after, @actions); |
194
|
|
|
|
|
|
|
} |
195
|
0
|
|
|
|
|
0
|
return $self; |
196
|
|
|
|
|
|
|
} |
197
|
|
|
|
|
|
|
|
198
|
30
|
|
100
|
|
|
100
|
$self->{action} ||= {}; |
199
|
30
|
|
100
|
|
|
121
|
$self->{action}->{$before} ||= {}; |
200
|
30
|
|
|
|
|
83
|
$self->{action}->{$before}->{$after} = \@_; |
201
|
30
|
|
|
|
|
121
|
return $self; |
202
|
|
|
|
|
|
|
} |
203
|
|
|
|
|
|
|
|
204
|
|
|
|
|
|
|
|
205
|
|
|
|
|
|
|
sub process { |
206
|
5
|
|
|
5
|
0
|
10334
|
my $self = shift; |
207
|
5
|
|
|
|
|
10
|
my $oldState = shift; |
208
|
5
|
|
|
|
|
7
|
my $newState = shift; |
209
|
5
|
|
|
|
|
10
|
my $event = shift; |
210
|
5
|
100
|
|
|
|
19
|
$self->{action}->{$oldState} || do { |
211
|
1
|
|
|
|
|
5
|
logger ($self->name() . ": no action specified for old state $oldState"); |
212
|
1
|
|
|
|
|
106
|
return; |
213
|
|
|
|
|
|
|
}; |
214
|
4
|
50
|
|
|
|
15
|
$self->{action}->{$oldState}->{$newState} || do { |
215
|
0
|
|
|
|
|
0
|
logger ($self->name() . ": no action specified for old state $oldState to new state $newState"); |
216
|
0
|
|
|
|
|
0
|
return; |
217
|
|
|
|
|
|
|
}; |
218
|
|
|
|
|
|
|
|
219
|
4
|
|
|
|
|
15
|
logger ($self->name() . ": iterating actions for $oldState -> $newState"); |
220
|
4
|
|
|
|
|
495
|
for my $action (@{$self->{action}->{$oldState}->{$newState}}) { |
|
4
|
|
|
|
|
16
|
|
221
|
5
|
|
|
|
|
22
|
logger ("action: $action"); |
222
|
5
|
|
|
|
|
449
|
my $action_obj = Synapse::Monitor::Listener::Action->new ($action); |
223
|
5
|
50
|
|
|
|
826
|
if ($action_obj) { $action_obj->process ($event) } |
|
5
|
|
|
|
|
20
|
|
224
|
|
|
|
|
|
|
else { |
225
|
0
|
|
|
|
|
0
|
logger ("$action: cannot instantiate object - skipping"); |
226
|
0
|
|
|
|
|
0
|
next; |
227
|
|
|
|
|
|
|
} |
228
|
|
|
|
|
|
|
}; |
229
|
|
|
|
|
|
|
} |
230
|
|
|
|
|
|
|
|
231
|
|
|
|
|
|
|
|
232
|
|
|
|
|
|
|
sub __evtfiles__() { |
233
|
3
|
|
|
3
|
|
141
|
opendir EVTDIR, $EVTDIR; |
234
|
3
|
|
|
|
|
184
|
my @files = readdir (EVTDIR); |
235
|
3
|
|
|
|
|
58
|
closedir EVTDIR; |
236
|
3
|
|
|
|
|
7
|
my @res = (); |
237
|
3
|
|
|
|
|
8
|
for my $file (@files) { |
238
|
21
|
100
|
|
|
|
129
|
$file =~ /\Q$EVTEXT\E$/ or next; |
239
|
3
|
50
|
|
|
|
63
|
-e "$EVTDIR/$file" or next; |
240
|
3
|
50
|
|
|
|
48
|
-e "$EVTDIR/$file.lock" and next; |
241
|
3
|
|
|
|
|
10
|
push @res, "$EVTDIR/$file"; |
242
|
|
|
|
|
|
|
} |
243
|
3
|
|
|
|
|
13
|
return @res; |
244
|
|
|
|
|
|
|
} |
245
|
|
|
|
|
|
|
|
246
|
|
|
|
|
|
|
|
247
|
|
|
|
|
|
|
sub __loadfile__($) { |
248
|
3
|
|
|
3
|
|
3
|
my $file = shift; |
249
|
3
|
50
|
|
|
|
118
|
open YAMLFILE, $file or return; |
250
|
3
|
|
|
|
|
70
|
my $data = join '', ; |
251
|
3
|
|
|
|
|
33
|
close YAMLFILE; |
252
|
3
|
|
|
|
|
153
|
return Load $data; |
253
|
|
|
|
|
|
|
} |
254
|
|
|
|
|
|
|
|
255
|
|
|
|
|
|
|
|
256
|
|
|
|
|
|
|
sub runonce { |
257
|
3
|
|
|
3
|
0
|
4230
|
my $class = shift; |
258
|
3
|
|
|
|
|
11
|
for my $file (__evtfiles__) { |
259
|
3
|
|
|
|
|
10
|
my $event = __loadfile__ $file; |
260
|
3
|
|
|
|
|
278
|
unlink $file; |
261
|
3
|
50
|
|
|
|
11
|
$event || next; |
262
|
|
|
|
|
|
|
|
263
|
3
|
|
50
|
|
|
10
|
my $id = $event->{id} || next; |
264
|
3
|
|
50
|
|
|
8
|
my $newState = $event->{state} || next; |
265
|
3
|
|
50
|
|
|
11
|
my $listener = $event->{listener} || next; |
266
|
|
|
|
|
|
|
|
267
|
3
|
|
66
|
|
|
29
|
my $oldState = Synapse::Monitor::Listener::State->new ($id) || Synapse::Monitor::Listener::State->create ($id => 'UNKNOWN'); |
268
|
3
|
|
|
|
|
958
|
$oldState = $oldState->label(); |
269
|
|
|
|
|
|
|
|
270
|
3
|
50
|
|
|
|
23
|
$oldState eq $newState and next; |
271
|
3
|
|
50
|
|
|
18
|
$listener = $class->new ($listener) || next; |
272
|
3
|
|
|
|
|
58
|
eval { $listener->process ($oldState, $newState, $event) }; |
|
3
|
|
|
|
|
11
|
|
273
|
3
|
|
|
|
|
16
|
Synapse::CLI::Config::execute ("Synapse::Monitor::Listener::State", $id, "set", "label", $newState); |
274
|
|
|
|
|
|
|
} |
275
|
|
|
|
|
|
|
} |
276
|
|
|
|
|
|
|
|
277
|
|
|
|
|
|
|
|
278
|
|
|
|
|
|
|
1; |
279
|
|
|
|
|
|
|
|
280
|
|
|
|
|
|
|
|
281
|
|
|
|
|
|
|
__END__ |