File Coverage

Criterion Covered Total %
statement 10 10 100.0
branch n/a
condition n/a
subroutine 4 4 100.0
pod n/a
total 14 14 100.0

line stmt bran cond sub pod time code
1             package ThreatNet::DATN2004;
3 1     1   1354 use 5.005;
  1         4  
  1         40  
4 1     1   5 use strict;
  1         2  
  1         35  
5 1     1   14 use vars qw{$VERSION};
  1         2  
  1         51  
6             BEGIN {
7 1     1   142 $VERSION = '0.20';
8             }
10             1;
12             =pod
14             =head1 NAME
16             ThreatNet::DATN2004 - Proposal: The Decentralised Active Threat Network
18             This document has been created to describe a concept that may be of use in
19             a variety of fields. It should be considered a general concept only and is
20             subject to change.
22             This CPAN/POD version of the document, first published in December 2004 at
23             L, has been released to
24             independantly timestamp and archive the concept and proposal in case of
25             future patent-related issues by companies and to attempt to keep the core
26             idea available to all.
28             =head1 INTRODUCTION
30             On the Internet there exists an increasing number of different ways in which
31             hosts are being misused or abused. Likewise there is also an increasing
32             number of ways in which these known-bad hosts are being identified. Most of
33             these occur in the process of a particular task, such as checking an email
34             message for spam status.
36             As these hosts are identified, their identify is transmitted across to
37             internet to members of threat networks. The most common of these are the
38             various email "black lists", most of which use DNS or some other method to
39             publish lists of known-bad ips or ip ranges. Mail processing services
40             submit requests to a DNS server storing these lists to determine if a
41             particular host contacting them is a known spammer.
43             This draft specification describes a system which would be used to
44             identify a specific category of these bad ips, hosts that can be
45             considered "Active Threats". An Active Threat is a host that is currently
46             engaged in anti-social, damaging or criminal behaviour, such as actively
47             sending out spam or viruses. This specification is NOT intended to deal
48             with long-term offenders, as they are addressed by a number of current
49             systems.
51             If applied to long term offenders, hosts would be registered as an Active
52             Threat when they commence their anti-social behaviour, and fall off any
53             list during periods in which they are not conducting this behaviour.
55             The general intent is to deal with only those hosts that are actively
56             engaged in damaging behaviour, whether or not they are long term offenders
57             or new offenders. And to deal with the hosts as soon as possible, ideally
58             within a few seconds.
60             =head1 GENERAL PRINCIPLES
62             The following general principles have been establish for this system,
63             and guide the sample implementation described below. They are listed
64             (loosely) in priority order.
66             =head2 Speed of Response
68             The ThreatNet concept is specifically intended to address active and
69             transient threats. If a Verified Threat is detected at time $t, other
70             members of the same Threat Network should start to recieve notification
71             before time $t + 1 (within 1 second). Full propagation to all members, and
72             any subsequent responses, should be complete within by $t + 60 (within
73             1 minute).
75             Within a second of a host being detected conducting some anti-social
76             behaviour, the member should be able to confirm this behaviour and the
77             host ip responsible for it, and classify the host as a Verified Threat,
78             issuing a message into a Threat Network. Notification to all members
79             of the Threat Network and any other linked networks should occur as
80             quickly as the Internet and Threat Network itself is able to spread this
81             information. Any members who wish to respond to the threat, or are able
82             to neutralise it, should be able to act within a few seconds and
83             certainly within a minute.
85             For the example of spam, this would allow newly compromised hosts
86             ("zombies") acting as spam or virus agents that are not on current block
87             lists to be dealt with and blocked before they are able to send
88             significant amounts of spam. This could also allow for compromised hosts
89             on dynamic IPs to be blocked each time they move IPs without
90             significantly damaging subsequent users of the IPs, or needing to block
91             the entire IP range. This could also help to reduce the impact of new
92             and fast-moving viruses, as any newly infected computers appearing within
93             properly managed ISPs could be disconnected and disabled before they are
94             able to make a significant number of attempts to reproduce.
96             =head2 Flexibility and Integration
98             The implementation should be as flexible as possible and integrate with
99             many other current systems, or provide a way for developers to do this
100             integration themselves. Although the initial idea for this concept was as
101             an anti-spam system, it should not be considered solely an anti-spam tool,
102             and should be able to be easily reconfigured for use in any other scenario
103             that involves similar sorts of rapid responses to threats.
105             The entire system should be defined as a set of separate components, any
106             of which can be replaced, improved or set up in a variety of different
107             ways. Each function or role should be identified separately so that new
108             systems and implementations can provide one particular role without
109             needing to implement all of them.
111             =head2 Distributed and Decentralised
113             Most existing anti-spam or anti-threat networks involve a central
114             authorative data source, and a central authorative distribution network to
115             spread it, typically a managed database of some sort and a DNS/rsync
116             combination. Centralising in this way can lead to the blocking systems
117             themselves becoming the target of both technical and legal attacks.
119             Extreme measures are often required to protect (technically and legally)
120             the blocking systems.
122             This proposal involves a decentralised system with no long term state
123             information and no central server or service that can be implemented in
124             an ad-hoc way. Having limited the scope of the network to short term and
125             transient threat this can be done relatively easily.
127             Short term state information is held locally at each node, and a mandory
128             "quiet period" is enforced when joining a new network to let the new member
129             synchronise with the network.
131             This quiet period will keep load and communication volume down, and let
132             new members join a network without the need for a synchronisation event.
134             By waiting for the network's block period before contributing, each node
135             can be sure that it has fully synchronised with the network before beginning
136             to contribute block entries that might otherwise be duplicates.
138             =head2 Ease of Implementation
140             In order to get this concept off the ground and running quickly, and to help
141             implement it as flexibly as possible, it is proposed that the system makes
142             use of common, time-tested, well-supported and mature systems wherever
143             possible.
145             =head2 Minimisation of Harm
147             Long term blocking, and the blocking of entire subnets, may not be suitable
148             for dealing with new and short term threats. One bad apple can ruin it for
149             the entire barrel. While the ability to "poison" a Threat Network may
150             still exist as it does in existing blocking systems, this poisoning would
151             need to be active and continuous, "re-poisoning" the Threat Network with
152             new messages each time the short block period nears its end.
154             For dynamic IP connections, any response taken against a single IP will
155             expire automatically within the block period, once the host stops use of
156             the IP.
158             =head2 Elegant Degradation and Failure
160             As with any security system, the greater and higher profile its use, the
161             bigger the target it becomes and more likely to be attacked. The system
162             should be designed from the start to be able to stand up to attack when
163             deployed in high-profile scenarios, and to degrade gracefully where
164             possible to retain at least partial functionality.
166             =head1 EXAMPLE IMPLEMENTATION
168             The following describes the set of elements which would be required to
169             create a single complete system, and a set of optional elements which
170             could be used to enhance or expand the system.
172             =head2 Required Element: Messaging Platform
174             At the core of the Threat Network is a messaging platform. As an extremely
175             mature and often-attacked network messaging system, IRC would appear an
176             ideal platform to use for this purpose. Libraries to connect to and work
177             with IRC already exists in almost every programming language. In addition,
178             it can be deployed extremely flexibly from either a non-authenticated
179             channel on an existing IRC network to secret and private dedicated servers
180             with encryption, certificate-backed authentication, and agent-based
181             monitoring.
183             =head2 Required Element: Language Specification
185             Each member of the network should interact in a standard and relatively
186             flexible language. While this specification does not attempt to describe
187             the details of any particular language, for example purposes only we will
188             start with the simplest possible language. The sample language contains
189             only a single message with only the IP address of a validated threat,
190             assumed to have been detected within a few seconds of the threat event
191             occurring.
193             =head2 Required Element: Network Agents
195             Within the IRC Threat Network, there exists a number of members, each
196             consisting of at least a software agent. Any agent connected to the system
197             will consist of at least the Event Listener element described below, and
198             one or more optional elements. At least one agent is required to implement
199             the Threat Provider element in order to create a working system.
201             =head2 Required Element: Event Listener
203             A pure Event Listener is a passive agent that listens on a Threat Network
204             for threat messages and optionally acts upon them. Every agent connected to
205             the Threat Network is required to implement an Event Listener to monitor
206             incoming messages, if only to prevent issuing duplicate messages.
208             Many of the agents that would implement only the Event Listener would act
209             as interfaces and adapters to other systems.
211             As one example, a ThreatNet to DNSBL adapter might listen for threat
212             messages, submit a DNSBL request against each IP, and for each IP that
213             isn't already listed, add it to the local DNSBL server with a short TTL
214             most likely matching the ThreatNet block period.
216             A ThreatNet to firewall adapter could listen for specific types of
217             high-impact threat messages and inject blocks into a firewall configuration,
218             allowing particular types of threats to be blocked at a connection level.
220             Depending on the scale of the network, and the ability of the adapter to
221             handle the volume of responses, the Event Listener may need to institute
222             some form of queuing or flood control. It would be expected that such
223             flood control would become common were the popularity of Threat Networks
224             to grow significantly.
226             =head2 Optional Element: Threat Cache / Threat Filter
228             While some Threat Network agents do not need to maintain state, a great
229             many agent types do, in particular anything that wishes to submit Verified
230             Threat messages of its own to the network.
232             A Threat Cache is simply a small local database, probably in-memory, that
233             stores all "current" threats. A Threat Filter handles and passes on only
234             the subset of events that meet a certain set of criteria. For agents
235             joining a network, the purpose of waiting for the full block period until
236             beginning to submit messages is to allow the agent's Threat Cache to fill
237             and thus synchronise with the rest of the members of the network.
239             In another example, an ISP may deploy a "Zombie Response" agent that
240             applies a filter to limit the event feed to only those Verified Threats
241             that are the inside the ISP's own network. When the agent identifies
242             an infected or trojaned customer by IP, it could terminate the connection,
243             disable the login, and flag the customer's account so that when they try
244             to reconnect, they will be informed they are infected with a virus or
245             that someone may be using their computer without their permission.
247             =head2 Optional Element: Threat Provider
249             A Threat Provider is an agent that submits Verified Threats messages to a
250             network. Because any message that is added to the network may result in a
251             variety of actions against the host, care should be taken to verify that
252             the host is indeed an Active Threat.
254             Any Threat Provider should also act as an Event Listener and ensure that
255             Verified Threat events are not submitted if they are already "current"
256             (in the Threat Cache). The default TTL on all threats will be 1 hour.
258             As an example, a spam "secondary MX trap" might be tied to a local Threat
259             Network agent that would aggregate and filter threats from the host, and
260             submit Veried Threat messages into the network.
262             By separating the Threat Network from the individual applications, new
263             and more accurate anti-spam scripts can incorporate a connection to the
264             agent and feed more advanced Verified Threat messages into the same
265             network, be it local, regional, or larger.
267             =head2 Optional Element: Network Bridge
269             To separate a private company-wide network, or a university network from
270             a larger network, you can make use of a Network Bridge. This is an agent
271             that connects to two or more different networks, maintains a combined
272             Threat Cache and feeds Verified Threat messages from one to the other,
273             potentially in both directions.
275             For example, a single or small group of universities could run a private
276             Threat Network for open use by members of the University, with a single
277             Network Bridge agent pulling in messages from a larger national or
278             international Threat Network. The bridge would be configured to filter
279             out any messages that reference hosts already listed on the university
280             DNSBL server.
282             =head2 Optional Element: Security Model
284             One advantage of using IRC is that any of the existing security features
285             that are currently part of IRC can also be applied to a threat network,
286             including voice right, operators, channel management bots, SIRC and more.
288             At the most open, a single ThreatNet channel could be run within general
289             IRC server with anyone free to participate and submit any entries they
290             wish. This would make it trivially easy to set up an environment for
291             testing use.
293             More secure than this, a bot could run the channel allowing anyone to
294             join, but only giving voice permissions to agents joining from
295             white-listed IP addresses.
297             On a larger scale, a large dedicated SIRC network could be created, with
298             server or network-level accounts and monitoring bots to kick and issue
299             Verified Threat events for any anti-social accounts connected to the
300             Threat Network.
302             =head1 CONCLUSION
304             While not useful for long term blocking, the Decentralised Active Threat
305             Network would dramatically improve the response time to threats, with the
306             goal creating a powerful and effective "nervous system" for detecting and
307             dealing with spammers, virus-infected or trojaned hosts, or other threats
308             for which the damage could be greatly reduced by dealing with them
309             immediately.
311             =head1 GLOSSARY OF TERMS
313             =head2 Active Threat
315             Any internet host that is currently engaged in anti-social or illegal
316             behaviour such as spamming, virus transmission or acting as part of a DDOS
317             or other attack.
319             =head2 Verified Threat
321             A host/ip that can be positively and reliably confirmed to be an Active
322             Threat.
324             =head2 Threat Network
326             An communications network, populated by software agents, for the purpose
327             of rapidly distributing information about, and responding to, Active
328             Threats.
330             =head2 Threat Language
332             A protocol or message format used to describe a Verified Threat on a
333             Threat Network.
335             =head2 Event Listener
337             A software agent which is connected to one or more Threat Networks and
338             receives Verified Threat messages from the network.
340             =head2 Threat Cache
342             A stored list of "current" Active Threats, primarily used to prevent the
343             flooding of duplicate messages onto the Threat Network.
345             =head2 Threat Filter
347             Any collection of IPs or IP networks against which a stream of events
348             is checked, removing or keeping only a certain subset of all events.
350             =head2 Threat Provider
352             An software agent which connects to a Threat Network and submits Verified
353             Threat messages to the network.
355             =head2 Network Bridge
357             A software agent that sits "between" two or more Threat Networks,
358             transmitting new Verified Threat messages between them, after checking
359             against an arbitrary number of Threat Filters.
361             =head2 Threat Response
363             Some action that undertaken to defend against or counter an Active
364             Threat, in response to a Verified Threat message. This could include
365             short-term DNSBL listing, firewall block entries, or actively disabling
366             the internet connection of the host.
368             =head1 PARTICIPATION
370             The website will provide a link to the current forums
371             of idea exchange for the ThreatNet project.
373             In addition, there is also a #threatnet channel on FreeNode for testing
374             implementations of the reference implementation described above.
376             =head1 AUTHOR
378             Adam Kennedy Eadamk@cpan.orgE
380             =head1 SEE ALSO
382             L, L, L
384             =head1 COPYRIGHT
386             Copyright 2004 - 2008 Adam Kennedy.
388             All ThreatNet modules on CPAN are generally free software;
389             you can redistribute them and/or modify them under the same
390             terms as Perl itself.
392             The full text of the license can be found in the
393             LICENSE file included with this documentation.
395             =cut