File Coverage

blib/lib/App/DubiousHTTP/Tests.pm

Criterion	Covered	Total	%
statement	15	118	12.7
branch	0	44	0.0
condition	0	16	0.0
subroutine	5	19	26.3
pod			n/a
total	20	197	10.1

line	stmt	bran	cond	sub	time	code
1	1			1	439	use strict;
	1				1
	1				21
2	1			1	3	use warnings;
	1				1
	1				26
3						package App::DubiousHTTP::Tests;
4	1			1	368	use App::DubiousHTTP::Tests::Common;
	1				1
	1				134
5	1			1	300	use App::DubiousHTTP;
	1				2
	1				20
6	1			1	3	use MIME::Base64 'encode_base64';
	1				1
	1				2153
7
8						my @cat;
9						for my $cat ( qw( Chunked Compressed Clen Broken Mime MessageRfc822 Range ) ) {
10						my $mod = 'App::DubiousHTTP::Tests::'.$cat;
11						eval "require $mod" or die "cannot load $mod: $@";
12						push @cat, $mod;
13						}
14
15	0			0		sub categories { @cat }
16						sub make_response {
17	0			0		my $page = <<'HTML';
18
19
20
21
22						HTTP standard conformance tests - HTTP evader
23
24
25						While HTTP seems to be a simple protocol it is in reality complex enough that
26						different implementations of the protocol vary how the behave in case of HTTP
27						responses which are either slightly invalid or valid but uncommon.
28						These interpretation differences is critical if a firewall behaves
29						differently then the browser it should protect because it can be abused to
30						bypass the protection of the firewall.
31
32
33
34						The following tests are intended to test the behavior of browsers regarding
35						invalid or uncommon HTTP responses. And if there is a firewall or proxy between
36						the test server and the browser then it can be seen how this affects the results
37						and if a bypass of the protection would be possible.
38						More information about bypassing firewalls using interpretation differences can
39						be found here.
40
41
42
43						Firewall evasion test - Bulk test with virus payload using XMLHttpRequest
44						Bulk test with innocent payload using XMLHttpRequest
45						Bulk test with innocent payload using script tag
46						Bulk test with innocent payload using img tag
47						Bulk test with innocent payload using iframe tag
48						Various non-bulk tests
49
50
51
52
53
54						Firewall evasion test - Bulk test with virus payload (XHR)
55
56
57
58						This bulk test tries to transfer the
59						href="http://www.eicar.org/86-0-Intended-use.html">EICAR test virus from the
60						server to the client. This test virus is commonly used for basic tests of
61						antivirus and should be detected by every firewall which does deep
62						inspection to filter out malware. Since this virus itself is not malicious it is
63						safe to run this test.
64
65						But, the transfer is done with various kinds of uncommon or even invalid HTTP
66						responses to check if the inspection of the firewall can be bypassed this way.
67						The response from the server will then compared to the expected payload and
68						hopefully all transfers will be blocked either by the firewall or are considered
69						invalid by the browser.
70
71						The test uses XMLHttpRequests to issue the request and get the response. In most but
72						not all cases this shows the same behavior as other HTTP requests by the browser
73						(i.e. loading image, script,...). But to verify that an evasion is actually
74						possible with normal download one should use the provided link to actually test
75						the evasion.
76
77						Run Test with EICAR test virus payload
78
79
80						Bulk test with innocent payload (XHR)
81
82
83
84						This is the same bulk test as the previous one but this time the payload is
85						completely innocent. This test can be used to find out the behavior of the
86						browsers itself, i.e. how uncommon or invalid HTTP responses are handled by the
87						browser. It can also be used to check if the use of proxies changes this
88						behavior and if firewalls block innocent payload if it is transferred using an
89						uncommon or invalid HTTP response.
90
91						Run Test with innocent payload
92
93
94						Bulk test with innocent Javascript
95
96
97
98						Contrary to the previous bulk tests this one is not done with XMLHttpRequest but
99						instead it analyzes which responses will successfully be interpreted as
100						JavaScript by the browser, i.e. by using the "script" tag.
101
102						Run Test with
103						innocent JavaScript payload
104
105
106						Bulk test with innocent Image
107
108
109
110						This bulk test will use "img" tags to download an innocent image to check which
111						uncommon responses can be used to load images.
112
113						Run Test with
114						innocent image payload
115
116
117						Bulk test with innocent Iframe
118
119
120
121						This bulk test will use "iframe" tags to download an innocent HTML to check which
122						uncommon responses can be used to load iframes. Warning!: IE and Edge seem
123						to have serious problems with some test cases here and will render the page
124						unresponsive.
125
126						Run Test with
127						innocent iframe payload
128
129
130						Non-Bulk tests
131
132
133
134						The following tests analyze the behavior of browsers in specific cases, like
135						loading an image, loading a script and loading HTML into an iframe. They offer a
136						download for the EICAR test virus. The subtests in these tests all follow the
137						same style: If the browser behaves like expected (i.e. fails or succeeds) the
138						relevant element (IMAGE, SCRIPT or HTML) will turn green, if it behaves
139						differently it will turn red. Yellow is similar successful as green but marks an
140						uncommon behavior. If this uncommon behavior is not implemented (i.e. load of
141						image or script failed) the element will be grey.
142						When trying to load HTML into an iframe it can happen that the iframe stays
143						empty or contains some error message or garbage instead of "HTML". In this case
144						it failed to load the content.
145
146
147						Which behavior is expected can be seen from the header preceding
148						the relevant section of subtests: if it says that the following requests are
149						VALID it is expected that loading succeeds, on INVALID requests it is expected
150						that they fail. In other words: anything turning red is bad and more so if it is
151						for INVALID requests. Because in this case the browser executes the payload even
152						if the HTTP response was invalid which might often be used to bypass firewalls
153						which behave differently.
154
155
156						HTML
157	0					$page =~s{href="(/[^"]+)"}{ 'href="'. garble_url($1). '"' }eg;
	0
158	0					for( grep { $_->TESTS } @cat ) {
	0
159	0					$page .= " ".html_escape($_->SHORT_DESC)." ";
160	0					$page .= $_->LONG_DESC_HTML;
161	0					$page .= " ID.">Run Test \n";
162						}
163	0					$page .= "";
164	0					return "HTTP/1.0 200 ok\r\n".
165						"Content-type: text/html\r\n".
166						"Content-length: ".length($page)."\r\n".
167						"\r\n".
168						$page;
169						}
170
171						sub auto {
172	0			0		my $self = shift;
173	0					my $type = shift;
174	0	0				return $self->auto_xhr(@_) if $type eq 'xhr';
175	0	0				return $self->auto_js(@_) if $type eq 'js';
176	0	0				return $self->auto_img(@_) if $type eq 'img';
177	0	0				return $self->auto_html(@_) if $type eq 'html';
178	0					die;
179						}
180
181						sub auto_xhr {
182	0			0		my ($self,$cat,$page,$spec,$qstring,$rqhdr) = @_;
183	0		0			$page \|\|= 'eicar.txt';
184	0					my $html = _auto_static_html();
185	0					my ($hdr,$body,$isbad) = content($page);
186	0					$html .= "\n";
235	0					return "HTTP/1.0 200 ok\r\n".
236						"Content-type: text/html\r\n".
237						"Content-length: ".length($html)."\r\n".
238						"ETag: ".App::DubiousHTTP->VERSION."\r\n".
239						"\r\n".
240						$html;
241						}
242
243						sub auto_img {
244	0			0		my ($self,$cat) = @_;
245						_auto_imgjshtml($cat, 'Browser behavior test with img tag', 'ok.png', sub {
246	0			0		my ($url,$id) = @_;
247	0					return "";
248	0					});
249						}
250
251						sub auto_js {
252	0			0		my ($self,$cat) = @_;
253						_auto_imgjshtml($cat, 'Browser behavior test with script tag', 'set_success.js', sub {
254	0			0		my ($url,$id) = @_;
255						#return "";
256						return <<"JS"
257						function(div) {
258						var s = document.createElement('script');
259						s.setAttribute('src','$url');
260						s.setAttribute('id','$id');
261						s.setAttribute('onload','set_load(\"$id\",\"js\");');
262						s.setAttribute('onreadystatechange','set_load(\"$id\",\"js\");');
263						s.setAttribute('onerror','set_fail(\"$id\",\"js\");');
264						div.appendChild(s);
265						}
266						JS
267	0					});
	0
268						}
269
270						sub auto_html {
271	0			0		my ($self,$cat) = @_;
272						_auto_imgjshtml($cat, 'Browser behavior test with iframe including HTML', 'parent_set_success.html', sub {
273	0			0		my ($url,$id) = @_;
274	0					return "";
275	0					});
276						}
277
278						sub _auto_imgjshtml {
279	0			0		my ($cat,$title,$page,$mkhtml) = @_;
280
281	0					my $jsglob = '';
282	0					$jsglob .= sprintf("reference='%x' + Math.floor(time()/1000).toString(16);\n", rand(2**32));
283	0	0				$jsglob .= "fast_feedback = 16384;\n" if $FAST_FEEDBACK;
284	0					my $rand = rand();
285	0					for(@cat) {
286	0	0	0			next if $cat ne 'all' && $_->ID ne $cat;
287	0					for($_->TESTS) {
288	0					my $num = $_->NUM_ID;
289	0					my $xid = quotemeta(html_escape($_->LONG_ID));
290	0					my $url = url_encode($_->url($page));
291	0					my $html = $mkhtml->("$url?rand=$rand",$xid);
292	0	0				$jsglob .= "checks.push({ "
293						. "num: $num, page: '$url', xid: '$xid', "
294						. 'desc: "'.quotemeta(html_escape($_->DESCRIPTION)) .'",'
295						. 'valid: '.$_->VALID .','
296						. 'html: '.($html =~m{^function} ? $html : '"'.quotemeta($html).'"')
297						."});\n";
298						}
299						}
300	0					$jsglob .= "div_title.innerHTML = ' ".html_escape($title)." ';";
301	0					$jsglob .= "runtests()\n";
302
303	0					my $html = _auto_static_html()."\n";
304	0					return "HTTP/1.0 200 ok\r\n".
305						"Content-type: text/html\r\n".
306						"Content-length: ".length($html)."\r\n".
307						"ETag: ".App::DubiousHTTP->VERSION."\r\n".
308						"\r\n".
309						$html;
310						}
311
312
313	0			0		sub _auto_static_html { return <<'HTML'; }
314
315
316
337
338						You need to have JavaScript enabled to run this tests.
339
340
341
342
343
344
345
346
347
348
349						Serious Problems
350						Behavior in Uncommon Cases
351						Debug
352
353
936						HTML
937
938						{
939
940						my (%msg,@map);
941						sub vendor_notice {
942	0	0		0		my $srcip = shift or return;
943	0	0				$srcip =~m{:} and return; # IPv6 not handled yet here
944	0	0				@map \|\| return;
945	0					my $ipn = 0;
946	0					$ipn = 256*$ipn + $_ for split(m{\.+},$srcip);
947	0					for(@map) {
948	0	0				next if $ipn<$_->[1];
949	0	0				next if $ipn>$_->[2];
950	0					my $vendor = $_->[0];
951	0					return ($vendor,$msg{$vendor});
952						}
953	0					return;
954						}
955
956						# load notice on startup
957						if (open(my $fh,'<','vendor_notice.txt')) {
958						my $vendor;
959						while (<$fh>) {
960						if ($vendor) {
961						if (m{^=end\s*$}) {
962						$vendor = undef;
963						} else {
964						$msg{$vendor} .= $_;
965						}
966						} elsif ( m{^=begin (\S+)}) {
967						$vendor = $1;
968						die "message for vendor $vendor already loaded"
969						if $msg{$vendor};
970						} elsif (my ($ip0,$net,$ip1,$vendor) =
971						m{^=map\s+([\d\.]+)(?:/(\d+)\|\s-\s([\d\.]+))\s+(\S+)}) {
972						for my $ip ($ip0,$ip1) {
973						defined $ip or next;
974						my @ip = split(m{\.+},$ip);
975						push @ip,0 while @ip<4;
976						$ip = 0;
977						$ip = 256*$ip + $_ for @ip;
978						}
979						$ip1 = $ip0 + (2 << (32-$net)) if defined $net;
980						push @map,[ $vendor,$ip0,$ip1 ];
981						} elsif (do { s{#.*}{}; m{\S}}) {
982						die "invalid line $_";
983						}
984						}
985						for my $vendor (keys %msg) {
986						die "no source-ip for $vendor defined"
987						if !grep { $_->[0] eq $vendor } @map;
988						}
989						warn "DEBUG: vendor notice loaded for $_\n" for (sort keys %msg);
990						}
991						}
992
993						sub manifest {
994	0			0		my ($self,$cat,$page,$spec) = @_;
995	0					my $data = "00000 \| trivial \| /clen/$page/close,clen,content \| 3 \| trivial response for retrieving body\n";
996	0					for(@cat) {
997	0	0	0			next if $cat ne 'all' && $_->ID ne $cat;
998	0					for($_->TESTS) {
999	0					$data .= sprintf("%05d \| %s \| %s \| %s \| %s\n",
1000						$_->NUM_ID, $_->LONG_ID, $_->url($page), $_->VALID, $_->DESCRIPTION);
1001						}
1002						}
1003	0					return "HTTP/1.0 200 ok\r\n".
1004						"Content-type: text/plain\r\n".
1005						"Content-length: ".length($data)."\r\n".
1006						"\r\n".
1007						$data;
1008						}
1009
1010
1011						1;

File Coverage

HTTP standard conformance tests - HTTP evader

Firewall evasion test - Bulk test with virus payload (XHR)

Bulk test with innocent payload (XHR)

Bulk test with innocent Javascript

Bulk test with innocent Image

Bulk test with innocent Iframe

Non-Bulk tests

".html_escape($_->SHORT_DESC)."

".html_escape($title)."

Serious Problems

Behavior in Uncommon Cases

Debug