Intrusion Detection Systems are a powerful tool in the fight against threats to the network.  Just like any tool, proper implementation and placement is required to ensure optimum performance.  If the IDS is in the wrong spot, it can miss critical traffic or it can be flooded with false positives that will overwhelm an analyst.  Recently, we saw the latter occur on an enterprise link.

Once we had placed our IDS, we started testing to ensure proper performance.  For testing purposes we gathered a sample set of traffic and ran it through a few pretty standard open-source IDS rule sets to see what kind of alerts we could expect.  The sample traffic set generated several interesting alerts.  What really caught our eye was an enormous number of alerts, which were for “non-compliant DNS traffic.”

This rule, as it’s name implies, flags on any non-standard traffic over port 53.  Seemingly these weren’t all true positives, so we started to dig in and figure out what was happening.

Our first step was to go talk to the network engineers, who were unable to shed any light on why a large amount of non-compliant DNS traffic would be transiting this link. So we then decided to do some traffic analysis of our own to determine what IPs were involved in these conversations.  To start, we pulled out all the relevant alerts containing the IP and port information from the fast.log file (IP’s have been changed to protect the innocent):

cat fast.log | grep –i ‘non-dns or non-compliant’ | tail
[**] [1:2014703:5] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Reserved Bit Set [**] [Priority: 1] {UDP} AAA.BBB.CCC.222:1072 -> XXX.YYY.ZZZ.43:53
[**] [1:2014702:6] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Opcode 8 through 15 set [**] [Priority: 1] {UDP} AAA.BBB.CCC.222:1072 -> XXX.YYY.ZZZ.43:53
[**] [1:2014703:5] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Reserved Bit Set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ:53
[**] [1:2014702:6] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Opcode 8 through 15 set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014703:5] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Reserved Bit Set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014702:6] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Opcode 8 through 15 set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014703:5] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Reserved Bit Set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014702:6] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Opcode 8 through 15 set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014703:5] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Reserved Bit Set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53
[**] [1:2014702:6] ET DNS Non-DNS or Non-Compliant DNS traffic on DNS port Opcode 8 through 15 set [**] [Priority: 1] {UDP} AAA.BBB.CCC.35:1029 -> XXX.YYY.ZZZ.43:53

So now that we had an idea of the structure of the alerts, we cut out all the non-essential information from this output.  In this instance, we wanted to see all the destination IP addresses:

4           AAA.BBB.CCC.39
4           AAA.BBB.CCC.26
4           AAA.BBB.CCC.180
4           AAA.BBB.CCC.182
6           AAA.BBB.CCC.194
16          AAA.BBB.CCC.44
16          AAA.BBB.CCC.92
16          AAA.BBB.CCC.179
20          AAA.BBB.CCC.153
40          AAA.BBB.CCC.154
88          AAA.BBB.CCC.179
88          AAA.BBB.CCC.200
88          AAA.BBB.CCC.74
88          AAA.BBB.CCC.102
88          AAA.BBB.CCC.65
90          AAA.BBB.CCC.101
92          AAA.BBB.CCC.73
92          AAA.BBB.CCC.35
92          AAA.BBB.CCC.38
92          AAA.BBB.CCC.40
92          AAA.BBB.CCC.64
94          AAA.BBB.CCC.180
94          AAA.BBB.CCC.72
94          AAA.BBB.CCC.37
94          AAA.BBB.CCC.37
94          AAA.BBB.CCC.41
98          AAA.BBB.CCC.36
104         AAA.BBB.CCC.196
104         AAA.BBB.CCC.198
1840        AAA.BBB.CCC.36
2502        AAA.BBB.CCC.42
2556        AAA.BBB.CCC.37
6066        AAA.BBB.CCC.43

Now we could see that indeed, there are a few IP addresses that seem to be the major offending destinations.  A little digging on these IP addresses revealed they are registered to a popular security vendor.  We took this information to the infrastructure team to discover if any of this vendor’s products were live on the network.  As it turns out, the products were deployed on the perimeter and were configured to use one of the vendor’s services that required communication over port 53.

Had we known the location or addressing of these security devices before we placed our IDS, we could have easily avoided this scenario.  Having a proper understanding of the network is a critical first step when designing a layout for network monitoring.