Abstract

We propose and investigate a game-theoretic approach to the malware filtering and detector placement problem which arises in network security. Assuming rational and intelligent attackers, we present a two-person zero-sum non-cooperative Markov security game framework as a basis for modeling the interaction between the attackers who generate malware traffic on a network and a corresponding intrusion detection system (IDS). Thus, we establish a formal model of the detector placement problem based on game theory and derive optimal strategies for both players. In addition, we test the strategies obtained in a realistic agent-based network simulation environment and compare the results of static and dynamic placement scenarios. The strategies obtained, especially the one of the IDS, as well as the simulations provide interesting insights to the actual deployment of malware detectors in a large-scale network environment.

@INPROCEEDINGS{Schmidtalpcan2007,
  author = {Stephan Schmidt and Tansu Alpcan and Sahin Albayrak and Achim M{"u}ller},
  title = {{A Monitor Placement Game for Intrusion Detection}},
  booktitle = {Proc. of CRITIS, 2nd International Workshop on Critical Information
	Infrastructures Security},
  year = {2007},
  series = {Lecture Notes in Computer Science},
  publisher = {Springer},
  note = {to appear},
  keywords = {IDS, DAI-Labor},
}