Computational Modelling (COMO)
The Computational Modeling Lab, COMO, is headed by Prof. dr. Bernard Manderick and Prof. Dr. Ann Nowé, and has on average 10 PhD students. The research group focuses on the one hand on the modelling of natural phenomena, and on the other hand on developing algorithms for complex problem solving inspired by these natural phenomena. COMO has experience in a wide range of learning techniques such as Reinforcement Learning, Genetic Algorithms, Neural Networks, Support Vector Machines, Bayesian Networks, etc. The research at COMO is organized around two major research tracks: 1) Machine learning techniques for data mining applications 2) Evolution and Learning in multi-agent systems (MAS) In the evolutionary approach, we use biological metaphors like natural selection, co-evolution and evolutionary transitions to investigate problems like how collaborative or cooperative behaviour can emerge in a complex environment of interacting agents that compete for the limited available resources. In evolutionary biology, we often see cooperation at the group level although the individuals are competing for limited resources. This dilemma has been overcome several times in evolutionary transitions like for instance from unicellular organisms to multi-cellular ones. Our goal here is to find the necessary and sufficient conditions under which cooperation can emerge between competing individuals and to use it in the context of MAS. The multi-agent learning research of COMO is based on Reinforcement Learning . The overall goal is to develop a genuine multi-agent RL for multi-stage multi-player games. Meaning that agents are Reinforcement Learners who can, with as few communication as possible, solve complex decision problems as a team. The approach developed at COMO is based on Learning Automata (LA). LA are adaptive decision making devices suited for operation in unknown environments. Originally they were developed in the area of mathematical psychology and used for modelling observed behaviour. In its current form, LA are closely related to RL approaches and most popular in the area of engineering. Since LA combine fast and accurate convergence with low computational complexity, they are applied to a broad range of modelling and control problems. However, the intuitive, yet analytically tractable concept of a learning automaton makes them also very suitable as a theoretical framework for Multi-agent Reinforcement Learning, including ant algorithms. An important issue in Multi-agent Learning is the definition of the solution concept, this is what we want the agents to learn. Very often the Nash Equilibria are suggested as the solutions to look for, however other solutions such as Pareto optimal solutions, or combinations of them might be more interesting from a systems performance perspective. The ESRL technique can both handle common interest games and conflicting interest games. The uniqueness of the approach lies in the fact that agents who only require private information do not have to know in which type of game they are involved, the games might have stochastic payoff, and come with delay. Currently the technique is extended to general multi-stage multi-player games. The experience of COMO in the application of reinforcement and evolutionary learning techniques has been adapted to several application domains, including telecom. Problems such as distributed load balancing is an interesting case study for learning in MAS, it possess the important properties of a MAS because it is distributed, communication is delayed and not for free. Agents have to take autonomously decisions based on limited information, and the state dependent non-stationarity is very apparent. COMO participates in several European initiatives. COMO is a main member of the NoE EvoNet. It is also a member of AgentLink III and of the ALAD SIG, the Special Interest Group within the AgentLink on Agents that Learn, Adapt & Discover. COMO participates in different nationally funded research projects some in close collaboration with industry. COMO is also actively involved in the organization of several Learning Agent Workshops at major conferences (AAMAS, ECML). COMO was also organizer of the previous edition of the European Workshop on Multi-agent systems (EUMAS 2005). Recently several PhD's have been completed at COMO in the domain of Evolution and Learning in Multi-agent systems. More information can be found at http://como.vub.ac.be.