STSM WG3 (Roberto Centeno Sánchez)

Start Date: 2010-04-01

End Date: 2010-06-01

Host Institution: City University London

Host Country: UK

Home Institution: Universidad Rey Juan Carlos

Home Country: Spain


A particular type of MultiAgent Systems (MAS from now on) is Open MAS. These systems are usually designed from a global perspective and with a general purpose in mind. However, at design time, the agents that will populate the system are unknown and the number of them may vary dynamically, due to they can join/leave the system at run time. Besides, those agents might be heterogeneous, self-interested, built with different architectures and languages, and even built by different people/companies.
With this in mind, designers cannot assume agents will behave according to the preferences of the system. In order to address this problem, organizational structures have been proposed as a promising solution. In these approaches, authors propose the use of organizational abstractions such as roles, norms, groups, etc. so as to regulate the activity of the participants within an organization. Therefore, the normative systems emerge as a key concept for regulating MAS.
However, norms, from the point of view of agents, are just information that tell them what actions they are (not) allowed to perform in the system. Thus, in order to be effective, norms should be coupled with detection mechanisms — to detect when they are not obeyed — and with penalties/rewards — to be applied when they are violated. In most cases, systems, as well as the norms and their penalties/rewards, are designed before knowing the agents that will populate them. In this sense, the question arises what happens if a current population of agents is not sensitive to these penalties/rewards.
Then, norms cannot be effectively enforced.
To deal with this problem, hard norms can be defined, which agents are not able to violate because the system relies on mechanisms to avoid such violations. For instance, in Electronic Institutions, by means of their infrastructure (AMELI), agents are only able to perform actions that are acceptable in the current state. Nevertheless, in some domains the use of this kind of norms is not feasible due to their complexity and size, it could be impossible
to take into account all possible exceptions. For example, in the traffic domain there exists a norm that says that it is forbidden to pass trough a solid line. However, it is possible and even desirable that sometimes such a norm could be violated, for instance if you can avoid an accident.
Therefore, it is often easier and more efficient to define norms based on penalties/rewards, instead of using hard norms. However, as we said before, these penalties/rewards should be effective for the current population of the system.
Addressing this situation, we propose an incentive mechanism, following our previous work “Organizing MAS: A Formal Model Based on Organizational Mechanisms”, and based on the theory “Economic Analysis of Law”, proposed by R.A. Posner. In this work the author analyzes and checks how normative systems avoid the waste of resources and increase the efficiency of societies. Following these ideas we propose an incentives infrastructure that allows estimating agents’ preferences, and can modify the consequences of actions in a way that agents have incentives to act in a certain manner. Employing this infrastructure, a desirable behavior can be induced in agents to fulfill the preferences of the system.
So, the main objective of the visit at the City University, collaborating with Dr. Eduardo Alonso, is to endow the incentive mechanism, stated before, with reinforcement learning techniques. In such a way, the mechanism will be able to i) discover an agent’s preferences, and; ii) selecting the appropriate incentive to induce agents to behave in a certain manner. That is, the mechanism will learn how to persuade agents modifying attributes that influence their utility within the system.

STSM report