Lama, Andrea (2025) Decision-making across scales: the case of shepherding control. [Tesi di dottorato]

Documento PDF PhD_Thesis_Lama_Andrea_MERC.pdf Download (40MB)

Abstract

In a variety of Natural and Artificial systems a wide range of collective behaviours can be observed arising from simple interactions that the single components (or agents) perform in real time with local information. Collective behaviour has been widely studied across disciplines; however, scientists have mostly focused on the emergence and the properties of collective behaviours of isolated complex systems, and on the collective response of complex systems to external stimuli. Less attention has been devoted to understand if and how collective behaviour can solve collective tasks. To achieve collective tasks, the agents need to possess sensing ability to measure some information of the system, computing abilities to think of a suitable action, and actuation capabilities to accordingly reach the desired goal. Sensing, computation and actuation are the three key elements of feedback control systems. As a representative case study where a group of intelligent agents needs to perform goal oriented actions, here the shepherding control problem is considered. The shepherding problem consists of a group of herder agents have to steer in a desired way the collective dynamics of a group of target agents. To achieve success, herders have to make decisions based on local information while aiming for a global control goal, the true essence of distributed control and decision-making. This is a common scenario in Natural and Artificial systems (e.g. group hunting in animals and swarm of robots containing chemical spills in the ocean) and aligns with the growing interest in the field of intelligent active matter, a branch of physics interested in agents which embed the three elements of feedback control in their dynamics. In this thesis, we discuss and advance the understanding on the shepherding problem from different aspects. We study a simple scenario where the herders have to contain the targets in a pre-assigned goal region, and we assume the herders follow a simple, two element control strategy: select one target, and appropriately drive the selected target. At the agent level, we show that this simple two-elements strategy allows the herders to complete the task, and we introduce and investigate the herdability conditions underlying herders' success. We provide scaling laws between number of targets, herders, and size of the sensing range of the herders. From a model-based approach we shift to a learning-based approach, and we train the control policy of the herders using reinforcement learning methods so as to look for optimal strategies. Specifically, we train two distinct policies which correspond to the target selection and target driving decision-making elements. We show that the herders successfully complete the task and that cooperation among herders spontaneously emerges to provide optimal shepherding performance. To discuss the case of systems of arbitrary size, we derive a shepherding field theory, e.g. partial differential equations that study the shepherding dynamics at the coarse-grained, continuum level of description. We show that the decision-making elements in the control strategy of the herders induce a specific type of coupling at the continuum level which is absent in the related physics literature, where such field theories are derived from agent models that lack the crucial ingredients of control-oriented actions. We show that the continuum framework we derive captures the key features of the shepherding task, and that it can accommodate different control-oriented behaviours through simple manipulations, providing a flexible tool to describe at the continuum level a range of control-oriented behaviours. Finally, we present ongoing work where we investigate experimentally what other elements of decision-making can be introduced in the strategy of the herders in addition to target selection and target driving. We perform experiments where human participants take the role of the herders and complete a virtual shepherding task. By analysing the data, we show that memory plays a crucial role in the shepherding strategy adopted by the participants to achieve better performance and to mitigate the finite sensing limitations.

Downloads

Actions (login required)

Modifica documento