High-performing teams learn intelligent and efficient communication and coordination strategies to maximize their joint utility. These teams implicitly understand the different roles of heterogeneous team members and adapt their communication protocols accordingly. Multi-Agent Reinforcement Learning (MARL) seeks to develop computational methods for synthesizing such coordination strategies, but formulating models for heterogeneous teams with different state, action, and observation spaces has remained an open problem. Without properly modeling agent heterogeneity, as in prior MARL work that leverages homogeneous graph networks, communication becomes less helpful and can even deteriorate the cooperativity and team performance. We propose Heterogeneous Policy Networks (HetNet) to learn efficient and diverse communication models for coordinating cooperative heterogeneous teams. Building on heterogeneous graph-attention networks, we show that HetNet not only facilitates learning heterogeneous collaborative policies per existing agent-class but also enables end-to-end training for learning highly efficient binarized messaging. Our empirical evaluation shows that HetNet sets a new state of the art in learning coordination and communication strategies for heterogeneous multi-agent teams by achieving an 8.1% to 434.7% performance improvement over the next-best baseline across multiple domains while simultaneously achieving a 200× reduction in the required communication bandwidth.
Most current flight systems are dependent on GPS for navigation. Recently, however, navigation in GPS-denied environments has become an area of intensive research. Additional navigation sensor data can be obtained from visual observations (stars or terrain), inertial measurement units, radar, measurements of the local magnetic field, or perhaps even gravity. Absolute and relative positioning via magnetic field measurements have been shown to be viable in many applications including ground navigation, low altitude aircraft flight, and spaceflight. There is greater variability in the magnetic field over shorter distances when flying at low altitude and in ground applications, leading to more accurate positioning. However, ground-based magnetic navigation is often heavily influenced by man-made structures, especially in urban environments. This is not the case for airborne magnetic navigation since the influence of buildings, roads, etc. is negligible for typical aircraft altitudes. For absolute magnetic navigation, the positioning accuracy decreases as altitude increases for a given vehicle velocity, but the observed time variability in the field can be reclaimed by traveling faster through the field. Thus, navigation accuracy becomes a balance of speed and altitude since the higher altitude can be counterbalanced by higher velocity. To understand these effects quantitatively, we explored various techniques to aid a simulated inertial measurement unit with magnetic information. Using a technique known as two-dimensional magnetic map matching, we simulated the performance of airborne magnetic navigation at fixed speed while varying the altitude, flight direction, magnetometer data collection time, reference magnetic map bias error, and type of trajectory (over land or over ocean).