
==== Front
Sci Rep
Sci Rep
Scientific Reports
2045-2322
Nature Publishing Group UK London

39294261
71931
10.1038/s41598-024-71931-0
Article
Linking cognitive strategy, neural mechanism, and movement statistics in group foraging behaviors
Urbaniak Rafal 1
Xie Marjorie 1234
Mackevicius Emily emily@basis.ai

14
1 Basis Research Institute, New York, 10026 USA
2 https://ror.org/03efmqc40 grid.215654.1 0000 0001 2151 2636 Arizona State University, School for the Future of Innovation in Society, Tempe, 85287 USA
3 https://ror.org/0316hvk63 grid.281219.1 0000 0004 0444 3589 New York Academy of Sciences, New York, 10006 USA
4 https://ror.org/00hj8s172 grid.21729.3f 0000 0004 1936 8729 Columbia University, New York, 10027 USA
18 9 2024
18 9 2024
2024
14 2177029 12 2023
2 9 2024
© The Author(s) 2024
2024
https://creativecommons.org/licenses/by-nc-nd/4.0/ Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Foraging for food is a rich and ubiquitous animal behavior that involves complex cognitive decisions, and interactions between different individuals and species. There has been exciting recent progress in understanding multi-agent foraging behavior from cognitive, neuroscience, and statistical perspectives, but integrating these perspectives can be elusive. This paper seeks to unify these perspectives, allowing statistical analysis of observational animal movement data to shed light on the viability of cognitive models of foraging strategies. We start with cognitive agents with internal preferences expressed as value functions, and implement this in a biologically plausible neural network, and an equivalent statistical model, where statistical predictors of agents’ movements correspond to the components of the value functions. We test this framework by simulating foraging agents and using Bayesian statistical modeling to correctly identify the factors that best predict the agents’ behavior. As further validation, we use this framework to analyze an open-source locust foraging dataset. Finally, we collect new multi-agent real-world bird foraging data, and apply this method to analyze the preferences of different species. Together, this work provides an initial roadmap to integrate cognitive, neuroscience, and statistical approaches for reasoning about animal foraging in complex multi-agent environments.

Subject terms

Cognitive neuroscience
Behavioural ecology
100000893 Simons Foundation Society of Fellows 100000065 U.S. Department of Health & Human Services | NIH | National Institute of Neurological Disorders and Stroke (NINDS) 1K99NS131256-01 Mackevicius Emily issue-copyright-statement© Springer Nature Limited 2024
==== Body
pmcIntroduction

One of the oldest functions of the embodied nervous system is exchanging information with other animals in order to successfully forage for food. When animals have compatible goals, they often form collaborative groups, exchanging information about the location of food, water, and predators. Over evolutionary time, collaborative exchange of information has gotten increasingly sophisticated, as ecosystems and societies expanded. Each species is genetically predisposed to perceive certain types of information, and some species, including humans, flexibly learn new ways of representing information from each other. This information may come in the form of pheromone trails, vocalizations, or digital content. From insects to humans, animals use information from the group to make decisions about how to forage for food.

For several reasons, birds are an attractive subject for studying animal cognition within collaborative groups. Birds are highly intelligent and communicative, often operate in multi-agent or even multi-species groups, and occupy an impressively diverse range of ecosystems across the globe. Small birds that spend winter in cold climates face extreme survival pressures and congregate in multi-species flocks that help them survive. To stay warm overnight, these birds burn through about 10% of their body weight, which they must regain the next day through foraging for food1, all while avoiding predators. There is a rich literature of experimental and theoretical work on the energy management strategies that individual birds employ to survive the winter2,3, including remembering the locations of hidden food caches, which relies on the hippocampus4, a brain region also important for memory in humans5. Birds that are part of a group eat more and spend less time scanning for predators than birds that are separated from a group, and they listen to other birds to decide when to forage6,7. Understanding this type of complex real-world cognitive behavior is an important component of uncovering principles of how the brain and mind work8–15.

Statistical models of groups of birds have successfully captured certain behaviors using a minimal set of rules—not foraging behaviors, to our knowledge. Three simple automaton rules are sufficient to generate surprisingly realistic-looking flocks of birds16. Flight patterns of murmurations of starlings are well described by local neighbor interactions17. Some statistical models of collective animal behavior, based on GLMs, allow for the possibility of switching between different hidden states18,19. There has also been a long-standing effort to quantify the distribution of step sizes when animals forage, since there are statistical challenges in assessing whether or not a distribution is heavy-tailed20. This paper borrows several aspects of these statistical models of collective behavior to describe multi-agent foraging, in combination with Bayesian methods to capture contexts in which birds’ behavior depends both on the collective dynamics, as well as cognitive factors such as memory of food locations.

From a cognitive perspective, foraging decisions depend on what agents find rewarding, and what they believe about the world. This has been formalized in reinforcement learning (RL) agents, who learn an internal representation of which states are rewarding, and which states are adjacent to each other. These agents make rational decisions, in the sense that they maximize the expected future value of their actions. RL agents perform well in a variety of foraging-related tasks21–23. The concept of value offers a method for precise calculations of what decisions are optimal in different conditions24. Furthermore, it is possible within this framework to define additional quantities that agents may value in a multi-agent setting, such as social information25, and to account for goals that change dynamically26–28.

In the multi-agent foraging context, it is important to capture information flow between agents. Different types of information flow generate measurably different foraging behaviors29. Social cognition is often described as the ability to do inverse reinforcement learning, that is, to infer another agent’s internal representations from its behavior30,31. While inverse RL gets difficult with the complexity of the RL task and the richness of the prior knowledge32, some success has been achieved in this field within the context of neural network driven foraging behaviors33–35.

From a neuroscience perspective, there is compelling evidence across different species to suggest which neural circuits compute which information36–39, though most neuroscience studies of foraging are restricted to single agents. Foraging is a well-established behavioral paradigm for studying neural representations in the hippocampus. Precise ‘place cell’ maps have been observed during foraging in rats, bats, and birds40–42. In addition, our work and others have shown that the hippocampus dynamically represents food locations43–47, as well as the locations of other individuals48. During foraging, the hippocampus appears to integrate relevant information into a predictive map representation, highly similar to representations used in RL algorithms to evaluate the value of different possible states, specifically the Successor Representation38. Several mechanisms have been proposed for how this representation is computed and learned within a biologically plausible neural network49–51. It is also possible to learn a vector of reward-related output weights from hippocampus-like representations using Hebbian plasticity52.

The goal of the combined cognitive/neural/statistical framework is to provide a family of possible descriptions of foraging behavior that can be adapted to capture different species and environments. The statistical approach is useful for disambiguating different hypothesized models, even with limited data. Building on previous work, we demonstrate how abstract cognitive descriptions of multi-agent foraging behavior can be mapped to a biologically plausible neural network implementation and to a statistical model. In the cognitive (RL-based) description, each state is assigned a value, which can depend on a variety of features, such as the locations of food or information about other agents. We identify analytically how features that agents value are represented in a biologically plausible neural model, and a statistical model of foraging movements. Within the combined cognitive/neural/statistical framework, it is possible to add and combine different features that may influence an agent’s behavior, such as food locations, or locations of other agents. Multi-agent foraging behavior can vary widely across different species and environments. This paper tests the framework on a variety of simulated datasets, as well as insect and bird datasets.

First, we simulate different types of multi-agent groups (random walkers, followers, and hungry birds), and find that it is possible to distinguish what different agents value by performing statistical inference that uses only their simulated foraging trajectories as data. The statistical perspective allows us to infer the extent to which the proposed features explain the behavior of a particular type of agent in a particular environment.

Furthermore, in a multi-agent context, communication of information between foragers is an important feature of group-level behavior, as is the impact of different environmental conditions. We thus explore how one might infer to what extent agents communicate with each other to facilitate foraging. We first ask whether the benefit of communicating information would be different for different environments, using multiple simulations with a range of communication-related hyper-parameters. In environments where food is highly clustered, it takes longer for birds to find food, but in all environments, using information communicated from other birds improves foraging success. The Bayesian inference methods are able to correctly compare the extent to which simulated agents communicate about the locations of the rewards.

We also apply our communication analysis to real-life locust foraging data53. The main result of that paper was that locusts use both socially derived and personally acquired evidence. This result emerged from analyzing locust data using a drift-diffusion model, where the accumulation of evidence is modeled as a leaky process. Our study is able to reach a similar conclusion by using lower-level models that do not rely on the adequacy of differential equations to estimate the role of social evidence (seeing other locusts reach the reward) and individual evidence (presence of food traces): we find that information about what other locusts are doing (whether they are feeding themselves) is predictive of locust behavior in addition to other factors such as distance to rewards or other locusts.

Finally, we record video data of multi-species multi-agent foraging birds in an outdoor winter environment and extract their trajectories. We use this data to estimate coefficients that represent how much different species of birds value being in different proximity to other birds. These proof-of-concept analyses demonstrate that, with additional data, this analysis framework can be used to compare decision-making parameters that govern foraging behavior across different species and environments.

Results

Translating between cognitive, neural, and statistical descriptions of foraging

We set out to translate an abstract cognitive description of foraging behavior into concrete empirically testable predictions about neural activity and statistics of movements. A cognitive policy can be expressed as a function that takes in a state S and returns a prediction of what action A the bird will do in that state. A combination of different factors may explain why an animal takes a particular action. Neural and statistical descriptions of foraging also predict actions based on states, but each type of description can appear to have quite a different language and notation, so it can be difficult to translate between them.

Suppose a bird’s action is influenced by the location of food. A cognitive description might state that the bird values food. A neural description might state that hippocampal representations activate downstream neurons in a way that drives the bird to navigate toward food locations. A statistical description might state that food location is a good predictor of where the bird will move. Starting from a formulation of the cognitive model in terms of reinforcement learning (RL) agents, we will translate this notation into neuroscience and statistical formulations.

Notation

Cognitive descriptions have been formalized in RL models. An RL agent learns an expected reward r(S) for each state, as well as a ‘world model’ for predicting future states St+1=T(At,St), where T is a state transition function, mapping from a given state to eligible next states, and At is the agent’s action. The agent computes the expected future value of each possible state V(St′)=∑t=0∞γtr(St′+t), where γ (between 0 and 1) is a temporal discounting rate. In different types of RL, there are different ways of estimating expected future states, including complex models of how future action policies will impact future states. In this study, in order to draw correspondences with neuroscience work, we will focus on a simpler Successor Representation form of RL38, where we represent estimates of future states as policy-independent, using a Markov transition function capturing the set of states available given the current state. At each step, the agent makes a rational decision to act in a way that maximizes expected future value. Specifically, at state S, the rational policy is to choose action Acog=argmaxA(V(T(A,S))).

A neuroscience description of how the brain makes foraging decisions involves specifying how the brain represents and transforms information through a neural network model of different brain areas, from perceptual areas to intermediate areas to motor areas. Perceptual brain areas represent a state S, which may get transformed across different brain areas, with the activity of motor output units determining Aneuro. Specifically, Aneuro=N4(N3(N2(N1(S)))), where Ni are neural network layers representing sequentially connected brain areas ranging from perceptual to motor.

A statistical description employs predictors to probabilistically predict the next action. The predictors could take arbitrary functional form, and in practice are related to observable variables, such as the location of other birds in the environment or the location of food. A statistical model of this sort can be formulated as a specification of P(Astat=A|S)=h(c1f1(A,S)+c2f2(A,S)+…+cnfn(A,S)), where Astat is where the bird moves next, A is any eligible next move, h is a monotonic function, and fi are predictors with coefficients ci representing the contribution of the respective predictor.

Why is it useful to “translate” a cognitive description into a neural or statistical description? Cognitive variables are more abstract than measurable quantities such as the firing rates of neurons, or the statistics of animal movements. Once we translate abstract cognitive variables into measurable quantities, we can use data from real-world multi-agent foraging behavior to inform cognitive descriptions, which are more interpretable.

With this notation in mind, we will first construct biologically plausible neural networks for which Aneuro=Acog, then define statistical models for which the most probable value of Astat is Acog, so we can infer parameters in the neural or cognitive models using empirical data.Figure 1 Neural network implementation of an RL agent in a grid world environment. Neural population vectors are re-shaped to match the 2D environment. (A) State representation, in the form of a one-hot vector of neural activity. Heatmap shows activity across the population of neurons in layer N1 when the agent is at state S (located at the white dot). (B) World model, in the form of a Successor Representation M. Heatmap shows activity across the population of neurons in layer N2 representing possible future states given state S. (C) Value computation. Heatmap shows activity in N3 representing value as a function of the state. (D) Action selection. Layer N4 samples possible actions, and selects the action that maximizes value. (E) Decomposition of value function into the sum of multiple factors, such as value from food (first term), and value from proximity to other agents (second term). This function is also accessible within a simple neural network architecture, with the equations shown.

A biologically plausible neural network that implements an RL policy

We construct a network model composed of four sequentially connected subnetworks (N1 through N4). The core of the model is a biologically plausible model of the hippocampus that computes the Successor Representation38,49–51. The Successor Representation, used in RL models, is a predictive map that represents the temporally discounted expected occupancy of future states, from any starting state. In a world with N states, the Successor Representation is an N x N matrix, defined as M=∑t=0∞(γT)t=(I-γJ)-1, where T is the transition matrix between states and I is the identity matrix. In the simplest formulations of biologically plausible neural networks that compute the Successor Representation, the inputs are one-hot vectors, so we construct N1(S) to be a sparsifying network that returns a population vector ϕ→S that is mostly zeros, with a one at the bin corresponding to state S. A weight vector w→ stores the expected reward of each state, such that w→⊤ϕ→S=r(S). The value can be computed using the Successor Representation in the following way V(S)=w→⊤Mϕ→S. This can be computed using simple linear networks N2=M, and N3=w→⊤. Putting it together, we construct a biologically plausible neural network that computes value N3(N2(N1(S)))=V(S). What remains is using value to inform action in a biologically plausible way.

When animals decide what action to take, they sample their local environment and evaluate their options52. This process can be interpreted as an agent computing the argmax function of possible options. An agent at state S perceives not only the current state ϕ→S=N1(S), but also adjacent states, N1(T(A,S)), by actively looking or sniffing in the direction of possible actions. Action-selection circuits in the brain are thought to use winner-take-all network dynamics54 to compute the argmax, so N4 computes the argmax function over possible actions. This gives us Aneuro=argmaxA(V(T(A,S)))=Acog, which yieldsan end-to-end simple neural implementation of an RL agent. Using this implementation, we can model neural representations in different areas of the brain (from sensory areas, to hippocampal memory areas, to decision-making areas), and also interpret the representations as components of an RL model (Fig. 1A–D).

A statistical interpretation of an RL policy

In order to infer properties of agents’ behavioral policy from data, we set out to frame descriptions of RL foraging agents in a way susceptible to statistical inference. In particular, we wanted to infer aspects of agents’ value function, which determines their behavioral policy (where they will move next). RL descriptions can decompose value into a collection of different factors, such as food, and proximity to other agents (Fig. 1E), each with different coefficients, representing how strongly the agent values each factor. From a statistical point of view, each coefficient represents how strongly that factor predicts the agent’s behavior. For further detail of the statistical formulation, and relation to cognitive and neural descriptions, see Appendix A.

Simulations of multi-agent foraging behavior

We first simulate three different types of foraging groups: hungry, follower, and random birds (Fig. 2A). These groups correspond to three types of things birds might value – food, proximity to other birds, and nothing. At each time in the simulation, each agent moves to a new location that is chosen within a local visibility range. Random birds move randomly within the visibility range. Hungry birds choose randomly among the 10 locations in their visibility range that are closest to food, and follower birds choose randomly among the 10 locations in their visibility range closest to other birds (See Appendix B for simulation algorithms).

Our goal was to test whether it is possible to infer what agents value from observing their movements and the reward locations. We fit a Bayesian model of where each agent will move next, using possible components of a value function (food and proximity to other birds) as statistical predictors. Each predictor is a function over states in the environment, computed for each agent at each timestep. For example, the proximity function is computed by assigning a “proximity score” to each state based on a given agent’s location and the locations of the other agents. Example proximity scores are shown in Fig. 2B, and food traces are shown in Fig. 2C. The mean prediction of where an agent will move next is modeled as a linear combination of the predictors, and stochastic variational inference (SVI) is used to infer the coefficientof each predictor. To handle heteroskedasticity, the standard deviation is also modeled linearly, so that the model does not assume constant standard deviation in the predictive distribution. For further detail on constructing the derived predictors, see “Methods”. The coefficients estimated using this Bayesian model allow us to disambiguate random, hungry, and follower birds, with hungry birds having a high coefficient on the food trace, follower birds having a high coefficient on the proximity score, and random birds having low coefficients on both (Fig. 2). In summary, we demonstrate a proof of concept how mechanistic hypotheses about what agents value can be inferred from observations about their movements and information about their environment such as food, other foragers, and predators.Figure 2 Inferring the values of random, hungry, and follower birds. (A) Example trajectories for simulated random, hungry, and follower birds. Yellow indicates food locations. (B) Proximity scores (gray) for points visible to bird 2 at a single time frame. (C) Food traces for the same frame as in (B). (D) Coefficient values representing the contributions of proximity (p) and trace (t) predictors to predicting the birds’ trajectories. Coefficient values were sampled from the posterior estimated using SVI on synthetic datasets of random, hungry, and follower birds.

Figure 3 Modeling the effect of information-sharing on group foraging outcomes. (A) Trajectories from a single run of simulated birds (n=8) that either search for food independently (top), or communicate food locations (bottom, communication parameter = 0.6) in an environment with high spatial food clustering. Reward locations are in yellow in a single 4x4 food patch. (B) Histograms of samples from the inferred posterior distributions for coefficients of proximity (p), food trace (t), and communication (c), for non-communicators (top), and communicators (bottom, communication parameter 0.6). Medians are shown as dashed lines. (C, top) Average time to first find food, across simulated groups of birds with different communication parameters. Colors indicate the size of the food patches in the environment. Lines indicate linear fits obtained using Bayesian models with regularizing priors. (C, bottom) Posterior distribution of slopes of the linear model estimated using Stochastic Variational Inference (see Appendix C for code snippets). Across environments with different patch sizes, there are negative estimated slope coefficients, that is, increased communication corresponds to decreased time to first food.

Next, we investigate the effect of information-sharing on foraging success under different environmental conditions. In real-world environments, animals may use not only their own sensory information but also communication with other animals to inform their decisions, for example, visual observations, or listening to other birds’ calls. We simulated grid world environments with food patches of varying degrees of spatial clustering, controlling for the total amount of food in the environment. In each environment, there were 16 total food items, distributed randomly in patches of size 1 × 1, 2 × 2, or 4 × 4.

We parameterized the extent to which agents share information about food locations. In these simulations, agents follow a policy A=argmaxA(V(T(A,S))), where their estimate of expected reward includes both food that they can directly observe (within a radius of 5 steps), as well as perceiving other birds eating at farther locations. In the real world, this could be achieved by observing other birds and/or listening to their calls. The weighting of social information (reward locations communicated by other birds), compared to individually observed information, is given by the communication parameter, which ranges from 0 (no communication) to 1 (full reliance on social information). Birds that communicate appear to navigate more directly to food locations than birds that search independently (Fig. 3A). We analyzed these simulated birds using a similar Bayesian model as in the Random/Hungry/Follower bird analysis but with an additional communication term, which updates when other birds arrive at food locations (Appendix B).

We simulated groups of birds with different values for the communication parameter and tested what communication coefficient was inferred by the Bayesian model (note that the inferred communication coefficient can be interpreted as the relative weighting of communication compared to other factors such as food trace, and is not expected to be numerically identical to the communication parameter). The model correctly infers non-zero communication coefficient (c) for simulations in which the communication parameter was set at non-zero values, and near-zero values for simulations set at zero values (Fig. 3B). This demonstrates the ability of our methods to at least recover this model parameter from synthetic data. Note that the model also infers that communicator birds have a non-zero coefficient for the proximity score (p).

To assess the role of communication across different environments, we simulated groups of birds with different values for the communication parameter, across environments with different degrees of food clustering. Foraging success was measured by the average time it took birds to reach their first food item. Overall, finding food took longer in environments where food was clustered into larger patches. In all environments tested, albeit to different degrees, communication was negatively correlated with the time it took to reach food (Fig. 3C), meaning that the birds found food faster the more they relied on information from other birds. This suggests that there is a potential in first building a model of agent behavior, then calibrating its parameters using low-level data, and next using the model in forward simulation mode to study a range of queries at varying level of abstraction: for instance, investigating how successful the identified strategy would be under different counterfactual scenarios.

Real multi-agent foraging datasets

Before testing this analysis framework on new real-world bird data, we tested it on previously published locust data53. A particularly interesting claim of that paper is that locusts integrate socially derived information (observations of other locusts feeding) in their decisions of where to forage. Günzel et al. model the locust decision mechanisms using a drift-diffusion dynamical systems model. Our goal was to show that our method can replicate this general conclusion.

We asked whether the cognitive RL model and Bayesian inference procedure could identify the use of social information in the locust behavior data set. To model the locust behavior, we used the same model as we used to analyze simulated bird communication (Fig. 3), with parameters set to mimic the assumptions of the paper (sight radius and preferred distance matched to locust observations), illustrating how expert knowledge can be relatively easily integrated with the framework. The posterior distributions obtained from Bayesian inference indicate that there is indeed social information used in the locusts’ decisions (Appendix D). This illustrates the point that while simple, the framework can still capture a relatively rich real-world behavior.

Studying real-world cognitive behaviors such as avian cognition is essential for understanding cognition and the brain, but it can be challenging to acquire this necessary data8–15.In larger birds such as cormorants, GPS trackers can continuously record foraging movements55, but these devices weigh more than many small wintering birds, so they would be impossible for them to carry. Machine vision has emerged as a revolutionary new technology for tracking animal behavior56,57. These methods are limited by the field of view of the cameras, so large-scale movements such as migration cannot be tracked, but within the field of view, they track behavior at high spatiotemporal resolution, with the added benefit of being non-invasive. Foraging behavior occurs on a smaller scale than migration, so is amenable to these methods.

We acquired videos of multi-agent multi-species groups of birds performing foraging behavior in the winter. Standard RGB videos were recorded simultaneously with thermal videos (FLIR E54 camera), which can be especially effective in detecting movements of birds58. The RGB videos are used for identifying features of the environment such as terrain and species, while the thermal videos are used for tracking the movement of birds (Fig. 4A). Even in a complex natural setting with a highly varying background, the thermal videos are useful for recovering position information, since birds stand out as warm objects against the cold ground. The RGB videos capture terrain information about the environment that birds are foraging in, and enable the identification of different species, using automated systems59. It is possible to observe trajectories of foraging birds by computing a simple maximum projection of thermal images (Fig. 4B). In addition, we adapted a deep-learning-based multi-agent tracking pipeline60 to automatically track bird locations in these videos (Fig. 4C,D).Figure 4 Tracking of multi-species foraging behavior, using concurrent RGB and thermal imaging. (A) RGB videos are used for identifying terrain and bird species, and thermal videos are used for tracking bird movements. (B) Maximum projection of approximately 30 seconds of thermal video data, showing the trajectories of birds. (C) Trajectories of a group of ducks (Mallards, Anas platyrhynchos) (D) Trajectoris of a mixed-species group of White-Throated Sparrows, Zonotrichia albicollis and Tufted Titmice, Baeolophus bicolor. (E,F) Histograms of inter-bird distances for the two groups. (G,H) Inferred posterior distributions of the proximity coefficient for different settings of the preferred proximity, ranging from 10 to 80 pixel units.

We wondered whether the analysis framework could be used to analyze different types of foraging behavior in groups of birds. In order to test the general applicability of the analysis approaches, we set out to test them on species with quite different foraging strategies, ranging from dabbling in ponds to foraging in leaf litter and trees. That said, we cannot guarantee full universality to all bird species and foraging strategies. For example, foraging strategies in dense underbrush or underwater would likely have too much occlusion to be susceptible to computer vision tracking. We chose two short video recordings, one of a group of Mallard ducks (Anas platyrhynchos, n = 17 individual birds), and one of a multi-species group of small songbirds (White-Throated Sparrows, Zonotrichia albicollis and Tufted Titmice, Baeolophus bicolor, n = 10 individual birds). First, we computed distributions of inter-bird distances (Fig. 4E,F). We next wondered whether the data was consistent with different foraging preferences, specifically different preferred proximity (Fig. 2B plots example proximity function, detailed definition in “Methods”).   We fit variations of the Bayesian model described above, with different settings for the preferred proximity distance, ranging from 10 to 80 distance units (Fig. 4G,H). An important future direction will be using a multi-camera setup to convert these trajectories to calibrated real-world 3D coordinates. In the duck data, a proximity preference of 40 had the strongest weighting, while in the sparrow and titmouse data, 30 carried the strongest weighting. Interestingly, both datasets showed a negative weighting for large distances (i.e., birds preferred to avoid large separations). These results were consistent with the empirical distributions of inter-bird distances (Fig. 4E,F), and also provide (1) added explanatory power, as they are fit on individual foraging decisions, and allow for the possibility that some common inter-bird distances may not be strongly predictive of where birds will go, and (2) connection to a framework where proximity preference can be combined with other foraging mechanisms in a statistical analysis of movement data, similar to what we showed in the synthetic data. While more data is necessary to resolve how well these preference profiles generalize to other settings, this framework offers a promising toolkit for inferring what preferences drive behavior within real-world multi-agent behaviors.

Methods

Translating between descriptions

Translating between different descriptions was done analytically. Notation used in the literature was investigated and compared, and effort was made to select notation that was similar across descriptions. Further detail in the Results section and Appendix A.

Derived predictor scores

In the statistical model, derived predictors are used to predict where agents will go next, based on factors such as food locations and the locations of other agents.

We first use an external visibility range hyper-parameter, which determines how far the birds can “see” to assign non-zero visibility scores to points in the birds’ vicinity. We employ a cosine decay function so that birds can better see closer things. Each visible location is assigned a trace score: 1 if it contains a reward, with exponential decay for distances farther away from rewards (length constant 6 units). To capture birds’ preferences for proximity to other birds, which often involve a preferred distance, where being too close or too far is less desirable, we construct a proximity score function. The proximity score is parameterized by three numbers: where it takes value zero, what the optimal distance is, and an exponential decay rate for larger distances. Suppose gw, opt pd stand for “getting worse”, “optimal”, and “proximity decay”. The scoring function used is:proximity(d)=sinπ2·gw·(d+3·gw)ifd≤gwsinπ2·(opt-gw)·(d-gw)elifd≤gw+1.5·(opt-gw)sinπ2·(opt-gw)·1.5·(opt-gw)·exp-pd·d-opt-0.5·(opt-gw)else

Scores accumulate additively across sources (e.g. being close to two rewards results in higher trace scores, examples in Fig. 2).

Inference task

The prediction task is as follows: Each bird b and each time frame t, given her range, uniquely determines the points available to b at t, {pi=⟨xi,yi⟩|pi∈Range(b,t)}. Each such pi gets a trace score, trace(pi), and a proximity score, proximity(pi). At time t+1 the bird moves to a new position p⟨b,t+1⟩. What we ideally want to be able to predict using trace(pi) and proximity(pi) is a transformed distance of pi from where the bird will go next, p⟨b,t+1⟩, for all points pi in Range(b,t). More formally, as the output variable we take:accuracy(pi)=-(xi-x⟨b,t+1⟩)2+(yi-y⟨b,t+1⟩)2max(xi-x⟨b,t+1⟩)2+(yi-y⟨b,t+1⟩)2+1,

so that score 1 is assigned to the point to which the bird will go next, and score 0 is assigned to the available points that are the furthest to where she will go next.

Given the scale of the grid, we fixed the hyper-parameters at fairly sensible values.(Specifically, in analyzing the simulated data, the following parameters were used: rewards decay = .5, visibility range = 9, maxStepSize = 4, negative proximity score starts at 1.5, optimal proximity = 3, proximity decay = 1. Additional options and visualizations are available using the notebooks on GitHub.) For each dataset, trace and proximity scores were computed for points in birds’ ranges, then normalized by dividing by their maximal values to ensure equal coefficient interpretability. Appendix C contains further information specifying the Bayesian models used.

Video recordings of groups of birds

Video recordings of groups of birds were obtained in Central Park, under a research permit from the City of New York Parks & Recreation Natural Resources Group. This research is purely observational and does not alter or influence the biology, behavior or ecology of the study animals or other species, so does not require an IACUC (Institutional Animal Care and Use Committee) protocol, as determined by the Columbia University IACUC. Videos of multi-species bird foraging behavior was obtained using a tripod-mounted FLIR handheld E54 thermal camera, and simultaneous RGB video recordings from a tripod-mounted Google Pixel 6a camera. Each video recording lasted up to 30 minutes, and camera equipment was attended at all times. The locations of birds within the thermal videos were tracked using the SLEAP algorithm60. To correct for cases where the algorithm confused the identity of two birds, data was post-processed using custom Python scripts, and checked manually. The RGB videos were used to determine species identities.

Discussion

Summary

We have presented a strategy for bridging cognitive, statistical, and neural descriptions of multi-agent foraging behavior. Starting from an abstract cognitive description of how each agent assesses what is valuable and decides what to do, we implemented each agent as a biologically plausible neural network. We simulated a group of these biologically plausible agents across different environments. From the statistical perspective, each statistical predictor corresponds to a component of the cognitive agent’s value function. Using the statistical model, we could infer properties of the value functions of different agents. We also validated in replication, by obtaining low-level counterparts of the main locust study53 hypothesis. Finally, we collected and analyzed high-resolution thermal and RGB videos of multi-agent multi-species groups of birds foraging in an outdoor setting. These results pave the way for a more integrated and comprehensive understanding of multi-agent foraging behavior by offering a family of descriptions that can be adapted to capture different species and environments.

Limitations of the current approach

Observing birds’ movements alone will generally not be sufficient to uniquely determine their cognitive strategies. To properly constrain our models, it will be useful to integrate prior information about distributions of food, movement statistics, metabolic rates, calorie consumption, hippocampus size, ecological variables, evolutionary context, and other expert knowledge from relevant disciplines. The fact that our approach employs flexible Bayesian generative models in principle allows for the inclusion of rich prior information, and the full expressiveness of these approaches will be an interesting direction for future study.

As we increase the complexity of our hypothesis space, we will need increasingly powerful inference algorithms. The current study tests one specific form of inter-agent communication, and one specific form of follower-tendency. In principle, this framework could support a richer collection of possible communication and follower mechanisms that may drive foraging decisions, as additional terms in the model, with additional coefficients to infer. However, expanding the model will require increasingly powerful inference techniques. Two particularly promising strategies involve the ability to infer hidden states that drive behavior61 and the ability to specify generative models programmatically62–64. To reduce complexity, it may make sense to include more coarse-grained models than the agent-based models in this paper, such as dynamical systems models.

When it comes to observation in natural environments, multi-species bird foraging, in particular, offers many interesting technical challenges, if the data is to be rich enough to support further fine-grained analyses of the sort discussed in this paper. Building off of recent progress in machine vision for animal tracking60,65–67, we will need to extend machine vision tracking methods to large 3D settings, at high enough resolution to identify particular species59, or even individuals68.

Expansion to other types of subjects

At least in principle, the key aspects of our modeling approach are not limited to avian or insect subjects. The simulations and analyses here involved small or medium-sized groups of agents foraging in 2D grid-world environments, which could apply to various multi-agent groups across the animal kingdom. The value function can be adapted to a particular animal clade by modifying the internal components based on features particular to that animal’s environment and its particular neural and cognitive resources. This framework could potentially even be applied to studying some human behaviors. For example, RL models of human planning based on the hippocampus also use Successor Representations69,70. Models of humans performing visual information-foraging tasks may decompose the value function into different visual features each given a different attention weight71.

Further directions

What strategies do specific groups of foragers use? The answer to this question will differ for different species and environments. It depends on variables such as the foragers’ diet, the distribution of food in the environment, and the foragers’ cognitive capacity. For example, different species of overwintering birds have different spatial memory abilities and relative hippocampus sizes and recruitment72,73. Within the unified framework outlined in this paper, it is possible to combine information about relative hippocampus size with information about movement and to build mathematical explications of foraging strategies into the models to be evaluated in light of data. In the future, it will also be important to relate the framework described here to evolutionary and developmental perspectives, and analyze multi-agent behavior across different types of real-world environments.

Supplementary Information

Supplementary Information.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-024-71931-0.

Acknowledgements

ELM received support through the Simons Society of Fellows, as well as the NIH NINDS (1K99NS131256-01 and 4R00NS131256-03). Some computations have been performed on equipment funded by the Polish National Science Centre OPUS grant 2021/41/B/HS1/01814. We are grateful for funding support to Basis Research Institute from private donations. We appreciate the advice provided by Sam Witty and other Basis research scientists in the development of the code base. We thank Palka Puri for helpful comments on the manuscript.

Author contributions

All authors contributed to conceptualizing the study, analytic derivations, data analysis, writing, and reviewing the manuscript. R.U. performed random-hungry-followers simulations, statistical modeling and inference, and code base development. M.X. performed communicator bird simulations. E.M. collected and processed the fieldwork video recordings of groups of wild birds.

Data availability

The datasets generated during the current study are available in a GitHub repository https://github.com/BasisResearch/collab-creatures, which also includes all code for the simulations, data transformations, animations, and inference, with accompanying notebooks.

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Rafal Urbaniak and Marjorie Xie.
==== Refs
References

1. Chaplin SB Daily energetics of the black-capped chickadee, Parus atricapillus, in winter J. Comp. Physiol. 1974 89 321 330 10.1007/BF00695350
Chaplin, S. B. Daily energetics of the black-capped chickadee, Parus atricapillus, in winter. J. Comp. Physiol. 89, 321–330 (1974).
2. Pravosudov, V. V. & Grubb, T. C. Jr. Energy management in passerine birds during the nonbreeding season: A review. Curr. Ornithol. 189–234 (1997).
3. Brodin A Theoretical models of adaptive energy management in small wintering birds Philos. Trans. R. Soc. B Biol. Sci. 2007 362 1857 1871 10.1098/rstb.2006.1812
Brodin, A. Theoretical models of adaptive energy management in small wintering birds. Philos. Trans. R. Soc. B Biol. Sci. 362, 1857–1871 (2007).
4. Krushinskaya N Some complex forms of feeding behaviour of nutcracker Nucifraga caryocatactes, after removal of old cortex Zh Evol Biokhim Fisiol 1966 11 563 568
Krushinskaya, N. Some complex forms of feeding behaviour of nutcracker Nucifraga caryocatactes, after removal of old cortex. Zh Evol Biokhim Fisiol 11, 563–568 (1966).
5. Scoville WB Milner B Loss of recent memory after bilateral hippocampal lesions J. Neurol. Neurosurg. Psychiatry 1957 20 11 10.1136/jnnp.20.1.11 13406589
Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11 (1957).13406589
6. Sullivan KA The advantages of social foraging in downy woodpeckers Anim. Behav. 1984 32 16 22 10.1016/S0003-3472(84)80319-X
Sullivan, K. A. The advantages of social foraging in downy woodpeckers. Anim. Behav. 32, 16–22 (1984).
7. Sullivan KA Information exploitation by downy woodpeckers in mixed-species flocks Behaviour 1984 91 294 311 10.1163/156853984X00128
Sullivan, K. A. Information exploitation by downy woodpeckers in mixed-species flocks. Behaviour 91, 294–311 (1984).
8. Gao P Ganguli S On simplicity and complexity in the brave new world of large-scale neuroscience Curr. Opin. Neurobiol. 2015 32 148 155 10.1016/j.conb.2015.04.003 25932978
Gao, P. & Ganguli, S. On simplicity and complexity in the brave new world of large-scale neuroscience. Curr. Opin. Neurobiol. 32, 148–155 (2015).25932978
9. Krakauer JW Ghazanfar AA Gomez-Marin A MacIver MA Poeppel D Neuroscience needs behavior: Correcting a reductionist bias Neuron 2017 93 480 490 10.1016/j.neuron.2016.12.041 28182904
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: Correcting a reductionist bias. Neuron 93, 480–490 (2017).28182904
10. Mobbs D Trimmer PC Blumstein DT Dayan P Foraging for foundations in decision neuroscience: Insights from ethology Nat. Rev. Neurosci. 2018 19 419 427 10.1038/s41583-018-0010-7 29752468
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. Foraging for foundations in decision neuroscience: Insights from ethology. Nat. Rev. Neurosci. 19, 419–427 (2018).29752468
11. Hall-McMaster S Luyckx F Revisiting foraging approaches in neuroscience Cogn. Affect. Behav. Neurosci. 2019 19 225 230 10.3758/s13415-018-00682-z 30607832
Hall-McMaster, S. & Luyckx, F. Revisiting foraging approaches in neuroscience. Cogn. Affect. Behav. Neurosci. 19, 225–230 (2019).30607832
12. Miller CT Natural behavior is the language of the brain Curr. Biol. 2022 32 R482 R493 10.1016/j.cub.2022.03.031 35609550
Miller, C. T. et al. Natural behavior is the language of the brain. Curr. Biol. 32, R482–R493 (2022).35609550
13. Dennis EJ Systems neuroscience of natural behaviors in rodents J. Neurosci. 2021 41 911 919 10.1523/JNEUROSCI.1877-20.2020 33443081
Dennis, E. J. et al. Systems neuroscience of natural behaviors in rodents. J. Neurosci. 41, 911–919 (2021).33443081
14. Niv Y The primacy of behavioral research for understanding the brain Behav. Neurosci. 2021 135 601 10.1037/bne0000471 34096743
Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci. 135, 601 (2021).34096743
15. Pravosudov VV Cognitive ecology in the wild-advances and challenges in avian cognition research Curr. Opin. Behav. Sci. 2022 45 101138 10.1016/j.cobeha.2022.101138
Pravosudov, V. V. Cognitive ecology in the wild-advances and challenges in avian cognition research. Curr. Opin. Behav. Sci. 45, 101138 (2022).
16. Reynolds, C. W. Flocks, herds and schools: A distributed behavioral model. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, 25–34 (1987).
17. Bialek W Statistical mechanics for natural flocks of birds Proc. Natl. Acad. Sci. 2012 109 4786 4791 10.1073/pnas.1118633109 22427355
Bialek, W. et al. Statistical mechanics for natural flocks of birds. Proc. Natl. Acad. Sci. 109, 4786–4791 (2012).22427355
18. Bod’Ová K Mitchell GJ Harpaz R Schneidman E Tkačik G Probabilistic models of individual and collective animal behavior PLoS ONE 2018 13 e0193049 10.1371/journal.pone.0193049 29513700
Bod’Ová, K., Mitchell, G. J., Harpaz, R., Schneidman, E. & Tkačik, G. Probabilistic models of individual and collective animal behavior. PLoS ONE 13, e0193049 (2018).29513700
19. Coen P Dynamic sensory cues shape song structure in drosophila Nature 2014 507 233 237 10.1038/nature13131 24598544
Coen, P. et al. Dynamic sensory cues shape song structure in drosophila. Nature 507, 233–237 (2014).24598544
20. Edwards AM Revisiting lévy flight search patterns of wandering albatrosses, bumblebees and deer Nature 2007 449 1044 1048 10.1038/nature06199 17960243
Edwards, A. M. et al. Revisiting lévy flight search patterns of wandering albatrosses, bumblebees and deer. Nature 449, 1044–1048 (2007).17960243
21. Mnih, V. et al. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).
22. Constantino SM Daw ND Learning the opportunity cost of time in a patch-foraging task Cogn. Affect. Behav. Neurosci. 2015 15 837 853 10.3758/s13415-015-0350-y 25917000
Constantino, S. M. & Daw, N. D. Learning the opportunity cost of time in a patch-foraging task. Cogn. Affect. Behav. Neurosci. 15, 837–853. 10.3758/s13415-015-0350-y (2015).25917000
23. Wispinski, N. J. et al. Adaptive patch foraging in deep reinforcement learning agents. arXiv preprint arXiv:2210.08085 (2022).
24. Kilpatrick, Z. P., Davidson, J. D. & Hady, A. E. Normative theory of patch foraging decisions. arXiv preprint arXiv:2004.10671 (2020).
25. Karpas ED Shklarsh A Schneidman E Information socialtaxis and efficient collective behavior emerging in groups of information-seeking agents Proc. Natl. Acad. Sci. 2017 114 5589 5594 10.1073/pnas.1618055114 28507154
Karpas, E. D., Shklarsh, A. & Schneidman, E. Information socialtaxis and efficient collective behavior emerging in groups of information-seeking agents. Proc. Natl. Acad. Sci. 114, 5589–5594 (2017).28507154
26. Kaelbling, L. P. Learning to achieve goals. In IJCAI, vol. 2, 1094–8 (Citeseer, 1993).
27. Todorov E Efficient computation of optimal actions Proc. Natl. Acad. Sci. 2009 106 11478 11483 10.1073/pnas.0710743106 19574462
Todorov, E. Efficient computation of optimal actions. Proc. Natl. Acad. Sci. 106, 11478–11483 (2009).19574462
28. Piray P Daw ND Linear reinforcement learning in planning, grid fields, and cognitive control Nat. Commun. 2021 12 4942 10.1038/s41467-021-25123-3 34400622
Piray, P. & Daw, N. D. Linear reinforcement learning in planning, grid fields, and cognitive control. Nat. Commun. 12, 4942 (2021).34400622
29. Bidari S El Hady A Davidson JD Kilpatrick ZP Stochastic dynamics of social patch foraging decisions Phys. Rev. Res. 2022 4 033128 10.1103/PhysRevResearch.4.033128 36090768
Bidari, S., El Hady, A., Davidson, J. D. & Kilpatrick, Z. P. Stochastic dynamics of social patch foraging decisions. Phys. Rev. Res. 4, 033128 (2022).36090768
30. Jara-Ettinger J Theory of mind as inverse reinforcement learning Curr. Opin. Behav. Sci. 2019 29 105 110 10.1016/j.cobeha.2019.04.010
Jara-Ettinger, J. Theory of mind as inverse reinforcement learning. Curr. Opin. Behav. Sci. 29, 105–110 (2019).
31. Berke, M. & Jara-Ettinger, J. Thinking about thinking through inverse reasoning. (2021).
32. Arora S Doshi P A survey of inverse reinforcement learning: Challenges, methods and progress Artif. Intell. 2021 297 103500 10.1016/j.artint.2021.103500
Arora, S. & Doshi, P. A survey of inverse reinforcement learning: Challenges, methods and progress. Artif. Intell. 297, 103500 (2021).
33. Wu Z Kwon M Daptardar S Schrater P Pitkow X Rational thoughts in neural codes Proc. Natl. Acad. Sci. 2020 117 29311 29320 10.1073/pnas.1912336117 33229521
Wu, Z., Kwon, M., Daptardar, S., Schrater, P. & Pitkow, X. Rational thoughts in neural codes. Proc. Natl. Acad. Sci. 117, 29311–29320 (2020).33229521
34. Evans, O., Stuhlmüller, A. & Goodman, N. Learning the preferences of ignorant, inconsistent agents. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016).
35. Zhi-Xuan T Mann J Silver T Tenenbaum J Mansinghka V Online bayesian goal inference for boundedly rational planning agents Adv. Neural. Inf. Process. Syst. 2020 33 19238 19250
Zhi-Xuan, T., Mann, J., Silver, T., Tenenbaum, J. & Mansinghka, V. Online bayesian goal inference for boundedly rational planning agents. Adv. Neural. Inf. Process. Syst. 33, 19238–19250 (2020).
36. López-Cruz A Parallel multimodal circuits control an innate foraging behavior Neuron 2019 102 407 419 10.1016/j.neuron.2019.01.053 30824353
López-Cruz, A. et al. Parallel multimodal circuits control an innate foraging behavior. Neuron 102, 407–419 (2019).30824353
37. Calhoun AJ Hayden BY The foraging brain Curr. Opin. Behav. Sci. 2015 5 24 31 10.1016/j.cobeha.2015.07.003
Calhoun, A. J. & Hayden, B. Y. The foraging brain. Curr. Opin. Behav. Sci. 5, 24–31 (2015).
38. Stachenfeld KL Botvinick MM Gershman SJ The hippocampus as a predictive map Nat. Neurosci. 2017 20 1643 1653 10.1038/nn.4650 28967910
Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. The hippocampus as a predictive map. Nat. Neurosci. 20, 1643–1653 (2017).28967910
39. Barack, D. L. & Platt, M. L. Engaging and exploring: cortical circuits for adaptive foraging decisions. Impulsivity: How Time and Risk Influence Decision Making 163–199 (2017).
40. O’Keefe, J. & Dostrovsky, J. The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat. Brain Res. (1971).
41. Yartsev MM Ulanovsky N Representation of three-dimensional space in the hippocampus of flying bats Science 2013 340 367 372 10.1126/science.1235338 23599496
Yartsev, M. M. & Ulanovsky, N. Representation of three-dimensional space in the hippocampus of flying bats. Science 340, 367–372 (2013).23599496
42. Payne H Lynch G Aronov D Neural representations of space in the hippocampus of a food-caching bird Science 2021 373 343 348 10.1126/science.abg2009 34437154
Payne, H., Lynch, G. & Aronov, D. Neural representations of space in the hippocampus of a food-caching bird. Science 373, 343–348 (2021).34437154
43. Muller RU Kubie JL The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells J. Neurosci. 1987 7 1951 1968 10.1523/JNEUROSCI.07-07-01951.1987 3612226
Muller, R. U. & Kubie, J. L. The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. J. Neurosci. 7, 1951–1968 (1987).3612226
44. Leutgeb S Independent codes for spatial and episodic memory in hippocampal neuronal ensembles Science 2005 309 619 623 10.1126/science.1114037 16040709
Leutgeb, S. et al. Independent codes for spatial and episodic memory in hippocampal neuronal ensembles. Science 309, 619–623 (2005).16040709
45. Sarel A Finkelstein A Las L Ulanovsky N Vectorial representation of spatial goals in the hippocampus of bats Science 2017 355 176 180 10.1126/science.aak9589 28082589
Sarel, A., Finkelstein, A., Las, L. & Ulanovsky, N. Vectorial representation of spatial goals in the hippocampus of bats. Science 355, 176–180 (2017).28082589
46. Gauthier JL Tank DW A dedicated population for reward coding in the hippocampus Neuron 2018 99 179 193 10.1016/j.neuron.2018.06.008 30008297
Gauthier, J. L. & Tank, D. W. A dedicated population for reward coding in the hippocampus. Neuron 99, 179–193 (2018).30008297
47. Chettih SN Mackevicius EL Hale S Aronov D Barcoding of episodic memories in the hippocampus of a food-caching bird Cell 2024 187 1922 1935 10.1016/j.cell.2024.02.032 38554707
Chettih, S. N., Mackevicius, E. L., Hale, S. & Aronov, D. Barcoding of episodic memories in the hippocampus of a food-caching bird. Cell 187, 1922–1935 (2024).38554707
48. Omer DB Maimon SR Las L Ulanovsky N Social place-cells in the bat hippocampus Science 2018 359 218 224 10.1126/science.aao3474 29326274
Omer, D. B., Maimon, S. R., Las, L. & Ulanovsky, N. Social place-cells in the bat hippocampus. Science 359, 218–224 (2018).29326274
49. Fang C Aronov D Abbott L Mackevicius EL Neural learning rules for generating flexible predictions and computing the successor representation eLife 2023 12 e80680 10.7554/eLife.80680 36928104
Fang, C., Aronov, D., Abbott, L. & Mackevicius, E. L. Neural learning rules for generating flexible predictions and computing the successor representation. eLife 12, e80680. 10.7554/eLife.80680 (2023).36928104
50. Bono J Zannone S Pedrosa V Clopath C Learning predictive cognitive maps with spiking neurons during behavior and replays Elife 2023 12 e80671 10.7554/eLife.80671 36927625
Bono, J., Zannone, S., Pedrosa, V. & Clopath, C. Learning predictive cognitive maps with spiking neurons during behavior and replays. Elife 12, e80671 (2023).36927625
51. George TM de Cothi W Stachenfeld KL Barry C Rapid learning of predictive maps with stdp and theta phase precession Elife 2023 12 e80663 10.7554/eLife.80663 36927826
George, T. M., de Cothi, W., Stachenfeld, K. L. & Barry, C. Rapid learning of predictive maps with stdp and theta phase precession. Elife 12, e80663 (2023).36927826
52. Zhang, T., Rosenberg, M., Perona, P. & Meister, M. Endotaxis: A neuromorphic algorithm for mapping, goal-learning, navigation, and patrolling. bioRxiv 2021–09 (2021).
53. Günzel Y Oberhauser FB Couzin-Fuchs E Information integration for decision-making in desert locusts iScience 2023 26 106388 10.1016/j.isci.2023.106388 37034978
Günzel, Y., Oberhauser, F. B. & Couzin-Fuchs, E. Information integration for decision-making in desert locusts. iScience 26, 106388. 10.1016/j.isci.2023.106388 (2023).37034978
54. Meegan DV Winner-takes-all and action selection Behav. Brain Sci. 1999 22 692 693 10.1017/S0140525X99412154
Meegan, D. V. Winner-takes-all and action selection. Behav. Brain Sci. 22, 692–693 (1999).
55. Cook TR Gubiani R Ryan PG Muzaffar SB Group foraging in socotra cormorants: A biologging approach to the study of a complex behavior Ecol. Evol. 2017 7 2025 2038 10.1002/ece3.2750 28405270
Cook, T. R., Gubiani, R., Ryan, P. G. & Muzaffar, S. B. Group foraging in socotra cormorants: A biologging approach to the study of a complex behavior. Ecol. Evol. 7, 2025–2038 (2017).28405270
56. Couzin, I. D. & Heins, C. Emerging technologies for behavioral research in changing environments. Trends Ecol. Evol. (2022).
57. Naik, H. et al. 3d-pop–an automated annotation approach to facilitate markerless 2d-3d tracking of freely moving birds with marker-based motion capture. arXiv preprint arXiv:2303.13174 (2023).
58. Matzner S Warfel T Hull R Thermaltracker-3d: A thermal stereo vision system for quantifying bird and bat activity at offshore wind energy sites Eco. Inform. 2020 57 101069 10.1016/j.ecoinf.2020.101069
Matzner, S., Warfel, T. & Hull, R. Thermaltracker-3d: A thermal stereo vision system for quantifying bird and bat activity at offshore wind energy sites. Eco. Inform. 57, 101069 (2020).
59. Van Horn, G. et al. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 595–604 (2015).
60. Pereira TD Sleap: A deep learning system for multi-animal pose tracking Nat. Methods 2022 19 486 495 10.1038/s41592-022-01426-1 35379947
Pereira, T. D. et al. Sleap: A deep learning system for multi-animal pose tracking. Nat. Methods 19, 486–495 (2022).35379947
61. Linderman, S. et al. Bayesian learning and inference in recurrent switching linear dynamical systems. In Artificial Intelligence and Statistics, 914–922 (PMLR, 2017).
62. Cusumano-Towner, M. F., Radul, A., Wingate, D. & Mansinghka, V. K. Probabilistic programs for inferring the goals of autonomous agents. arXiv preprint arXiv:1704.04977 (2017).
63. Bingham E Pyro: Deep universal probabilistic programming J. Mach. Learn. Res. 2019 20 973 978
Bingham, E. et al. Pyro: Deep universal probabilistic programming. J. Mach. Learn. Res. 20, 973–978 (2019).
64. Das, R., Tenenbaum, J. B., Solar-Lezama, A. & Tavares, Z. Combining functional and automata synthesis to discover causal reactive programs. (2023).
65. Graving JM Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning Elife 2019 8 e47994 10.7554/eLife.47994 31570119
Graving, J. M. et al. Deepposekit, a software toolkit for fast and robust animal pose estimation using deep learning. Elife 8, e47994 (2019).31570119
66. Lauer J Multi-animal pose estimation, identification and tracking with deeplabcut Nat. Methods 2022 19 496 504 10.1038/s41592-022-01443-0 35414125
Lauer, J. et al. Multi-animal pose estimation, identification and tracking with deeplabcut. Nat. Methods 19, 496–504 (2022).35414125
67. Sun, J. J. et al. Bkind-3d: Self-supervised 3d keypoint discovery from multi-view videos. arXiv preprint arXiv:2212.07401 (2022).
68. Ferreira AC Deep learning-based methods for individual recognition in small birds Methods Ecol. Evol. 2020 11 1072 1085 10.1111/2041-210X.13436
Ferreira, A. C. et al. Deep learning-based methods for individual recognition in small birds. Methods Ecol. Evol. 11, 1072–1085 (2020).
69. Momennejad I Learning structures: Predictive representations, replay, and generalization Curr. Opin. Behav. Sci. 2020 32 155 166 10.1016/j.cobeha.2020.02.017 35419465
Momennejad, I. Learning structures: Predictive representations, replay, and generalization. Curr. Opin. Behav. Sci. 32, 155–166. 10.1016/j.cobeha.2020.02.017 (2020).35419465
70. De Cothi W Predictive maps in rats and humans for spatial navigation Curr. Biol. 2022 10.1016/j.cub.2022.06.090 35863351
De Cothi, W. et al. Predictive maps in rats and humans for spatial navigation. Curr. Biol.[SPACE]10.1016/j.cub.2022.06.090 (2022).35863351
71. Radulescu A Niv Y Ballard I Holistic reinforcement learning: The role of structure and attention Trends Cogn. Sci. 2019 23 278 292 10.1016/j.tics.2019.01.010 30824227
Radulescu, A., Niv, Y. & Ballard, I. Holistic reinforcement learning: The role of structure and attention. Trends Cogn. Sci. 23, 278–292. 10.1016/j.tics.2019.01.010 (2019).30824227
72. Hampton RR Shettleworth SJ Hippocampus and memory in a food-storing and in a nonstoring bird species Behav. Neurosci. 1996 110 946 10.1037/0735-7044.110.5.946 8918998
Hampton, R. R. & Shettleworth, S. J. Hippocampus and memory in a food-storing and in a nonstoring bird species. Behav. Neurosci. 110, 946 (1996).8918998
73. Hoshooley JS Sherry DF Greater hippocampal neuronal recruitment in food-storing than in non-food-storing birds Dev. Neurobiol. 2007 67 406 414 10.1002/dneu.20316 17443797
Hoshooley, J. S. & Sherry, D. F. Greater hippocampal neuronal recruitment in food-storing than in non-food-storing birds. Dev. Neurobiol. 67, 406–414 (2007).17443797
