Foraging
This notebook gathers different kinds of environments for foraging and target search in various scenarios, adapted for their use in the reinforcement learning paradigm.
TargetEnv
TargetEnv
TargetEnv (Nt=10, L=1.3, r=1.5, lc=[[1], [1]], agent_step=1, num_agents=1, destructive=False, lc_distribution='constant')
Class defining the a Foraging environment with multiple targets and two actions: continue in the same direction and turn by a random angle.
| Type | Default | Details | |
|---|---|---|---|
| Nt | int | 10 | Number of targets. |
| L | float | 1.3 | Size of the (squared) world. |
| r | float | 1.5 | Radius with center the target position. It defines the area in which agent detects the target. |
| lc | list | [[1], [1]] | Cutoff length. Displacement away from target |
| agent_step | int | 1 | Displacement of one step. The default is 1. |
| num_agents | int | 1 | Number of agents that forage at the same time. The default is 1. > 1 not fully implemented |
| destructive | bool | False | True if targets are destructive. The default is False. |
| lc_distribution | str | constant | Distribution from where to sample l_c. Can be ‘power_law’, ‘pareto’ or something else. See comments self.check_encounter for explanations |
ResetEnv
1D
ResetEnv_1D
ResetEnv_1D (L=1.3, D=1.0)
Class defining a 1D environment where the agent can either continue walking or reset to the origin. Reward = 1 if the agent crosses the distance L, else = 0.
| Type | Default | Details | |
|---|---|---|---|
| L | float | 1.3 | Distance to cross to get reward |
| D | float | 1.0 | Diffusion coefficient |
reset_search_loop
reset_search_loop (T, reset_policy, env)
Loop that runs the reset environment with a given reset policy.
| Details | |
|---|---|
| T | Number of steps |
| reset_policy | Reset policy |
| env | Environment |
parallel_Reset1D_exp
parallel_Reset1D_exp (T, rates, L, D)
Runs the Reset 1D loop in parallel for different exponential resetting rates.
parallel_Reset1D_sharp
parallel_Reset1D_sharp (T:int, resets:<built-infunctionarray>, L:float, D:float)
Runs the Reset 1D loop in parallel for different sharp resetting times.
| Type | Details | |
|---|---|---|
| T | int | Number of steps |
| resets | array | Array of resetting times |
| L | float | Distance to cross to get reward |
| D | float | Diffusion coefficient |
| Returns | array | Rewards obtained for each resetting time |
2D
ResetEnv_2D
ResetEnv_2D (dist_target=0.2, radius_target=0.5, D=1.0)
Class defining a 2D environment where the agent can either continue walking or reset to the origin. Reward = 1 if the agent enters the area defined by the distance and radius of the target.
| Type | Default | Details | |
|---|---|---|---|
| dist_target | float | 0.2 | Distance from init position and target |
| radius_target | float | 0.5 | Radius of the target |
| D | float | 1.0 | Diffusion coefficient of the walker |
parallel_Reset2D_policies
parallel_Reset2D_policies (T, reset_policies, dist_target, radius_target, D)
Runs the Reset 2D loop in parallel for different sharp resetting times.
| Type | Details | |
|---|---|---|
| T | Number of steps | |
| reset_policies | Array of reset policies | |
| dist_target | Distance of the target | |
| radius_target | Radius of the target | |
| D | Diffusion coefficient of the walker | |
| Returns | array | Rewards obtained for each reset policy |
parallel_Reset2D_exp
parallel_Reset2D_exp (T, rates, dist_target, radius_target, D)
Runs the Reset 2D loop in parallel for different sharp resetting times.
| Type | Details | |
|---|---|---|
| T | Number of steps | |
| rates | Reset rates | |
| dist_target | Distance of the target | |
| radius_target | Radius of the target | |
| D | Diffusion coefficient of the walker | |
| Returns | array | Rewards obtained for each exponential rate |
parallel_Reset2D_sharp
parallel_Reset2D_sharp (T, resets, dist_target, radius_target, D)
Runs the Reset 2D loop in parallel for different sharp resetting times.
| Type | Details | |
|---|---|---|
| T | Number of steps | |
| resets | Array of resetting times | |
| dist_target | Distance to target | |
| radius_target | Radius of the target | |
| D | Diffusion coefficient of the walker | |
| Returns | array | Rewards obtained for each sharp resetting time |
Turn + Reset Environment
Only 2D is considered here as the 1D case is trivial (the agent would never turn, only reset)
TurnResetEnv_2D
TurnResetEnv_2D (dist_target=0.2, radius_target=0.5, agent_step=1.0)
*Class defining a Foraging environment with a single target and three possible actions:
- Continue in the same direction
- Turn by a random angle
- Reset to the origin
The agent makes steps of constant length given by agent_step.*
| Type | Default | Details | |
|---|---|---|---|
| dist_target | float | 0.2 | Distance from init position and target |
| radius_target | float | 0.5 | Radius of the target |
| agent_step | float | 1.0 | Diffusion coefficient of the walker |
search_loop_turn_reset_sharp
search_loop_turn_reset_sharp (T, reset, turn, env)
*Runs a search loop of T steps. There is a single counter that works as follows:
- Starts at 0
- For each turn or continue action gets +1
- If reset or reach the target is set to 0*
| Details | |
|---|---|
| T | Number of steps |
| reset | Reset time |
| turn | Turn time |
| env | Environment |