Signal8 1.7__tar.gz → 1.9__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of Signal8 might be problematic. Click here for more details.

@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: Signal8
3
- Version: 1.7
3
+ Version: 1.9
4
4
  Summary: A multi-agent environment inspired by the Lewis Signaling Game, featuring eight unique problem configurations with both static and dynamic obstacles.
5
5
  Home-page: https://github.com/ethanmclark1/signal8
6
6
  Author: Ethan Clark
@@ -39,15 +39,15 @@ env.reset(options={"problem_name": "v_cluster"}))
39
39
  ## List of Problem Scenarios
40
40
 
41
41
  | Names | Description |
42
- | :------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43
- | `Cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
44
- | `L-Shaped` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
45
- | `Vertical` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
46
- | `Horizontal` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
47
- | `Top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
48
- | `Bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
49
- | `Right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
50
- | `Left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
42
+ | :-----------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43
+ | `v_cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
44
+ | ``h_cluster`` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
45
+ | `v_wall` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
46
+ | `h_wall` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
47
+ | `top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
48
+ | `bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
49
+ | `right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
50
+ | `left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
51
51
 
52
52
  ## Paper Citation
53
53
 
@@ -27,15 +27,15 @@ env.reset(options={"problem_name": "v_cluster"}))
27
27
  ## List of Problem Scenarios
28
28
 
29
29
  | Names | Description |
30
- | :------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
31
- | `Cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
32
- | `L-Shaped` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
33
- | `Vertical` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
34
- | `Horizontal` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
35
- | `Top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
36
- | `Bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
37
- | `Right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
38
- | `Left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
30
+ | :-----------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
31
+ | `v_cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
32
+ | ``h_cluster`` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
33
+ | `v_wall` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
34
+ | `h_wall` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
35
+ | `top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
36
+ | `bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
37
+ | `right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
38
+ | `left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
39
39
 
40
40
  ## Paper Citation
41
41
 
@@ -0,0 +1,225 @@
1
+ import time
2
+ import copy
3
+ import random
4
+ import logging
5
+ import threading
6
+ import numpy as np
7
+
8
+ from gymnasium.utils import EzPickle
9
+ from utils.scenario import BaseScenario
10
+ from utils.core import Agent, Goal, Obstacle, World
11
+ from utils.simple_env import SimpleEnv, make_env
12
+ from utils.problems import get_problem, get_problem_list
13
+
14
+ class raw_env(SimpleEnv, EzPickle):
15
+ def __init__(
16
+ self,
17
+ num_agents=1,
18
+ render_mode="human"):
19
+
20
+ scenario = Scenario()
21
+ world = scenario.make_world(num_agents)
22
+
23
+ super().__init__(
24
+ scenario=scenario,
25
+ world=world,
26
+ render_mode=render_mode,
27
+ max_cycles=500,
28
+ continuous_actions=False,
29
+ )
30
+
31
+ env = make_env(raw_env)
32
+
33
+ class Scenario(BaseScenario):
34
+ def make_world(self, num_agents):
35
+ world = World()
36
+ self._add_logger()
37
+ world.problem_scenarios = get_problem_list()
38
+
39
+ self.obstacle_locks = []
40
+ self.scripted_obstacle_threads = []
41
+ self.scripted_obstacle_running = False
42
+
43
+ world.agents = [Agent() for _ in range(num_agents)]
44
+ for i, agent in enumerate(world.agents):
45
+ agent.name = f"agent_{i}"
46
+ agent.collide = True
47
+
48
+ world.goals = [Goal() for _ in range(len(world.agents))]
49
+ for i, goal in enumerate(world.goals):
50
+ goal.name = f"goal_{i}"
51
+ goal.collide = False
52
+
53
+ # Minimum number of obstacles in a problem scenario
54
+ world.obstacles = [Obstacle() for _ in range(4)]
55
+ for i, obstacle in enumerate(world.obstacles):
56
+ obstacle.name = f"obs_{i}"
57
+
58
+ return world
59
+
60
+ # Get constraints on entities given the problem name
61
+ def _set_problem_scenario(self, world, np_random, problem_name):
62
+ if problem_name is None:
63
+ problem_name = np_random.choice(world.problem_scenarios)
64
+ problem = get_problem(problem_name)
65
+ world.problem_name = problem_name
66
+ world.start_constr = problem['start']
67
+ world.goal_constr = problem['goal']
68
+ world.static_obstacle_constr = problem['static_obs']
69
+ world.dynamic_obstacle_constr = problem['dynamic_obs']
70
+
71
+ """
72
+ Returns goal constraints that haven't been selected to be used as an obstacle
73
+ for the precision farming case (i.e., crop that wasn't selected as goal becomes an obstacle)
74
+ """
75
+ def _reset_agents_and_goals(self, world, np_random):
76
+ temp_start_constr = list(copy.copy(world.start_constr))
77
+ temp_goal_constr = list(copy.copy(world.goal_constr))
78
+ for i, agent in enumerate(world.agents):
79
+ agent.color = np.array([0, 0.8, 0])
80
+ agent.state.p_vel = np.zeros(world.dim_p)
81
+ agent_constr = random.choice(temp_start_constr)
82
+ agent.state.p_pos = np_random.uniform(*zip(*agent_constr))
83
+ agent.start_pos = copy.copy(agent.state.p_pos)
84
+ temp_start_constr.remove(agent_constr)
85
+
86
+ agent.goal = world.goals[i]
87
+ agent.goal.color = np.array([0, 0, 0.8])
88
+ agent.goal.state.p_vel = np.zeros(world.dim_p)
89
+ goal_constr = random.choice(temp_goal_constr)
90
+ agent.goal.state.p_pos = np_random.uniform(*zip(*goal_constr))
91
+ temp_goal_constr.remove(goal_constr)
92
+
93
+ return temp_goal_constr
94
+
95
+ # Reset position of dynamic obstacles
96
+ def _reset_dynamic_obstacle(self, world, obstacle, np_random, temp_dynamic_obs_constr):
97
+ obstacle.size = 0.025
98
+ obstacle.movable = True
99
+ self.obstacle_locks += [threading.Lock()]
100
+ obstacle.color = np.array([0.5, 0, 0])
101
+ obstacle.state.p_vel = np.zeros(world.dim_p)
102
+ dynamic_obs_constr = random.choice(temp_dynamic_obs_constr)
103
+ obstacle.state.p_pos = np_random.uniform(*zip(*dynamic_obs_constr))
104
+ obstacle.action_callback = self.get_scripted_action
105
+ temp_dynamic_obs_constr.remove(dynamic_obs_constr)
106
+
107
+ # Reset position of static obstacles, taking leftover entities from goal constraints
108
+ def _reset_static_obstacle(self, world, obstacle, np_random, temp_static_obs_constr):
109
+ obstacle.color = np.array([0.2, 0.2, 0.2])
110
+ obstacle.state.p_vel = np.zeros(world.dim_p)
111
+ static_obs_constr = random.choice(temp_static_obs_constr)
112
+ obstacle.state.p_pos = np_random.uniform(*zip(*static_obs_constr))
113
+ temp_static_obs_constr.remove(static_obs_constr)
114
+
115
+ # Add or remove obstacles to match the number of obstacles in problem scenario
116
+ def _match_obstacles_to_problem(self, world, num_static):
117
+ num_total_obstacles = num_static + len(world.dynamic_obstacle_constr)
118
+ if len(world.obstacles) > num_total_obstacles:
119
+ world.obstacles = world.obstacles[:num_total_obstacles]
120
+ elif len(world.obstacles) < num_total_obstacles:
121
+ additional_obstacles = [Obstacle() for _ in range(len(world.obstacles), num_total_obstacles)]
122
+ [setattr(obstacle, 'name', f"obs_{i+len(world.obstacles)}") for i, obstacle in enumerate(additional_obstacles)]
123
+ world.obstacles.extend(additional_obstacles)
124
+
125
+ # Reset position of obstacles
126
+ def _reset_obstacles(self, world, np_random, leftover_entities):
127
+ temp_static_obs_constr = list(copy.copy(world.static_obstacle_constr))
128
+ temp_static_obs_constr += leftover_entities
129
+ temp_dynamic_obs_constr = list(copy.copy(world.dynamic_obstacle_constr))
130
+
131
+ num_dynamic_obs = len(temp_dynamic_obs_constr)
132
+ self._match_obstacles_to_problem(world, len(temp_static_obs_constr))
133
+
134
+ for i, obstacle in enumerate(world.obstacles):
135
+ if i < num_dynamic_obs:
136
+ self._reset_dynamic_obstacle(world, obstacle, np_random, temp_dynamic_obs_constr)
137
+ else:
138
+ self._reset_static_obstacle(world, obstacle, np_random, temp_static_obs_constr)
139
+
140
+ # Start a thread for each dynamic obstacle
141
+ def _start_scripted_obstacles(self, world):
142
+ idx = 0
143
+ self.scripted_obstacle_running = True
144
+ for obstacle in world.obstacles:
145
+ if obstacle.movable:
146
+ t = threading.Thread(target=self.run_scripted_obstacle, args=(world, obstacle, idx))
147
+ t.start()
148
+ self.scripted_obstacle_threads.append(t)
149
+ idx += 1
150
+
151
+ # Reset entities in world
152
+ def reset_world(self, world, np_random, problem_name=None):
153
+ self._stop_scripted_obstacles()
154
+ self._set_problem_scenario(world, np_random, problem_name)
155
+ leftover_entities = self._reset_agents_and_goals(world, np_random)
156
+ self._reset_obstacles(world, np_random, leftover_entities)
157
+ self._start_scripted_obstacles(world)
158
+
159
+ # Do not need to implement this function
160
+ def reward(self, agent, world):
161
+ return 0
162
+
163
+ def observation(self, agent, world):
164
+ return np.concatenate((agent.state.p_pos, agent.state.p_vel))
165
+
166
+ # TODO: Implement behavior for scripted obstacles
167
+ """
168
+ Disaster Response: Increase size of obstacle to resemble increasing size of fire/flood
169
+ Precision Farming: Move obstacle in a zamboni pattern to resemble the tractor
170
+ """
171
+ def get_scripted_action(self, obs, world):
172
+ action = np.zeros(world.dim_p)
173
+
174
+ problem_name = world.problem_name
175
+ if problem_name == 'disaster_response_0':
176
+ obs.size *= 1.125
177
+ elif problem_name == 'disaster_response_1':
178
+ obs.size *= 1.100
179
+ elif problem_name == 'disaster_response_2':
180
+ obs.size *= 1.150
181
+ elif problem_name == 'disaster_response_3':
182
+ obs.size *= 1.050
183
+ elif problem_name == 'precision_farming_0':
184
+ action[0] = +1.0
185
+ elif problem_name == 'precision_farming_1':
186
+ action[0] = -1.0
187
+ elif problem_name == 'precision_farming_2':
188
+ action[0] = +1.0
189
+ elif problem_name == 'precision_farming_3':
190
+ action[0] = -1.0
191
+
192
+ return action
193
+
194
+ # Run a thread for each scripted obstacle
195
+ def run_scripted_obstacle(self, world, obstacle, obstacle_idx):
196
+ sensitivity = 5.0
197
+ while self.scripted_obstacle_running:
198
+ with self.obstacle_locks[obstacle_idx]:
199
+ self.logger.debug(f'{obstacle.name} started at size: {obstacle.size}, position: {obstacle.state.p_pos}')
200
+ action = self.get_scripted_action(obstacle, world)
201
+ obstacle.action = action * sensitivity
202
+ obstacle.move()
203
+ self.logger.debug(f'{obstacle.name} is now: {obstacle.size}, position: {obstacle.state.p_pos}')
204
+ time.sleep(0.1)
205
+
206
+ # Stop all threads for scripted obstacles
207
+ def _stop_scripted_obstacles(self):
208
+ self.scripted_obstacle_running = False
209
+ for t in self.scripted_obstacle_threads:
210
+ t.join()
211
+ self.scripted_obstacle_threads.clear()
212
+ self.obstacle_locks.clear()
213
+
214
+ # Create a logger to log information from threads
215
+ def _add_logger(self):
216
+ self.logger = logging.getLogger('Dynamic Obstacles')
217
+ self.logger.setLevel(logging.DEBUG)
218
+
219
+ ch = logging.StreamHandler()
220
+ ch.setLevel(logging.DEBUG)
221
+
222
+ formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
223
+ ch.setFormatter(formatter)
224
+
225
+ self.logger.addHandler(ch)
@@ -0,0 +1,7 @@
1
+ import Signal8
2
+
3
+ env = Signal8.env()
4
+ env.reset(options={"problem_name": "disaster_response_0"})
5
+ env.step(1)
6
+ observation, reward, termination, truncation, info = env.last()
7
+ env.unwrapped.scenario.stop_scripted_obstacles()
@@ -0,0 +1,169 @@
1
+ import numpy as np
2
+
3
+
4
+ class EntityState: # physical/external base state of all entities
5
+ def __init__(self):
6
+ # physical position
7
+ self.p_pos = None
8
+ # physical velocity
9
+ self.p_vel = None
10
+
11
+
12
+ class AgentState(EntityState): # state of agents
13
+ def __init__(self):
14
+ super().__init__()
15
+
16
+
17
+ class Entity: # properties and state of physical world entity
18
+ def __init__(self):
19
+ # name
20
+ self.name = ""
21
+ # properties:
22
+ self.size = 0.1
23
+ # entity can move / be pushed
24
+ self.movable = False
25
+ # entity collides with others
26
+ self.collide = True
27
+ # color
28
+ self.color = None
29
+ # state
30
+ self.state = EntityState()
31
+ # mass
32
+ self.initial_mass = 1.0
33
+
34
+ @property
35
+ def mass(self):
36
+ return self.initial_mass
37
+
38
+
39
+ class Goal(Entity): # properties of goal entities
40
+ def __init__(self):
41
+ super().__init__()
42
+
43
+
44
+ class Obstacle(Entity): # properties of obstacles entities
45
+ def __init__(self):
46
+ super().__init__()
47
+ # entity can be moved / pushed
48
+ self.movable = False
49
+ # action
50
+ self.action = None
51
+ # script behavior to execute
52
+ self.action_callback = None
53
+
54
+ # updates the state of the obstacle
55
+ def move(self, dt=0.1, damping=0.25):
56
+ self.state.p_vel = self.state.p_vel * (1 - damping)
57
+ self.state.p_vel += (self.action / self.mass) * dt
58
+ self.state.p_pos += self.state.p_vel * dt
59
+
60
+
61
+ class Agent(Entity): # properties of agent entities
62
+ def __init__(self):
63
+ super().__init__()
64
+ # agents are movable by default
65
+ self.movable = True
66
+ # reached goal state
67
+ self.reached_goal = False
68
+ # reached start state after reaching goal
69
+ self.returned_to_safety = False
70
+ # starting physical position
71
+ self.start_pos = None
72
+ # state
73
+ self.state = AgentState()
74
+ # action
75
+ self.action = None
76
+
77
+ class World: # multi-agent world
78
+ def __init__(self):
79
+ # list of agents and entities (can change at execution-time!)
80
+ self.agents = []
81
+ self.goals = []
82
+ self.obstacles = []
83
+ # position dimensionality
84
+ self.dim_p = 2
85
+ # color dimensionality
86
+ self.dim_color = 3
87
+ # simulation timestep
88
+ self.dt = 0.1
89
+ # physical damping
90
+ self.damping = 0.25
91
+ # contact response parameters
92
+ self.contact_force = 1e2
93
+ self.contact_margin = 1e-3
94
+ # problem scenarios
95
+ self.problem_scenarios = []
96
+ self.problem_name = None
97
+ self.start_constr = None
98
+ self.goal_constr = None
99
+ self.static_obstacle_constr = None
100
+ self.dynamic_obstacle_constr = None
101
+
102
+ # return all entities in the world
103
+ @property
104
+ def entities(self):
105
+ return self.agents + self.goals + self.obstacles
106
+
107
+ # update state of the world
108
+ def step(self):
109
+ # gather forces applied to entities
110
+ p_force = [None] * len(self.entities)
111
+ # apply agent physical controls
112
+ p_force = self.apply_action_force(p_force)
113
+ # apply environment forces
114
+ p_force = self.apply_environment_force(p_force)
115
+ # integrate physical state
116
+ self.integrate_state(p_force)
117
+
118
+ # gather agent action forces
119
+ def apply_action_force(self, p_force):
120
+ # set applied forces
121
+ for i, agent in enumerate(self.agents):
122
+ if agent.movable:
123
+ p_force[i] = agent.action
124
+ return p_force
125
+
126
+ # gather physical forces acting on agents
127
+ def apply_environment_force(self, p_force):
128
+ # simple (but inefficient) collision response
129
+ for a, agent_a in enumerate(self.agents):
130
+ for b, agent_b in enumerate(self.agents):
131
+ if b <= a:
132
+ continue
133
+ [f_a, f_b] = self.get_collision_force(agent_a, agent_b)
134
+ if f_a is not None:
135
+ if p_force[a] is None:
136
+ p_force[a] = 0.0
137
+ p_force[a] = f_a + p_force[a]
138
+ if f_b is not None:
139
+ if p_force[b] is None:
140
+ p_force[b] = 0.0
141
+ p_force[b] = f_b + p_force[b]
142
+ return p_force
143
+
144
+ # integrate physical state
145
+ def integrate_state(self, p_force):
146
+ for i, agent in enumerate(self.agents):
147
+ agent.state.p_vel = agent.state.p_vel * (1 - self.damping)
148
+ if p_force[i] is not None:
149
+ agent.state.p_vel += (p_force[i] / agent.mass) * self.dt
150
+ agent.state.p_pos += agent.state.p_vel * self.dt
151
+
152
+ # get collision forces for any contact between two agents
153
+ def get_collision_force(self, agent_a, agent_b):
154
+ if (not agent_a.collide) or (not agent_b.collide):
155
+ return [None, None] # not a collider
156
+ if agent_a is agent_b:
157
+ return [None, None] # don't collide against itself
158
+ # compute actual distance between entities
159
+ delta_pos = agent_a.state.p_pos - agent_b.state.p_pos
160
+ dist = np.sqrt(np.sum(np.square(delta_pos)))
161
+ # minimum allowable distance
162
+ dist_min = agent_a.size + agent_b.size
163
+ # softmax penetration
164
+ k = self.contact_margin
165
+ penetration = np.logaddexp(0, -(dist - dist_min) / k) * k
166
+ force = self.contact_force * delta_pos / dist * penetration
167
+ force_a = +force if agent_a.movable else None
168
+ force_b = -force if agent_b.movable else None
169
+ return [force_a, force_b]
@@ -0,0 +1,114 @@
1
+ problems = {
2
+ 'disaster_response_0': {
3
+ 'start': (((-0.05, 0.05), (-1, -0.90)),),
4
+ 'goal': (((-0.85, -0.75), (0.90, 1)),),
5
+ 'static_obs': (
6
+ ((-1, -0.3), (0.30, 0.50)),
7
+ ((0.15, 0.40), (0.60, 1)),
8
+ ((-0.85, -0.75), (0.05, 0.15))
9
+ ),
10
+ 'dynamic_obs': (((0.95, 1), (0.95, 1)),)
11
+ },
12
+ 'disaster_response_1': {
13
+ 'start': (((-0.05, 0.05), (-1, -0.90)),),
14
+ 'goal': (((0.90, 1), (-0.05, 0.05)),),
15
+ 'static_obs': (
16
+ ((-0.25, 0), (-0.3, 0.65)),
17
+ ((0.25, 0.45), (0.5, 1)),
18
+ ((0.75, 0.85), (-0.75, -0.30))
19
+ ),
20
+ 'dynamic_obs': (((-0.6, -0.45), (0.95, 1)),)
21
+ },
22
+ 'disaster_response_2': {
23
+ 'start': (((-0.05, 0.05), (-1, -0.90)),),
24
+ 'goal': (((-0.05, 0.05), (0, 0.10)),),
25
+ 'static_obs': (
26
+ ((-1, -0.25), (-0.75, -0.55)),
27
+ ((0.25, 0.65), (-0.85, -0.15)),
28
+ ((0, 0.20), (0.75, 1))
29
+ ),
30
+ 'dynamic_obs': (((-0.15, 0), (0.90, 1)),)
31
+ },
32
+ 'disaster_response_3': {
33
+ 'start': (((-0.05, 0.05), (-1, -0.90)),),
34
+ 'goal': (((0.90, 1), (0.90, 1)),),
35
+ 'static_obs': (
36
+ ((-0.025, 0.125), (0.5, 1)),
37
+ ((-1, -0.35), (0.5, 0.75)),
38
+ ((0.1, 1), (-0.70, -0.35))
39
+ ),
40
+ 'dynamic_obs': (((-1, -0.70), (0.80, 0.875)),)
41
+ },
42
+ 'precision_farming_0': {
43
+ 'start': (((0.80, 1), (0.80, 1)),),
44
+ 'goal': (((-0.95, -0.70), (0.525, 0.90)),
45
+ ((-0.625, -0.475), (0.525, 0.90)),
46
+ ((-0.95, -0.70), (-0.15, 0.225)),
47
+ ((-0.625, -0.475), (-0.15, 0.225)),
48
+ ((-0.95, -0.70), (-0.825, -0.45)),
49
+ ((-0.625, -0.475), (-0.825, -0.45)),
50
+ ),
51
+ 'static_obs': (
52
+ ((-0.85, -0.5), (0.275, 0.475)),
53
+ ((-0.85, -0.5), (-0.40, -0.20)),
54
+ ((0.30, 0.50), (-0.90, -0.65)),
55
+ ),
56
+ 'dynamic_obs': (((0.80, 1), (-1, -0.80)),)
57
+ },
58
+ 'precision_farming_1': {
59
+ 'start': (((-1, -0.80), (-1, -0.80)),),
60
+ 'goal': (((-0.95, -0.575), (0.65, 0.90)),
61
+ ((-0.95, -0.575), (0.20, 0.45)),
62
+ ((-0.275, 0.10), (0.65, 0.90)),
63
+ ((-0.275, 0.10), (0.20, 0.45)),
64
+ ((0.40, 0.775), (0.65, 0.90)),
65
+ ((0.40, 0.775), (0.20, 0.45)),
66
+ ),
67
+ 'static_obs': (
68
+ ((-0.525, -0.325), (0.20, 0.90)),
69
+ ((0.15, 0.35), (0.20, 0.90)),
70
+ ((-1, -0.825), (-0.15, 0.15)),
71
+ ),
72
+ 'dynamic_obs': (((0.80, 1), (-1, -0.80)),)
73
+ },
74
+ 'precision_farming_2': {
75
+ 'start': (((-1, -0.80), (-0.20, 0)),),
76
+ 'goal': (((-0.1875, 0.1875), (0.65, 0.90)),
77
+ ((-0.1875, 0.1875), (0.325, 0.575)),
78
+ ((-0.1875, 0.1875), (0, 0.25)),
79
+ ((-0.1875, 0.1875), (-0.325, -0.075)),
80
+ ((-0.1875, 0.1875), (-0.65, -0.40)),
81
+ ((-0.1875, 0.1875), (-0.975, -0.725)),
82
+ ),
83
+ 'static_obs': (
84
+ ((-0.45, -0.25), (0.125, 0.85)),
85
+ ((-0.45, -0.25), (-0.85, -0.125)),
86
+ ((0.25, 0.45), (0.125, 0.85)),
87
+ ((0.25, 0.45), (-0.85, -0.125)),
88
+ ((-0.56875, -0.21875), (0.875, 1)),
89
+ ),
90
+ 'dynamic_obs': (((-1, -0.80), (0, 0.20)),)
91
+ },
92
+ 'precision_farming_3': {
93
+ 'start': (((-0.10, 0.10), (0.80, 1)),),
94
+ 'goal': (((-0.80, -0.425), (-0.20, 0)),
95
+ ((-0.1875, 0.1875), (-0.20, 0)),
96
+ ((0.425, 0.80), (-0.20, 0)),
97
+ ((-0.80, -0.425), (-0.925, -0.725)),
98
+ ((-0.1875, 0.1875), (-0.925, -0.725)),
99
+ ((0.425, 0.80), (-0.925, -0.725)),
100
+ ),
101
+ 'static_obs': (
102
+ ((-0.85, 0.85), (-0.65, -0.275)),
103
+ ((0.10, 0.30), (0.80, 1)),
104
+ ),
105
+ 'dynamic_obs': (((-0.30, -0.10), (0.80, 1)),)
106
+ },
107
+ }
108
+
109
+ def get_problem_list():
110
+ return list(problems.keys())
111
+
112
+ def get_problem(scenario_name):
113
+ problem = problems[scenario_name]
114
+ return problem
@@ -63,9 +63,7 @@ class SimpleEnv(AECEnv):
63
63
  self.scenario.reset_world(self.world, self.np_random)
64
64
  self.agents = [agent.name for agent in self.world.agents]
65
65
  self.possible_agents = self.agents[:]
66
- self._index_map = {
67
- agent.name: idx for idx, agent in enumerate(self.world.agents)
68
- }
66
+ self._index_map = {agent.name: idx for idx, agent in enumerate(self.world.agents)}
69
67
 
70
68
  self._agent_selector = agent_selector(self.agents)
71
69
 
@@ -80,11 +78,6 @@ class SimpleEnv(AECEnv):
80
78
  space_dim = 0
81
79
  else:
82
80
  space_dim = 1
83
- if not agent.silent:
84
- if self.continuous_actions:
85
- space_dim += self.world.dim_c
86
- else:
87
- space_dim *= self.world.dim_c
88
81
 
89
82
  obs_dim = len(self.scenario.observation(agent, self.world))
90
83
  state_dim += obs_dim
@@ -135,10 +128,11 @@ class SimpleEnv(AECEnv):
135
128
  )
136
129
  return np.concatenate(states, axis=None)
137
130
 
138
- def reset(self, seed=None, return_info=False, options=None):
131
+ def reset(self, seed=None, return_info=False, options=None):
139
132
  if seed is not None:
140
133
  self.seed(seed=seed)
141
- problem_scenario = options['problem_name'] if options is not None else 'v_cluster'
134
+
135
+ problem_scenario = options['problem_name']
142
136
  self.scenario.reset_world(self.world, self.np_random, problem_scenario)
143
137
 
144
138
  self.agents = self.possible_agents[:]
@@ -151,11 +145,11 @@ class SimpleEnv(AECEnv):
151
145
  self.agent_selection = self._agent_selector.reset()
152
146
  self.steps = 0
153
147
 
154
- self.current_actions = [None] * len(self.world.policy_agents)
148
+ self.current_actions = [None] * len(self.world.agents)
155
149
 
156
150
  def _execute_world_step(self):
157
151
  # set action for each agent
158
- for i, agent in enumerate(self.world.policy_agents):
152
+ for i, agent in enumerate(self.world.agents):
159
153
  action = self.current_actions[i]
160
154
  scenario_action = []
161
155
  if agent.movable:
@@ -166,10 +160,7 @@ class SimpleEnv(AECEnv):
166
160
  else:
167
161
  scenario_action.append(action % mdim)
168
162
  action //= mdim
169
- if not agent.silent:
170
- scenario_action.append(action)
171
- self._set_action(scenario_action, agent,
172
- self.action_spaces[agent.name])
163
+ self._set_action(scenario_action, agent, self.action_spaces[agent.name])
173
164
 
174
165
  self.world.step()
175
166
 
@@ -191,41 +182,69 @@ class SimpleEnv(AECEnv):
191
182
 
192
183
  # set env action for a particular agent
193
184
  def _set_action(self, action, agent, action_space, time=None):
194
- agent.action.u = np.zeros(self.world.dim_p)
195
- agent.action.c = np.zeros(self.world.dim_c)
196
-
197
185
  if agent.movable:
198
186
  # physical action
199
- agent.action.u = np.zeros(self.world.dim_p)
187
+ agent.action = np.zeros(self.world.dim_p)
200
188
  if self.continuous_actions:
201
189
  # Process continuous action as in OpenAI MPE
202
- agent.action.u[0] += action[0][1] - action[0][2]
203
- agent.action.u[1] += action[0][3] - action[0][4]
190
+ agent.action[0] += action[0][1] - action[0][2]
191
+ agent.action[1] += action[0][3] - action[0][4]
204
192
  else:
205
193
  # process discrete action
206
194
  if action[0] == 1:
207
- agent.action.u[0] = -1.0
208
- if action[0] == 2:
209
- agent.action.u[0] = +1.0
210
- if action[0] == 3:
211
- agent.action.u[1] = -1.0
212
- if action[0] == 4:
213
- agent.action.u[1] = +1.0
195
+ agent.action[0] = -1.0
196
+ elif action[0] == 2:
197
+ agent.action[0] = +1.0
198
+ elif action[0] == 3:
199
+ agent.action[1] = -1.0
200
+ elif action[0] == 4:
201
+ agent.action[1] = +1.0
214
202
  sensitivity = 5.0
215
- if agent.accel is not None:
216
- sensitivity = agent.accel
217
- agent.action.u *= sensitivity
218
- action = action[1:]
219
- if not agent.silent:
220
- # communication action
221
- if self.continuous_actions:
222
- agent.action.c = action[0]
223
- else:
224
- agent.action.c = np.zeros(self.world.dim_c)
225
- agent.action.c[action[0]] = 1.0
203
+ agent.action *= sensitivity
226
204
  action = action[1:]
227
205
  # make sure we used all elements of action
228
206
  assert len(action) == 0
207
+
208
+ # Check if episode is terminated or truncated
209
+ def _episode_status(self):
210
+ dynamic_obs = [obs for obs in self.world.obstacles if obs.movable]
211
+ static_obs = [obs for obs in self.world.obstacles if not obs.movable]
212
+
213
+ goal_dist_threshold = self.world.agents[0].size + self.world.agents[0].goal.size
214
+ static_obs_threshold = self.world.agents[0].size + static_obs[0].size
215
+
216
+ goal_dist = [np.linalg.norm(agent.state.p_pos - agent.goal.state.p_pos) for agent in self.world.agents]
217
+ static_obs_dist = [min(np.linalg.norm(agent.state.p_pos - obs.state.p_pos) for obs in static_obs)
218
+ for agent in self.world.agents]
219
+
220
+ crossed_threshold_static = [dist <= static_obs_threshold for dist in static_obs_dist]
221
+
222
+ try:
223
+ for i in range(len(dynamic_obs)):
224
+ self.scenario.obstacle_locks[i].acquire()
225
+ dynamic_obs_dist = [min(np.linalg.norm(agent.state.p_pos - obs.state.p_pos) for obs in dynamic_obs)
226
+ for agent in self.world.agents]
227
+ dynamic_obs_threshold = [agent.size + obs.size for agent, obs in zip(self.world.agents, dynamic_obs)]
228
+
229
+ crossed_threshold_dynamic = [dist <= threshold for dist, threshold in zip(dynamic_obs_dist, dynamic_obs_threshold)]
230
+
231
+ truncations = [crossed_stat or crossed_dyn for crossed_stat, crossed_dyn in zip(crossed_threshold_static, crossed_threshold_dynamic)]
232
+ truncations = [True] * self.num_agents if self.steps >= self.max_cycles else truncations
233
+
234
+ terminations = [False] * self.num_agents
235
+ for i, dist in enumerate(goal_dist):
236
+ if dist <= goal_dist_threshold:
237
+ self.agents[i].reached_goal = True
238
+
239
+ for i, agent in enumerate(self.world.agents):
240
+ if agent.reached_goal:
241
+ dist_to_start = np.linalg.norm(agent.state.p_pos - agent.start_pos)
242
+ start_dist_threshold = self.world.agents[0].size * 2
243
+ if dist_to_start <= start_dist_threshold:
244
+ terminations[i] = True
245
+ finally:
246
+ [self.scenario.obstacle_locks[i].release() for i, _ in enumerate(dynamic_obs)]
247
+ return {'terminations': terminations, 'truncations': truncations}
229
248
 
230
249
  def step(self, action):
231
250
  if (
@@ -234,19 +253,18 @@ class SimpleEnv(AECEnv):
234
253
  ):
235
254
  self._was_dead_step(action)
236
255
  return
237
-
256
+
238
257
  current_idx = self._index_map[self.agent_selection]
258
+ next_idx = (current_idx + 1) % self.num_agents
259
+ self.agent_selection = self._agent_selector.next()
239
260
  self.current_actions[current_idx] = action
240
261
 
241
- self._execute_world_step()
242
- goal_dist = np.linalg.norm(self.world.agents[0].state.p_pos-self.world.agents[0].goal.state.p_pos)
243
- landmark_dist = [np.linalg.norm(self.world.agents[0].state.p_pos - landmark.state.p_pos) for landmark in self.world.landmarks[1:]]
244
- self.steps += 1
245
- min_dist = self.world.agents[0].size + self.world.agents[0].goal.size
246
- if self.steps >= self.max_cycles or min(landmark_dist) <= min_dist:
247
- self.truncations[self.agent_selection] = True
248
- if min_dist >= goal_dist:
249
- self.terminations[self.agent_selection] = True
262
+ if next_idx == 0:
263
+ self.steps += 1
264
+ self._execute_world_step()
265
+ status = self._episode_status()
266
+ self.terminations = status['terminations']
267
+ self.truncations = status['truncations']
250
268
 
251
269
  if self.render_mode == "human":
252
270
  self.render()
@@ -307,29 +325,6 @@ class SimpleEnv(AECEnv):
307
325
  0 < x < self.width and 0 < y < self.height
308
326
  ), f"Coordinates {(x, y)} are out of bounds."
309
327
 
310
- # text
311
- if isinstance(entity, Agent):
312
- if entity.silent:
313
- continue
314
- if np.all(entity.state.c == 0):
315
- word = "_"
316
- elif self.continuous_actions:
317
- word = (
318
- "[" +
319
- ",".join(
320
- [f"{comm:.2f}" for comm in entity.state.c]) + "]"
321
- )
322
-
323
- message = entity.name + " sends " + word + " "
324
- message_x_pos = self.width * 0.05
325
- message_y_pos = self.height * 0.95 - \
326
- (self.height * 0.05 * text_line)
327
- self.game_font.render_to(
328
- self.screen, (message_x_pos,
329
- message_y_pos), message, (0, 0, 0)
330
- )
331
- text_line += 1
332
-
333
328
  def close(self):
334
329
  if self.renderOn:
335
330
  pygame.event.pump()
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: Signal8
3
- Version: 1.7
3
+ Version: 1.9
4
4
  Summary: A multi-agent environment inspired by the Lewis Signaling Game, featuring eight unique problem configurations with both static and dynamic obstacles.
5
5
  Home-page: https://github.com/ethanmclark1/signal8
6
6
  Author: Ethan Clark
@@ -39,15 +39,15 @@ env.reset(options={"problem_name": "v_cluster"}))
39
39
  ## List of Problem Scenarios
40
40
 
41
41
  | Names | Description |
42
- | :------------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43
- | `Cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
44
- | `L-Shaped` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
45
- | `Vertical` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
46
- | `Horizontal` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
47
- | `Top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
48
- | `Bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
49
- | `Right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
50
- | `Left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
42
+ | :-----------: | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
43
+ | `v_cluster` | The environment features a central square formed by four obstacles, with the start on the left and the goal on the right side. |
44
+ | ``h_cluster`` | The environment showcases a central L-shaped configuration of four obstacles, with the starting point on the left boundary and the goal situated at the inner elbow of the L. |
45
+ | `v_wall` | Four obstacles are aligned vertically in the environment's center, with the starting point on the left boundary and the goal on the right boundary. |
46
+ | `h_wall` | Four obstacles are arranged in a horizontal line at the environment's midsection, with the start on the left boundary and the goal on the right boundary. |
47
+ | `top` | Four obstacles are placed at the top of the environment, while the start is situated in the bottom half on the left boundary and the goal is in the bottom half on the right boundary. |
48
+ | `bottom` | Four obstacles are placed at the bottom of the environment, while the start is situated in the top half on the left boundary and the goal is in the top half on the right boundary. |
49
+ | `right` | Four obstacles are situated on the right side of the environment, with the start positioned in the left half along the bottom boundary and the goal located in the left half along the top boundary. |
50
+ | `left` | Four obstacles are situated on the left side of the environment, with the start positioned in the right half along the bottom boundary and the goal located in the left half along the top boundary. |
51
51
 
52
52
  ## Paper Citation
53
53
 
@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
2
2
 
3
3
  setup(
4
4
  name="Signal8",
5
- version="1.7",
5
+ version="1.9",
6
6
  packages=find_packages(),
7
7
  author="Ethan Clark",
8
8
  author_email="eclark715@gmail.com.com",
@@ -1,151 +0,0 @@
1
- import math
2
- import numpy as np
3
-
4
- from gymnasium.utils import EzPickle
5
- from .utils.scenario import BaseScenario
6
- from .utils.core import Agent, Landmark, World, Action
7
- from .utils.simple_env import SimpleEnv, make_env
8
- from .utils.problems import get_problem, get_problem_list
9
-
10
- class raw_env(SimpleEnv, EzPickle):
11
- def __init__(
12
- self,
13
- agent_radius=0.1,
14
- num_obstacles=4,
15
- obstacle_radius=0.1,
16
- dynamic_obstacles=False,
17
- max_cycles=500,
18
- continuous_actions=False,
19
- render_mode="human"
20
- ):
21
-
22
- scenario = Scenario()
23
- world = scenario.make_world(
24
- agent_radius,
25
- num_obstacles,
26
- obstacle_radius,
27
- dynamic_obstacles
28
- )
29
-
30
- super().__init__(
31
- scenario=scenario,
32
- world=world,
33
- render_mode=render_mode,
34
- max_cycles=max_cycles,
35
- continuous_actions=continuous_actions,
36
- )
37
-
38
- self.metadata["agent_radius"] = agent_radius
39
- self.metadata["num_obstacles"] = num_obstacles
40
- self.metadata["obstacle_radius"] = obstacle_radius
41
- self.metadata["dynamic_obstacles"] = dynamic_obstacles
42
-
43
- env = make_env(raw_env)
44
-
45
- class Scenario(BaseScenario):
46
- def make_world(self, agent_radius, num_obstacles, obstacle_radius, dynamic_obstacles):
47
- world = World(dynamic_obstacles)
48
- world.problem_scenarios = get_problem_list()
49
-
50
- num_dynamic_obstacles = math.floor(num_obstacles / 2) if dynamic_obstacles else 0
51
- num_agents = num_dynamic_obstacles + 1
52
-
53
- world.agents = [Agent() for _ in range(num_agents)]
54
- for i, agent in enumerate(world.agents):
55
- agent.adversary = True if i > 0 else False
56
- base_name = "adversary" if agent.adversary else "agent"
57
- base_index = i - 1 if agent.adversary else 0
58
- agent.name = f"{base_name}_{base_index}"
59
- agent.collide = False
60
- agent.silent = True
61
- agent.blind = agent.adversary
62
- agent.size = agent_radius
63
-
64
- world.landmarks = [Landmark() for _ in range(num_obstacles - num_dynamic_obstacles + 1)]
65
- world.landmarks[0].name = "goal_0"
66
- world.landmarks[0].collide = False
67
- world.landmarks[0].movable = False
68
- world.landmarks[0].size = agent_radius
69
-
70
- for i, landmark in enumerate(world.landmarks[1:]):
71
- landmark.name = f"obs_{i}"
72
- landmark.collide = False
73
- landmark.size = obstacle_radius
74
-
75
- return world
76
-
77
- def reset_world(self, world, np_random, problem_name="v_cluster"):
78
- self.get_problem_scenario(world, problem_name)
79
- world.problem_name = problem_name
80
-
81
- # set state and color of both agents and landmarks
82
- for i, agent in enumerate(world.agents):
83
- if agent.adversary:
84
- agent.color = np.array([0.5, 0, 0])
85
- agent.state.p_vel = np.zeros(world.dim_p)
86
- agent.state.p_pos = np_random.uniform(*zip(*world.dynamic_obstacle_constr[(i+1) % len(world.dynamic_obstacle_constr)]))
87
- agent.action_callback = self.get_scripted_action
88
- else:
89
- agent.color = np.array([0, 0.8, 0])
90
- agent.state.p_vel = np.zeros(world.dim_p)
91
- agent.state.p_pos = np_random.uniform(*zip(*world.start_constr))
92
- agent.goal = world.landmarks[0]
93
- agent.goal.color = np.array([0, 0, 0.8])
94
- agent.goal.state.p_vel = np.zeros(world.dim_p)
95
- agent.goal.state.p_pos = np_random.uniform(*zip(*world.goal_constr))
96
-
97
- for landmark in world.landmarks[1:]:
98
- landmark.color = np.array([0.2, 0.2, 0.2])
99
- landmark.state.p_vel = np.zeros(world.dim_p)
100
- landmark.state.p_pos = np_random.uniform(*zip(*world.static_obstacle_constr))
101
-
102
- # Do not need to implement this function
103
- def reward(self, agent, world):
104
- return 0
105
-
106
- def observation(self, agent, world):
107
- return np.concatenate((agent.state.p_pos, agent.state.p_vel))
108
-
109
- # Get constraints on entities given the problem name
110
- def get_problem_scenario(self, world, problem_name):
111
- problem = get_problem(problem_name, world.dynamic_obstacles)
112
- world.start_constr = problem['start']
113
- world.goal_constr = problem['goal']
114
- world.static_obstacle_constr = problem['static_obs']
115
-
116
- if world.dynamic_obstacles:
117
- world.dynamic_obstacle_constr = problem['dynamic_obs']
118
-
119
- # Get scripted action for adversarial agents s.t. they move in a straight line along their constraint
120
- def get_scripted_action(self, agent, world):
121
- action = Action()
122
- action.u = np.zeros(world.dim_p)
123
- action.c = np.zeros(world.dim_c)
124
-
125
- constr = world.dynamic_obstacle_constr(agent.name[-1])
126
- constr_size = [(abs(point[0]) + abs(point[1])) for point in constr]
127
- dimension = 'horizontal' if constr_size[0] > constr_size[1] else 'vertical'
128
-
129
- # Always start off moving in the positive direction
130
- if dimension == 'horizontal':
131
- if (agent.state.p_vel == 0).all():
132
- action.u[0] = +1.0
133
- else: # If moving, continue in the direction of velocity
134
- if agent.state.p_pos[0] <= constr[0][0]: # If at left constraint, move right
135
- action.u[0] = +1.0
136
- elif agent.state.p_pos[0] >= constr[1][0]: # If at right constraint, move left
137
- action.u[0] = -1.0
138
- else: # Otherwise, follow the direction of velocity
139
- action.u[0] = np.sign(agent.state.p_vel[0])
140
- else:
141
- if (agent.state.p_vel == 0).all():
142
- action.u[1] = +1.0
143
- else: # If moving, continue in the direction of velocity
144
- if agent.state.p_pos[1] <= constr[0][1]: # If at bottom constraint, move up
145
- action.u[1] = +1.0
146
- elif agent.state.p_pos[1] >= constr[1][1]: # If at top constraint, move down
147
- action.u[1] = -1.0
148
- else: # Otherwise, follow the direction of velocity
149
- action.u[1] = np.sign(agent.state.p_vel[1])
150
-
151
- return action
@@ -1,4 +0,0 @@
1
- import Signal8
2
-
3
- env = Signal8.env(dynamic_obstacles=True, render_mode="human")
4
- env.reset(options={"problem_name": "v_cluster"})
@@ -1,228 +0,0 @@
1
- import numpy as np
2
-
3
-
4
- class EntityState: # physical/external base state of all entities
5
- def __init__(self):
6
- # physical position
7
- self.p_pos = None
8
- # physical velocity
9
- self.p_vel = None
10
-
11
-
12
- class AgentState(
13
- EntityState
14
- ): # state of agents (including communication and internal/mental state)
15
- def __init__(self):
16
- super().__init__()
17
- # communication utterance
18
- self.c = None
19
-
20
-
21
- class Action: # action of the agent
22
- def __init__(self):
23
- # physical action
24
- self.u = None
25
- # communication action
26
- self.c = None
27
-
28
-
29
- class Entity: # properties and state of physical world entity
30
- def __init__(self):
31
- # name
32
- self.name = ""
33
- # properties:
34
- self.size = 0.050
35
- # entity can move / be pushed
36
- self.movable = False
37
- # entity collides with others
38
- self.collide = True
39
- # material density (affects mass)
40
- self.density = 25.0
41
- # color
42
- self.color = None
43
- # max speed and accel
44
- self.max_speed = None
45
- self.accel = None
46
- # state
47
- self.state = EntityState()
48
- # mass
49
- self.initial_mass = 1.0
50
-
51
- @property
52
- def mass(self):
53
- return self.initial_mass
54
-
55
-
56
- class Landmark(Entity): # properties of landmark entities
57
- def __init__(self):
58
- super().__init__()
59
-
60
-
61
- class Agent(Entity): # properties of agent entities
62
- def __init__(self):
63
- super().__init__()
64
- # agents are movable by default
65
- self.movable = True
66
- # cannot send communication signals
67
- self.silent = False
68
- # cannot observe the world
69
- self.blind = False
70
- # physical motor noise amount
71
- self.u_noise = None
72
- # communication noise amount
73
- self.c_noise = None
74
- # control range
75
- self.u_range = 1.0
76
- # state
77
- self.state = AgentState()
78
- # action
79
- self.action = Action()
80
- # script behavior to execute
81
- self.action_callback = None
82
-
83
-
84
- class World: # multi-agent world
85
- def __init__(self, dynamic_obstacles=False):
86
- # list of agents and entities (can change at execution-time!)
87
- self.agents = []
88
- self.landmarks = []
89
- # communication channel dimensionality
90
- self.dim_c = 0
91
- # position dimensionality
92
- self.dim_p = 2
93
- # color dimensionality
94
- self.dim_color = 3
95
- # simulation timestep
96
- self.dt = 0.1
97
- # physical damping
98
- self.damping = 0.25
99
- # contact response parameters
100
- self.contact_force = 1e2
101
- self.contact_margin = 1e-3
102
- # problem scenarios
103
- self.dynamic_obstacles = dynamic_obstacles
104
- self.problem_scenarios = None
105
- self.problem_name = None
106
- self.start_constr = None
107
- self.goal_constr = None
108
- self.static_obstacle_constr = None
109
- self.dynamic_obstacle_constr = None
110
-
111
- # return all entities in the world
112
- @property
113
- def entities(self):
114
- return self.agents + self.landmarks
115
-
116
- # return all agents controllable by external policies
117
- @property
118
- def policy_agents(self):
119
- return [agent for agent in self.agents if agent.action_callback is None]
120
-
121
- # return all agents controlled by world scripts
122
- @property
123
- def scripted_agents(self):
124
- return [agent for agent in self.agents if agent.action_callback is not None]
125
-
126
- # update state of the world
127
- def step(self):
128
- # set actions for scripted agents
129
- for agent in self.scripted_agents:
130
- agent.action = agent.action_callback(agent, self)
131
- # gather forces applied to entities
132
- p_force = [None] * len(self.entities)
133
- # apply agent physical controls
134
- p_force = self.apply_action_force(p_force)
135
- # apply environment forces
136
- p_force = self.apply_environment_force(p_force)
137
- # integrate physical state
138
- self.integrate_state(p_force)
139
- # update agent state
140
- for agent in self.agents:
141
- self.update_agent_state(agent)
142
-
143
- # gather agent action forces
144
- def apply_action_force(self, p_force):
145
- # set applied forces
146
- for i, agent in enumerate(self.agents):
147
- if agent.movable:
148
- noise = (
149
- np.random.randn(*agent.action.u.shape) * agent.u_noise
150
- if agent.u_noise
151
- else 0.0
152
- )
153
- p_force[i] = agent.action.u + noise
154
- return p_force
155
-
156
- # gather physical forces acting on entities
157
- def apply_environment_force(self, p_force):
158
- # simple (but inefficient) collision response
159
- for a, entity_a in enumerate(self.entities):
160
- for b, entity_b in enumerate(self.entities):
161
- if b <= a:
162
- continue
163
- [f_a, f_b] = self.get_collision_force(entity_a, entity_b)
164
- if f_a is not None:
165
- if p_force[a] is None:
166
- p_force[a] = 0.0
167
- p_force[a] = f_a + p_force[a]
168
- if f_b is not None:
169
- if p_force[b] is None:
170
- p_force[b] = 0.0
171
- p_force[b] = f_b + p_force[b]
172
- return p_force
173
-
174
- # integrate physical state
175
- def integrate_state(self, p_force):
176
- for i, entity in enumerate(self.entities):
177
- if not entity.movable:
178
- continue
179
- entity.state.p_vel = entity.state.p_vel * (1 - self.damping)
180
- if p_force[i] is not None:
181
- entity.state.p_vel += (p_force[i] / entity.mass) * self.dt
182
- if entity.max_speed is not None:
183
- speed = np.sqrt(
184
- np.square(entity.state.p_vel[0]) +
185
- np.square(entity.state.p_vel[1])
186
- )
187
- if speed > entity.max_speed:
188
- entity.state.p_vel = (
189
- entity.state.p_vel
190
- / np.sqrt(
191
- np.square(entity.state.p_vel[0])
192
- + np.square(entity.state.p_vel[1])
193
- )
194
- * entity.max_speed
195
- )
196
-
197
- entity.state.p_pos += entity.state.p_vel * self.dt
198
-
199
- def update_agent_state(self, agent):
200
- # set communication state (directly for now)
201
- if agent.silent:
202
- agent.state.c = np.zeros(self.dim_c)
203
- else:
204
- noise = (
205
- np.random.randn(*agent.action.c.shape) * agent.c_noise
206
- if agent.c_noise
207
- else 0.0
208
- )
209
- agent.state.c = agent.action.c + noise
210
-
211
- # get collision forces for any contact between two entities
212
- def get_collision_force(self, entity_a, entity_b):
213
- if (not entity_a.collide) or (not entity_b.collide):
214
- return [None, None] # not a collider
215
- if entity_a is entity_b:
216
- return [None, None] # don't collide against itself
217
- # compute actual distance between entities
218
- delta_pos = entity_a.state.p_pos - entity_b.state.p_pos
219
- dist = np.sqrt(np.sum(np.square(delta_pos)))
220
- # minimum allowable distance
221
- dist_min = entity_a.size + entity_b.size
222
- # softmax penetration
223
- k = self.contact_margin
224
- penetration = np.logaddexp(0, -(dist - dist_min) / k) * k
225
- force = self.contact_force * delta_pos / dist * penetration
226
- force_a = +force if entity_a.movable else None
227
- force_b = -force if entity_b.movable else None
228
- return [force_a, force_b]
@@ -1,64 +0,0 @@
1
- problems = {
2
- 'v_cluster': {
3
- 'start': ((-0.15, 0.15), (-1, -0.80)),
4
- 'goal': ((-0.15, 0.15), (0.80, 1)),
5
- 'static_obs': ((-0.15, 0.15), (-0.15, 0.15))
6
- },
7
- 'h_cluster': {
8
- 'start': ((-1, -0.80), (-0.15, 0.15)),
9
- 'goal': ((0.80, 1), (-0.15, 0.15)),
10
- 'static_obs': ((-0.15, 0.15), (-0.15, 0.15))
11
- },
12
- 'v_wall': {
13
- 'start': ((-1, 1), (-1, -0.80)),
14
- 'goal': ((-1, 1), (0.80, 1)),
15
- 'static_obs': ((-0.6, 0.6), (-0.075, 0.075))
16
- },
17
- 'h_wall': {
18
- 'start': ((-1, -0.80), (-1, 1)),
19
- 'goal': ((0.80, 1), (-1, 1)),
20
- 'static_obs': ((-0.075, 0.075), (-0.6, 0.6))
21
- },
22
- 'left': {
23
- 'start': ((0, 1), (-1, -0.80)),
24
- 'goal': ((0, 1), (0.80, 1)),
25
- 'static_obs': ((-1, 0), (-1, 1))
26
- },
27
- 'right': {
28
- 'start': ((-1, 0), (-1, -0.80)),
29
- 'goal': ((-1, 0), (0.80, 1)),
30
- 'static_obs': ((0, 1), (-1, 1))
31
- },
32
- 'top': {
33
- 'start': ((-1, -0.80), (-1, 0)),
34
- 'goal': ((0.80, 1), (-1, 0)),
35
- 'static_obs': ((-1, 1), (0, 1))
36
- },
37
- 'bottom': {
38
- 'start': ((-1, -0.80), (0, 1)),
39
- 'goal': ((0.80, 1), (0, 1)),
40
- 'static_obs': ((-1, 1), (-1, 0))
41
- },
42
- }
43
-
44
- def get_problem_list():
45
- return list(problems.keys())
46
-
47
- def get_problem(scenario_name, dynamic_obstacles):
48
- static_scenario = problems[scenario_name]
49
-
50
- if dynamic_obstacles:
51
- dynamic_elements = {
52
- 'v_cluster': {'dynamic_obs': (((-1, 1), (-0.5, -0.35)), ((-1, 1), (0.35, 0.5)))},
53
- 'h_cluster': {'dynamic_obs': (((-0.5, -0.35), (-1, 1)), ((0.35, 0.5), (-1, 1)))},
54
- 'v_wall': {'dynamic_obs': (((-1, 1), (-0.25, -0.075)), ((-1, 1), (0.075, 0.25)))},
55
- 'h_wall': {'dynamic_obs': (((-0.25, -0.075), (-1, 1)), ((0.075, 0.25), (-1, 1)))},
56
- 'left': {'dynamic_obs': (((-1, 0.5), (-0.45, -0.55)), ((-1, 0.5), (0.45, 0.55)))},
57
- 'right': {'dynamic_obs': (((-0.5, 1), (-0.45, -0.55)), ((-0.5, 1), (0.45, 0.55)))},
58
- 'top': {'dynamic_obs': (((-0.45, -0.55), (-0.5, 1)), ((0.45, 0.55), (-0.5, 1)))},
59
- 'bottom': {'dynamic_obs': (((-0.45, -0.55), (-1, 0.5)), ((0.45, 0.55), (-1, 0.5)))},
60
- }
61
- dynamic_scenario = {**static_scenario, **dynamic_elements[scenario_name]}
62
- return dynamic_scenario
63
-
64
- return static_scenario
File without changes
File without changes
File without changes
File without changes
File without changes