rlgym-rlbot 0.3.0__tar.gz

rlgym_rlbot-0.3.0/LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

rlgym_rlbot-0.3.0/PKG-INFO

@@ -0,0 +1,218 @@
+Metadata-Version: 2.4
+Name: rlgym-rlbot
+Version: 0.3.0
+Summary: Wrapper logic to use bots trained with RLGym v2 using the RocketSimTransitionEngine in RLBot v5
+Author: Jonathan Keegan
+Project-URL: Repository, https://github.com/JPK314/rlgym-rlbot.git
+Project-URL: Issues, https://github.com/JPK314/rlgym-rlbot/issues
+Keywords: rocket-league,gym,reinforcement-learning,simulation,rlgym,rocketsim
+Classifier: Programming Language :: Python :: 3
+Requires-Python: <3.13,>=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: rlbot==2.0.0b52
+Requires-Dist: rlbot_flatbuffers>=0.18.8
+Requires-Dist: rlgym[api]>=2.0.1
+Requires-Dist: rlgym-compat>=2.0.0
+Provides-Extra: rocketsim
+Requires-Dist: rocketsim>=2.2.1; extra == "rocketsim"
+Dynamic: license-file
+
+# RLBot v5 - RLGym v2 Wrapper
+
+A high accuracy wrapper interface allowing bots trained with [RLGym v2](https://rlgym.org/) using the [RocketSimEngine transition engine](https://github.com/lucas-emery/rocket-league-gym/blob/main/rlgym/rocket_league/sim/rocketsim_engine.py) to play in [RLBot v5](https://wiki.rlbot.org/v5/) with minimal configuration required by the user.
+
+## Making a bot with this wrapper
+
+Below is a simple snippet from what the main file in the folder of a bot using this wrapper might look like:
+
+```py
+from rlgym_rlbot import RLGymBot, RLGymBotConfig
+
+class MyBot(RLGymBot):
+    def __init__(self):
+        super().__init__(
+            obs_builder=MyObsBuilder(),
+            action_parser=RepeatAction(LookupTableAction(), 8),
+            default_agent_id="jpk314/mybot/v0.1",
+            config=RLGymBotConfig(above_240_fps_mode=True),
+        )
+        self.agent = Agent()
+
+    def get_action(self, obs, game_state, packet):
+        return self.agent.act(obs)
+
+
+if __name__ == "__main__":
+    MyBot().run()
+```
+In the above example, I am using a custom obs builder (implementation not shown) and a standard action parser using imports from the RLGym[rl] Python package. Where you put your configuration objects' implementations so that they can be used in the above snippet is up to you. The other missing definition is the Agent class - this is an abstraction you might have in your learning framework that takes something of type ObsType and converts it to something of type ActionType. Typically this is something that converts a batch of ObsType to a Torch tensor, runs it through your model, and then converts the result to a batch of ActionType. You will need to know what you're using and how to rip it out of the learning framework you're using to use it here instead.
+
+The `RLGymBotConfig` class contains a couple useful knobs which are described in the following section.
+
+Other mandatory configuration files that need to go in your bot's folder are described [here](https://wiki.rlbot.org/v5/botmaking/config-files/) on the RLBot v5 wiki.
+
+### Optional Overridable Methods
+
+#### Mixing with hard-coded actions or action sequences
+The method `get_hardcoded_action` allows you to override whatever your model was planning to do and instead do whatever `ControllerState` you return from this method. The default return value is `None` which means the model will continue with whatever it was doing. This is most valuable for hard coding kickoff maneuvers. You can manage the maneuver state in your subclass of RLGymBot. If you ever return a non-`None` value and then return `None` again, the wrapper will simulate an on-the-fly reset of the environment for your config objects so that the model can resume playing.
+
+#### Sending messages
+Sending messages to other bots or in the chat can be done during the `get_hardcoded_action` call (which happens every tick during gameplay) or during the `get_other_packet_output` call (which happens the rest of the time) by calling `self.send_match_comm`.
+
+#### Recovering from one or more incorrectly taken actions
+The method `decide_missed_action_recovery_style` allows for custom logic to decide which recovery style to use - either `MissedActionRecoveryStyle.RESET` or `MissedActionRecoveryStyle.IGNORE`. The inputs to this method provide a bunch of useful values that you may want in order to determine how to handle this, including
+- The current packet and game state
+- The number of ticks left in the currently in use action (remember that the output of your action parser is an action which corresponds to a sequence of ControllerStates, one per tick)
+- The past ticks where an incorrect `ControllerState` was used and the `ControllerState` that was intended based on the output of your action parser
+
+The default return value is `IGNORE`. There are situations where you may want to perform an on-the-fly reset to generate a new action from the current state instead. You probably do NOT want to perform an on-the-fly reset if you're currently in the middle of a flip, for example, because it's not possible with the RocketSimEngine to start an episode in the middle of a flip.
+
+#### Recovering from a missed tick needed for stepping the environment
+The method `decide_missed_step_tick_recovery_style` allows for custom logic to decide which recovery style to use - either `MissedStepTickRecoveryStyle.RESET` or `MissedStepTickRecoveryStyle.USE_NEAREST`. The inputs to this method provide a bunch of useful values that you may want in order to determine how to handle this, including
+- The current packet and game state
+- The number of ticks left in the currently in use action (remember that the output of your action parser is an action which corresponds to a sequence of ControllerStates, one per tick)
+- The last `ControllerState` used by the bot
+
+The default return value is `USE_NEAREST`. Unless your config objects are really sensitive, this is probably good enough for all situations. However, if you come up with a scenario where you think it would be preferable to perform an on-the-fly reset instead of using a slightly inaccurate (or temporally shifted, whatever way you want to think about it) game state for your config objects, you can return `RESET` in that scenario instead.
+
+#### Handling cosmetic actions (kickoff wiggle)
+The method `get_other_packet_output` can be used to return `ControllerState`s during times where your actions don't actually matter, in particularly during the Countdown match phase. For the first 440 ticks of countdown, you can do whatever wheel wiggling you desire by returning controller states here. Why 440? Countdown always lasts at least 479 ticks from my testing, and so this is just a safe margin.
+
+#### Handling reward function output
+The method `handle_reward` is called if you pass in a `RewardFunction` to the `RLGymBot` constructor on every step. You can decide what to do with this (display it on screen? write it to a file?) by overwriting this method. By default it is a no-op.
+
+#### Various standard Bot python interface methods and fields
+Many convenience methods from the `Bot` class in the [python interface](https://github.com/RLBot/python-interface) are available here as well. These include:
+- `initialize`
+- `retire`
+- `set_loadout`
+- `handle_match_comm`
+- `send_match_comm`
+- `set_game_state`
+
+There are convenience fields as well:
+- `self.player_id`
+- `self.name`
+- `self.index`
+- `self.team`
+- `self.match_config`
+- `self.field_info`
+- `self.ball_prediction`
+- `self.logger`
+- `self.renderer`
+
+## Config Options
+The `RLGymBotConfig` class contains the following options:
+
+### step_offset
+StepOffset dataclass with fields `offset` and `relative_to` - default `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END`. This option determines when during the course of executing the last action a game state is taken to be used for simulating a step of the env (most importantly, for building your observation). See below for specific notes on how this works.
+- Use `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END` if you use the [RocketSimEngine transition engine](https://github.com/lucas-emery/rocket-league-gym/blob/main/rlgym/rocket_league/sim/rocketsim_engine.py) with `rlbot_delay=True`
+- Use `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_END` if you use the above with `rlbot_delay=False`
+- Use `offset=1` and `relative_to=StepOffsetRelativeTo.ACTION_START` if you are using RLGym v1 or rlgym-sim
+
+Here is a helpful diagram explaining how these values fully work. For simplicity, let's say every action (by this I mean result from the action parser) has a shape of (4,8) i.e. it defines 4 ticks worth of actions sent to Rocket League. `.` represents a regular state, and `-` represents a state transition.
+
+Values `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_START` would look like the below - in particular, the (n+1)th action is decided using the state that the nth action is about to start from:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | └Action 3 tick 4, etc
+|| | | ||| | | ||| | └Action 3 tick 3
+|| | | ||| | | ||| └Action 3 tick 2
+|| | | ||| | | ||└Action 3 tick 1
+|| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|| | | ||| | | └Action 2 tick 4
+|| | | ||| | └Action 2 tick 3
+|| | | ||| └Action 2 tick 2
+|| | | ||└Action 2 tick 1
+|| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | └Action 1 tick 4
+|| | └Action 1 tick 3
+|| └Action 1 tick 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 and 2 created using this state, actions 1 and 2 decided using obs 1 and 2 respectively
+```
+
+Values `offset=1` and `relative_to=StepOffsetRelativeTo.ACTION_START` would look like the below - in particular, an action is decided using the state one tick after the last action's env action ticks began:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|||| | | ||| | | ||| | └Action 3 tick 4, etc
+|||| | | ||| | | ||| └Action 3 tick 3
+|||| | | ||| | | ||└Action 3 tick 2
+|||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|||| | | ||| | | └Action 3 tick 1
+|||| | | ||| | └Action 2 tick 4
+|||| | | ||| └Action 2 tick 3
+|||| | | ||└Action 2 tick 2
+|||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|||| | | └Action 2 tick 1
+|||| | └Action 1 tick 4
+|||| └Action 1 tick 3
+|||└Action 1 tick 2
+||└Obs 2 created using this state, action 2 decided using obs 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+
+Values `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_END` would look like the below - in particular, the next action action is decided using the state after the last action's env action ticks are done:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4, etc
+|| | | ||| | | ||| | | └Action 3 tick 4
+|| | | ||| | | ||| | └Action 3 tick 3
+|| | | ||| | | ||| └Action 3 tick 2
+|| | | ||| | | ||└Action 3 tick 1 
+|| | | ||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | ||| | | └Action 2 tick 4
+|| | | ||| | └Action 2 tick 3
+|| | | ||| └Action 2 tick 2
+|| | | ||└Action 2 tick 1
+|| | | |└Obs 2 created using this state, action 2 decided using obs 2
+|| | | └Action 1 tick 4
+|| | └Action 1 tick 3
+|| └Action 1 tick 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+
+Values `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END` would look like the below - in particular, an action is decided using the state one tick before the last action's env action ticks are done:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | ||└Action 3 tick 4, etc
+|| | | ||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|| | | ||| | | ||| | | └Action 3 tick 3
+|| | | ||| | | ||| | └Action 3 tick 2
+|| | | ||| | | ||| └Action 3 tick 1
+|| | | ||| | | ||└Action 2 tick 4
+|| | | ||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | ||| | | └Action 2 tick 3
+|| | | ||| | └Action 2 tick 2
+|| | | ||| └Action 2 tick 1
+|| | | ||└Action 1 tick 4
+|| | | |└Obs 2 created using this state, action 2 decided using obs 2
+|| | | └Action 1 tick 3
+|| | └Action 1 tick 2
+|| └Action 1 tick 1
+|└"void" state transition where sim is stepped without any cars' controls being set
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+### standard_map
+Boolean - default `true`. This option is passed to [rlgym-compat](https://github.com/JPK314/rlgym-compat)'s GameState in order to configure boost pad ordering.
+
+### sim_extra_info
+Boolean - default `false`. This option determines whether rlgym-compat's [SimExtraInfo](https://github.com/JPK314/rlgym-compat/blob/main/rlgym_compat/sim_extra_info.py) is used for extra information that otherwise cannot be derived from the RLBot GamePacket.
+
+### agent_ids_fn
+Optional Callable - default `None`. This option determines how rlgym-compat assigns `AgentID`s to players when creating the compat GameState from the RLBot v5's `flat.GamePacket`. The input is a `GamePacket` and the output is a map from `PlayerInfo.player_id` to your desired `AgentID` data type and value for your config objects. Normally this would be handled by your `StateMutator`, but that functionality is not implemented yet and this acts as a simple override without the other potential `StateMutator` behavior.
+
+
+## What does "accurate" mean? What is this wrapper doing?
+This wrapper attempts to replicate the training conditions of your bot in RLGym as accurately as possible. This means a lot of consideration is put into tick-perfectly matching everything where possible while maintaining the flow and functionality expected by arbitrary config objects and by proxy your model during training using RLGym v2.
+
+### Tick perfectly matching actions
+This wrapper assumes that your computer is running Rocket League in such a way that you have enough time for your actions to always get submitted for use between the latest received packet's physics frame and the next physics frame. The optimal conditions for this are when you play using a capped frame rate that is either 120 or 240, because these frame rates have the largest windows between when the wrapper receives a packet from RLBot Core and when you can no longer submit an action and have it be used during the state transition between the latest received packet's physics tick and the next physics tick. This window is about 6 milliseconds on average with a small standard deviation when the framerate is capped to 120 or 240 (NOT 360!), and is about 4 milliseconds on average with a much wider standard deviation when left uncapped. Other frame rates do very poorly and are not recommended. Possibly if you modify the command line argument RLBot passes for packet send rate from 240 to another number you may be able to get up to a 6 millisecond window using another frame rate, but it's untested and annoying to configure. Just use 120 or 240 capped, or hope for the best uncapped.
+
+### On-the-fly resets and countdown actions
+This wrapper tries to tick perfectly match how your config objects and model interact with each other and the sim in RLGym v2. However, there are some situations where this is just not possible. One such situation is when an on-the-fly reset is performed. These can happen if your `decide_*_recovery_style` methods return `RESET` or if you stop taking a hard-coded action. Due to an implementation detail, in RLGym v2's RocketSimEngine with the default constructor parameter (`rlbot_delay=True`), after resetting, all agents will do one tick of an all-zeros action before beginning the action generated due to the reset state's observation. In the language of this wrapper, this means that after resetting, if you have `offset < 0` and `relative_to = StepOffsetRelativeTo.ACTION_END`, in theory you should take `offset` ticks of the zero action in order to perfectly match how everything works during training. This is obviously not desirable against an opponent with no such restrictions, and so instead the observation is generated using the game state from `offset` ticks ago, and the fact that a non-all-zeros action was taken for the ticks since that point until this new action is due to start (this tick) is assumed to come out in the wash.
+
+Due to the duration of the Countdown phase being unreliable, it is impossible to perfectly time the reset so that you start the action as soon as Countdown ends (your actions start getting used on the state transition between the last packet of Countdown and the first packet of Kickoff). During the Countdown match phase, the cars settle onto the ground and cannot move (wheel wiggling is purely cosmetic), so the difference between using the proper tick of Countdown and some older tick of countdown is essentially zero. This wrapper uses the 440th (or next available after) tick after the Countdown phase began as the tick on which to generate the observation that will be used once Countdown ends, and simply sends the first ControllerState of this action every tick from then on until the latest packet is Kickoff instead of Countdown. This should work essentially perfectly UNLESS you intend to jump on the first tick. Because the ControllerState is sent repeatedly during Countdown, the game will think you are just holding jump and so once you can actually move you won't jump, you'll just be holding jump. So... don't do that!

rlgym_rlbot-0.3.0/README.md

@@ -0,0 +1,198 @@
+# RLBot v5 - RLGym v2 Wrapper
+
+A high accuracy wrapper interface allowing bots trained with [RLGym v2](https://rlgym.org/) using the [RocketSimEngine transition engine](https://github.com/lucas-emery/rocket-league-gym/blob/main/rlgym/rocket_league/sim/rocketsim_engine.py) to play in [RLBot v5](https://wiki.rlbot.org/v5/) with minimal configuration required by the user.
+
+## Making a bot with this wrapper
+
+Below is a simple snippet from what the main file in the folder of a bot using this wrapper might look like:
+
+```py
+from rlgym_rlbot import RLGymBot, RLGymBotConfig
+
+class MyBot(RLGymBot):
+    def __init__(self):
+        super().__init__(
+            obs_builder=MyObsBuilder(),
+            action_parser=RepeatAction(LookupTableAction(), 8),
+            default_agent_id="jpk314/mybot/v0.1",
+            config=RLGymBotConfig(above_240_fps_mode=True),
+        )
+        self.agent = Agent()
+
+    def get_action(self, obs, game_state, packet):
+        return self.agent.act(obs)
+
+
+if __name__ == "__main__":
+    MyBot().run()
+```
+In the above example, I am using a custom obs builder (implementation not shown) and a standard action parser using imports from the RLGym[rl] Python package. Where you put your configuration objects' implementations so that they can be used in the above snippet is up to you. The other missing definition is the Agent class - this is an abstraction you might have in your learning framework that takes something of type ObsType and converts it to something of type ActionType. Typically this is something that converts a batch of ObsType to a Torch tensor, runs it through your model, and then converts the result to a batch of ActionType. You will need to know what you're using and how to rip it out of the learning framework you're using to use it here instead.
+
+The `RLGymBotConfig` class contains a couple useful knobs which are described in the following section.
+
+Other mandatory configuration files that need to go in your bot's folder are described [here](https://wiki.rlbot.org/v5/botmaking/config-files/) on the RLBot v5 wiki.
+
+### Optional Overridable Methods
+
+#### Mixing with hard-coded actions or action sequences
+The method `get_hardcoded_action` allows you to override whatever your model was planning to do and instead do whatever `ControllerState` you return from this method. The default return value is `None` which means the model will continue with whatever it was doing. This is most valuable for hard coding kickoff maneuvers. You can manage the maneuver state in your subclass of RLGymBot. If you ever return a non-`None` value and then return `None` again, the wrapper will simulate an on-the-fly reset of the environment for your config objects so that the model can resume playing.
+
+#### Sending messages
+Sending messages to other bots or in the chat can be done during the `get_hardcoded_action` call (which happens every tick during gameplay) or during the `get_other_packet_output` call (which happens the rest of the time) by calling `self.send_match_comm`.
+
+#### Recovering from one or more incorrectly taken actions
+The method `decide_missed_action_recovery_style` allows for custom logic to decide which recovery style to use - either `MissedActionRecoveryStyle.RESET` or `MissedActionRecoveryStyle.IGNORE`. The inputs to this method provide a bunch of useful values that you may want in order to determine how to handle this, including
+- The current packet and game state
+- The number of ticks left in the currently in use action (remember that the output of your action parser is an action which corresponds to a sequence of ControllerStates, one per tick)
+- The past ticks where an incorrect `ControllerState` was used and the `ControllerState` that was intended based on the output of your action parser
+
+The default return value is `IGNORE`. There are situations where you may want to perform an on-the-fly reset to generate a new action from the current state instead. You probably do NOT want to perform an on-the-fly reset if you're currently in the middle of a flip, for example, because it's not possible with the RocketSimEngine to start an episode in the middle of a flip.
+
+#### Recovering from a missed tick needed for stepping the environment
+The method `decide_missed_step_tick_recovery_style` allows for custom logic to decide which recovery style to use - either `MissedStepTickRecoveryStyle.RESET` or `MissedStepTickRecoveryStyle.USE_NEAREST`. The inputs to this method provide a bunch of useful values that you may want in order to determine how to handle this, including
+- The current packet and game state
+- The number of ticks left in the currently in use action (remember that the output of your action parser is an action which corresponds to a sequence of ControllerStates, one per tick)
+- The last `ControllerState` used by the bot
+
+The default return value is `USE_NEAREST`. Unless your config objects are really sensitive, this is probably good enough for all situations. However, if you come up with a scenario where you think it would be preferable to perform an on-the-fly reset instead of using a slightly inaccurate (or temporally shifted, whatever way you want to think about it) game state for your config objects, you can return `RESET` in that scenario instead.
+
+#### Handling cosmetic actions (kickoff wiggle)
+The method `get_other_packet_output` can be used to return `ControllerState`s during times where your actions don't actually matter, in particularly during the Countdown match phase. For the first 440 ticks of countdown, you can do whatever wheel wiggling you desire by returning controller states here. Why 440? Countdown always lasts at least 479 ticks from my testing, and so this is just a safe margin.
+
+#### Handling reward function output
+The method `handle_reward` is called if you pass in a `RewardFunction` to the `RLGymBot` constructor on every step. You can decide what to do with this (display it on screen? write it to a file?) by overwriting this method. By default it is a no-op.
+
+#### Various standard Bot python interface methods and fields
+Many convenience methods from the `Bot` class in the [python interface](https://github.com/RLBot/python-interface) are available here as well. These include:
+- `initialize`
+- `retire`
+- `set_loadout`
+- `handle_match_comm`
+- `send_match_comm`
+- `set_game_state`
+
+There are convenience fields as well:
+- `self.player_id`
+- `self.name`
+- `self.index`
+- `self.team`
+- `self.match_config`
+- `self.field_info`
+- `self.ball_prediction`
+- `self.logger`
+- `self.renderer`
+
+## Config Options
+The `RLGymBotConfig` class contains the following options:
+
+### step_offset
+StepOffset dataclass with fields `offset` and `relative_to` - default `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END`. This option determines when during the course of executing the last action a game state is taken to be used for simulating a step of the env (most importantly, for building your observation). See below for specific notes on how this works.
+- Use `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END` if you use the [RocketSimEngine transition engine](https://github.com/lucas-emery/rocket-league-gym/blob/main/rlgym/rocket_league/sim/rocketsim_engine.py) with `rlbot_delay=True`
+- Use `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_END` if you use the above with `rlbot_delay=False`
+- Use `offset=1` and `relative_to=StepOffsetRelativeTo.ACTION_START` if you are using RLGym v1 or rlgym-sim
+
+Here is a helpful diagram explaining how these values fully work. For simplicity, let's say every action (by this I mean result from the action parser) has a shape of (4,8) i.e. it defines 4 ticks worth of actions sent to Rocket League. `.` represents a regular state, and `-` represents a state transition.
+
+Values `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_START` would look like the below - in particular, the (n+1)th action is decided using the state that the nth action is about to start from:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | └Action 3 tick 4, etc
+|| | | ||| | | ||| | └Action 3 tick 3
+|| | | ||| | | ||| └Action 3 tick 2
+|| | | ||| | | ||└Action 3 tick 1
+|| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|| | | ||| | | └Action 2 tick 4
+|| | | ||| | └Action 2 tick 3
+|| | | ||| └Action 2 tick 2
+|| | | ||└Action 2 tick 1
+|| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | └Action 1 tick 4
+|| | └Action 1 tick 3
+|| └Action 1 tick 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 and 2 created using this state, actions 1 and 2 decided using obs 1 and 2 respectively
+```
+
+Values `offset=1` and `relative_to=StepOffsetRelativeTo.ACTION_START` would look like the below - in particular, an action is decided using the state one tick after the last action's env action ticks began:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|||| | | ||| | | ||| | └Action 3 tick 4, etc
+|||| | | ||| | | ||| └Action 3 tick 3
+|||| | | ||| | | ||└Action 3 tick 2
+|||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|||| | | ||| | | └Action 3 tick 1
+|||| | | ||| | └Action 2 tick 4
+|||| | | ||| └Action 2 tick 3
+|||| | | ||└Action 2 tick 2
+|||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|||| | | └Action 2 tick 1
+|||| | └Action 1 tick 4
+|||| └Action 1 tick 3
+|||└Action 1 tick 2
+||└Obs 2 created using this state, action 2 decided using obs 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+
+Values `offset=0` and `relative_to=StepOffsetRelativeTo.ACTION_END` would look like the below - in particular, the next action action is decided using the state after the last action's env action ticks are done:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4, etc
+|| | | ||| | | ||| | | └Action 3 tick 4
+|| | | ||| | | ||| | └Action 3 tick 3
+|| | | ||| | | ||| └Action 3 tick 2
+|| | | ||| | | ||└Action 3 tick 1 
+|| | | ||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | ||| | | └Action 2 tick 4
+|| | | ||| | └Action 2 tick 3
+|| | | ||| └Action 2 tick 2
+|| | | ||└Action 2 tick 1
+|| | | |└Obs 2 created using this state, action 2 decided using obs 2
+|| | | └Action 1 tick 4
+|| | └Action 1 tick 3
+|| └Action 1 tick 2
+|└Action 1 tick 1
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+
+Values `offset=-1` and `relative_to=StepOffsetRelativeTo.ACTION_END` would look like the below - in particular, an action is decided using the state one tick before the last action's env action ticks are done:
+```
+.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
+|| | | ||| | | ||| | | ||└Action 3 tick 4, etc
+|| | | ||| | | ||| | | |└Obs 4 created using this state, action 4 decided using obs 4
+|| | | ||| | | ||| | | └Action 3 tick 3
+|| | | ||| | | ||| | └Action 3 tick 2
+|| | | ||| | | ||| └Action 3 tick 1
+|| | | ||| | | ||└Action 2 tick 4
+|| | | ||| | | |└Obs 3 created using this state, action 3 decided using obs 3
+|| | | ||| | | └Action 2 tick 3
+|| | | ||| | └Action 2 tick 2
+|| | | ||| └Action 2 tick 1
+|| | | ||└Action 1 tick 4
+|| | | |└Obs 2 created using this state, action 2 decided using obs 2
+|| | | └Action 1 tick 3
+|| | └Action 1 tick 2
+|| └Action 1 tick 1
+|└"void" state transition where sim is stepped without any cars' controls being set
+└Reset occurs, obs 1 created using this state, action 1 decided using obs 1
+```
+### standard_map
+Boolean - default `true`. This option is passed to [rlgym-compat](https://github.com/JPK314/rlgym-compat)'s GameState in order to configure boost pad ordering.
+
+### sim_extra_info
+Boolean - default `false`. This option determines whether rlgym-compat's [SimExtraInfo](https://github.com/JPK314/rlgym-compat/blob/main/rlgym_compat/sim_extra_info.py) is used for extra information that otherwise cannot be derived from the RLBot GamePacket.
+
+### agent_ids_fn
+Optional Callable - default `None`. This option determines how rlgym-compat assigns `AgentID`s to players when creating the compat GameState from the RLBot v5's `flat.GamePacket`. The input is a `GamePacket` and the output is a map from `PlayerInfo.player_id` to your desired `AgentID` data type and value for your config objects. Normally this would be handled by your `StateMutator`, but that functionality is not implemented yet and this acts as a simple override without the other potential `StateMutator` behavior.
+
+
+## What does "accurate" mean? What is this wrapper doing?
+This wrapper attempts to replicate the training conditions of your bot in RLGym as accurately as possible. This means a lot of consideration is put into tick-perfectly matching everything where possible while maintaining the flow and functionality expected by arbitrary config objects and by proxy your model during training using RLGym v2.
+
+### Tick perfectly matching actions
+This wrapper assumes that your computer is running Rocket League in such a way that you have enough time for your actions to always get submitted for use between the latest received packet's physics frame and the next physics frame. The optimal conditions for this are when you play using a capped frame rate that is either 120 or 240, because these frame rates have the largest windows between when the wrapper receives a packet from RLBot Core and when you can no longer submit an action and have it be used during the state transition between the latest received packet's physics tick and the next physics tick. This window is about 6 milliseconds on average with a small standard deviation when the framerate is capped to 120 or 240 (NOT 360!), and is about 4 milliseconds on average with a much wider standard deviation when left uncapped. Other frame rates do very poorly and are not recommended. Possibly if you modify the command line argument RLBot passes for packet send rate from 240 to another number you may be able to get up to a 6 millisecond window using another frame rate, but it's untested and annoying to configure. Just use 120 or 240 capped, or hope for the best uncapped.
+
+### On-the-fly resets and countdown actions
+This wrapper tries to tick perfectly match how your config objects and model interact with each other and the sim in RLGym v2. However, there are some situations where this is just not possible. One such situation is when an on-the-fly reset is performed. These can happen if your `decide_*_recovery_style` methods return `RESET` or if you stop taking a hard-coded action. Due to an implementation detail, in RLGym v2's RocketSimEngine with the default constructor parameter (`rlbot_delay=True`), after resetting, all agents will do one tick of an all-zeros action before beginning the action generated due to the reset state's observation. In the language of this wrapper, this means that after resetting, if you have `offset < 0` and `relative_to = StepOffsetRelativeTo.ACTION_END`, in theory you should take `offset` ticks of the zero action in order to perfectly match how everything works during training. This is obviously not desirable against an opponent with no such restrictions, and so instead the observation is generated using the game state from `offset` ticks ago, and the fact that a non-all-zeros action was taken for the ticks since that point until this new action is due to start (this tick) is assumed to come out in the wash.
+
+Due to the duration of the Countdown phase being unreliable, it is impossible to perfectly time the reset so that you start the action as soon as Countdown ends (your actions start getting used on the state transition between the last packet of Countdown and the first packet of Kickoff). During the Countdown match phase, the cars settle onto the ground and cannot move (wheel wiggling is purely cosmetic), so the difference between using the proper tick of Countdown and some older tick of countdown is essentially zero. This wrapper uses the 440th (or next available after) tick after the Countdown phase began as the tick on which to generate the observation that will be used once Countdown ends, and simply sends the first ControllerState of this action every tick from then on until the latest packet is Kickoff instead of Countdown. This should work essentially perfectly UNLESS you intend to jump on the first tick. Because the ControllerState is sent repeatedly during Countdown, the game will think you are just holding jump and so once you can actually move you won't jump, you'll just be holding jump. So... don't do that!