This revision adds an RPC interface (using GRPC) which enablesfor training reinforcement learning agents in 0 AD. This also includes a python wrapper for conveniently interacting with 0 AD including setting scenarios and controlling players in lock step.
Notes about remaining features, etc:
- Windows support for premake
- As this adds a new dependency (GRPC), should it be only available as a feature flag? Or should we update the installation instructions? Or should we add a copy of grpc/protobuf to the libraries directory?
- There are currently some outstanding features:
- I have not implemented all the features that may be desireable for configuring scenarios (such as ceasefire durations, custom victory conditions).The revision was originally written using gRPC but has since been updated to use mongoose (greatly simplifying the build process - thanks, The outstanding options are commented out in source/rlinterface/proto/RLAPI.proto.
- The python wrapper includes a wrapper for the game engine actions (in source/tools/clients/python/zero_ad/actions.py).@wraitii!).
An example using this to train RL agents with a variety of different state, I have not added support for all the game engine commands yet and have mostly been using movement and attack commands, myselfaction spaces can be found at https://github.com/brollb/simple-0ad-example.
This being said,**Note**: This does not contain an actual OpenAI gym environment as they require a clearly defined action/state space (which it is not obvious what the optimal representation is - or even if there is a single optimal representation). I wanted to make a revision with what I currently have completed so I could get feedback.That said, It also could be useful to have the foundational components integrated (features in this PR) then add other features in subsequent revisions.
Let me know what you think!it is easy to create a gym using the code in this revision and there are multiple examples in the beforementioned repository. A longer discussion about this can be found at https://github.com/0ad/0ad/pull/25#pullrequestreview-262056969