//Since this moves and renames files and folders, it becomes hard to see the changes being made and compare the 'before and after'. [[ https://code.wildfiregames.com/D2630?vs=on&id=15804#C185201OL1 | Follow this link to more easily view the 'before and after' changes side by side. ]]//
Merge from https://github.com/Dunedan/XpartaMuPP
This code is based from [[ https://trac.wildfiregames.com/browser/ps/trunk/source/tools/XpartaMuPP?rev=19467 | r19467 ]] .
The following revisions are lost because of this:
> [[ https://trac.wildfiregames.com/changeset/21927/ | r21927 | 2018-11-07 12:32:21 -0500 (Wed, 07 Nov 2018) ]] The files in this path were not marked as moved in rP21926...
> [[ https://trac.wildfiregames.com/changeset/21926/ | r21926 | Nov 7, 2018, 6:23:56 PM ]] Split XpartaMuPP and EcheLOn into separate directories and rename parent folder to "lobbybots" following rP18609.
> **It should be that the below commits are now merged into this patch.**
> ~~[[ https://trac.wildfiregames.com/changeset/22460/ | r22460 | 2019-07-12 13:40:40 -0400 (Fri, 12 Jul 2019) ]] Fix lineendings. (partially lost)
> [[ https://trac.wildfiregames.com/changeset/21924/ | r21924 | 2018-11-07 12:31:01 -0100 ]] Support connecting the lobby bots without TLS errors if the server does not devlier a valid, non-selfsigned certificate.~~
- The database url can be specified now. This enables the use of a different database engine. Recently the ratings database was migrated to a MySQL(mariaDB) database. This has been tested and there appears to be the need for some changes to get this working without issue. (#5634)
- The README.md from this git repo was not included. It should be a merged with our version.
- This changes the directory structure and file naming convention.
I found at least the following patch that would need to be changed after this:
- [[ https://github.com/Dunedan/XpartaMuPP/commit/014f52bc4d8f36fc570d3139b4fd90f54c072e47 | Initial commit with code from SVN ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/aa7b7ed2c97c0f318d30eb874fc7f6f6d8334a22 | Untangle XpartaMuPP and EcheLOn ]]
> Up until now XpartaMuPP was able to relay requests meant for EcheLOn to EcheLOn. As 0ad now sends such requests directly to EcheLOn we don't need that functionality anymore, which makes the code way more readable. Additionally this closes a vulnerability which allowed clients to manipulate the ratings of other players by submitting requests directly to EcheLOn, but including the in the sent XML, which normally only XpartaMuPP did for requests forwarded to EcheLOn.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/597b7960efbb4ddfd679a4a697ee88162a369be2 | Use MUC-plugin provided participant information ]]
> It turns out the XEP-0045 plugin for SleekXMPP already stores information about all participants in a MUC room. So instead of collecting and storing this information in the bots again, simply rely on the information provided by the plugin. This reduces the required memory for the additional storage of nick-JID-relations in the bots and reduces the code complexity.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/c5beb0c741bb83a64a975d2f4f969b0f6b9d7792 | Change handling for offline EcheLOn ]]
> Previously EcheLOn was assumed offline if he wasn't present in the MUC nick list maintained by XpartaMuPP. This was prone to race conditions on EcheLOn joining or leaving the room. Now instead of assuming if EcheLOn is online or not a IQ query is sent in every case and in case EcheLOn isn't online the error response is handled appropriately. Additionally this commit sends responses back to the 0ad clients in case EcheLOn is offline to better comply with the request/response model of IQ queries. These responses are currently empty result responses, as 0ad shows nasty error messages in the chat window for error responses.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/188b324c11f0bc5563ff4a8bba41700643880b60 | Return correct stanza id for requests to EcheLOn ]]
> For all requests made from clients to XpartaMuPP, which get forwarded to EcheLOn, return the correct stanza id to the client. This is implemented using a size limited dictionary for the mapping of ids between clients and XpartaMuPP and ids between XpartaMuPP and EcheLOn so ids without a response from EcheLOn get evicted over time while avoiding an ever growing memory consumption of XpartaMuPP in such a case.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/f245250ec8e33d55917519cb39b144d0f9001bbe | Process events in the main event loop ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/016c5744bf7161a570fdf2fd5cbb42ecb9853b50 | Use the send queue when sending messages ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/806c4b895206f4673c381ce39a9942d9e19387c5 | Separate usage of JIDs and nicks ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/0a6b833a331dcc6c5e6d01f35f9da446df180f4b | Let the bots answer with meaningful messages ]]
> As we have two separate bots with clearly defined responsibilities now, it adds a nice touch to let them tell what their purpose is, when they're asked. As part of that "Ratings" got a nick change to "RatingsBot" to not trigger the message accidentally when talking about ratings and "WFGbot" got a nick change to "WFGBot" to be in line with "RatingsBot" in terms of capitalization.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/7b39038edb9280eadcf2c7dd4cc51ea35468656f | Use JID objects instead of passing around strings ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/1e6ea63f37ed6481c2d31d6d00a4705851ae3ee4 | Use callbacks for EcheLOn responses ]]
> Using callbacks has the advantages, that only expected iq results will be processed and not all iq results which might happen to find its way to XpartaMuPP.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/55898c720f2b0d101b3e862f46582f469a29b3ac | Don't try to predict if player can get response ]]
> Remove the checks if player is in the room, when sending a response, as there is proper error handling for the case that the player isn't online anymore.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/1c5a3a66d0b018a7c9d69cc024da8f5d1bda2749 | Remove relay of player online information ]]
> Previously XpartaMuPP relayed information of players coming online to EcheLOn. As EcheLOn has to be part of the lobby-MUC as well (e.g. to post game results) it can watch and handle this information without having to rely on XpartaMuPP.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/02ad11d5681bf07ed304daf73e74571d4953a315 | Improve performance for gamelist sending ]]
> The gamelist stanza is now created only once when sending it to all players instead of once per player.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/93f2a38fced76d80eb67609c45c038e587df36dc | Fix handling with newer sleekxmpp versions ]]
> At some point sleekxmpp seemed to have changed the data structure or iq.plugins, making it incompatible with xpartamupp. This commit removes the affected pieces, which weren't necessary anyway.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/0e6a8060843475b2de023dc1eb9b63929e85fa18 | Replace deprecated optparse with argparse ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/6fd9ceb9b5f765bf949de8727bd3d2f95db26096 | Remove unused attribute ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/bcb78e02f97d2aa8121f61a2e2f64637ceb44b52 | Fix stanza ids when broadcasting gamelists ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/cd0d593f96babc728caf6591ba281e73a15d123c | Limit the max amount of open games to 128 ]]
> XpartaMuPP was vulnerable to DoS, by opening an infinitive amount of games, which causes the responses to occupy all available memory and make the lobby unusable, because the responses become too big and to compute intensive. Even 128 might be still too high for a useable lobby, but at least it's not possible anymore for clients to use it to fill up all available memory. The proper fix for all related performance problems is PubSub, which will be introduced later.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/cfa2f30565f35717739b93520030ad0f5ffba38e | Fix a small log formatting bug ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/d064798045999fe3f8e8f315ba0d50d454c73ca1 | Prevent use of non-0ad resources ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/398eda1706893e6db5bc26cd2e81deaf1019d41f | Remove deprecated `threaded`-parameter ]]
> The `threaded`-parameter for SleekXMPP got deprecated several years ago, probably because it got replaced by the `block`-parameter. In any case its name was misleading, as all it does is to control whether the event dispatcher will run in a separate thread or not and not if threads are used at all or not (hint: they are).
- [[ https://github.com/Dunedan/XpartaMuPP/commit/65b2d1e308397083dc5bfe0c6891b64fe5f9f3de | Use XMPP Pings instead of "whitespace pings" ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/1ba2ddbf0cde193bdfb2ebca0c9de3d2d3160dd6 | Fix module for sleekxmpp.jid.JID ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/5f5b1b291605ec70039d35ed7289f00588434c46 | Provide proper variable types for calling methods ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/09c7f0abb3994321c4130a84a70cfe450c5ac4a8 | Nicer variable interpolation for log statements ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/1ae27e082980e09d1a97e65c28e7ee97a5f62bd9 | Make it a proper Python package ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/d76457e45f287560373a4b1a4cdee1954ced2c4f | First round of Python style fixes ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/7a5894604bb296a0aa0af633bf63c30f872bca9d | Move common stanzas into a separate module ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/9b3c59c479fe8e476617711e2789d91efd8ccc9a | Move PlayerXmppPlugin to stanzas ]]
> Also remove PlayerXmppPlugin from echelon, as it apparently wasn't used there.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/5a6073d081c1171aa8fe86bdd5b5830a5fd69054 | Fix some smaller smells ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/ce80bab844e26e5a4cc4f2c03fa5c3ff46cc1552 | Some code style improvements ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/f85e37da5d33c6ce23764265af1b4077b06c2bc7 | Some style fixes ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/cea2b63b695d762b3a17617d6ddf4c9127ad3ed0 | Clean up iq handling ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/ca7b65d80874675a7b607b38e46a5102cfd4d237 | Mark internal methods as internal ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/ce5788cf3a158352abb749b8af22f7bfa3b9afd1 | Split iq handler functions ]]
> Reduces the complexity of the iq handling code.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/16b4ed73b4a590d56acc63be95034dd59efb790e | Replace bare-except with broad-except ]]
> Following "explicit is better than implicit". Also uses loggings functionality for printing stacktraces instead of using the traceback module.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/b02e5a50b4fcab87731e2c5deff693706b8bcb03 | Improve structure of relay methods ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/ae3ce1af8d928d00385a75dc58726c81401c7bbd | Rename GamesList to Games ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/d8e6dd30a4773849d8bc70395b0e324ebe3fe8dc | Rename LeaderboardList to Leaderboard ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/0a77daa6b6d1f2057a40169aa28c5c08646ab30f | Inline _relay_rating_list_request ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/7a02844f1f236c570414f748f05e3c58613caaf5 | Make database initialization non-global ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/6c2c87b320e09f410a26eafe4351e11285be5f00 | Add cmd script for database operations ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/bd80adda4cb22e8606c368cdc06247b5df74374c | Use thread-local sessions for SQLAlchemy ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/652a5868d24a31c878903c19f08840ae299abfa8 | Add the option to specify a database url ]]
> Allows the use of alternative file locations for the SQLite database as well as completely
- [[ https://github.com/Dunedan/XpartaMuPP/commit/e0544197b4161a2e451dbb0ab13bf9dc6af34e7c | Add first unit tests ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/f1858ba4ba9f09182406b450807ee8c27fbfbf73 | Fix failing tests ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/04c1fd97b931d8acd739b5472d3a5fa0250beb25 | Add module level docstrings ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/e9cad9d5b6ad79d5c007db0aa51db926a3f3f84d | Improve some docstrings ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/c69a0fb5704f7fc74e36b82ab6c3981ba45d90df | Improved docstrings for XpartaMuPP ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/482a342d8834574a733326623712f99910cccee8 | Improve docstrings ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/b72609134f4794216e62877574f5e42a1ad26b21 | Improve docstrings ]]
> Ensure all methods have proper docstrings including the notion of arguments and returned values, although those aren't perfectly documented yet.
- [[ https://github.com/Dunedan/XpartaMuPP/commit/bd6707363d0b4119f764a72578e7dd2c21b9dfe9 | Change indentation from 2 spaces to 4 spaces ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/f8bafb8410398a5fd103e5a106e79f2b9feda0bb | Unify quotes ]]
> - double quotes for every human readable message - single quotes for everything else - exception if something in the quoted string would need quoting
- [[ https://github.com/Dunedan/XpartaMuPP/commit/0e2648e6e60b44cae999c75662d96a6e528d5898 | Small wording fix ]]
- [[ https://github.com/Dunedan/XpartaMuPP/commit/6c1908371635769c69a1cf422b82a23abed0b7a1 | Update copyright year ]]