Merge from https://github.com/Dunedan/XpartaMuPP
This code is based from r19467 .
The following revisions are lost because of this:
r21927 | 2018-11-07 12:32:21 -0500 (Wed, 07 Nov 2018) The files in this path were not marked as moved in rP21926...
r21926 | Nov 7, 2018, 6:23:56 PM Split XpartaMuPP and EcheLOn into separate directories and rename parent folder to "lobbybots" following rP18609.It should be that the below commits are now merged into this patch.
r22460 | 2019-07-12 13:40:40 -0400 (Fri, 12 Jul 2019) Fix lineendings. (partially lost)
r21924 | 2018-11-07 12:31:01 -0100 Support connecting the lobby bots without TLS errors if the server does not devlier a valid, non-selfsigned certificate.
NOTICE:
- The database url can be specified now. This enables the use of a different database engine. Recently the ratings database was migrated to a MySQL(mariaDB) database. This has been tested and there appears to be the need for some changes to get this working without issue. (#5634)
- The README.md from this git repo was not included. It should be a merged with our version.
- This changes the directory structure and file naming convention.
I found at least the following patch that would need to be changed after this:
Changes
General Improvements:
Up until now XpartaMuPP was able to relay requests meant for EcheLOn to EcheLOn. As 0ad now sends such requests directly to EcheLOn we don't need that functionality anymore, which makes the code way more readable. Additionally this closes a vulnerability which allowed clients to manipulate the ratings of other players by submitting requests directly to EcheLOn, but including the in the sent XML, which normally only XpartaMuPP did for requests forwarded to EcheLOn.
It turns out the XEP-0045 plugin for SleekXMPP already stores information about all participants in a MUC room. So instead of collecting and storing this information in the bots again, simply rely on the information provided by the plugin. This reduces the required memory for the additional storage of nick-JID-relations in the bots and reduces the code complexity.
Previously EcheLOn was assumed offline if he wasn't present in the MUC nick list maintained by XpartaMuPP. This was prone to race conditions on EcheLOn joining or leaving the room. Now instead of assuming if EcheLOn is online or not a IQ query is sent in every case and in case EcheLOn isn't online the error response is handled appropriately. Additionally this commit sends responses back to the 0ad clients in case EcheLOn is offline to better comply with the request/response model of IQ queries. These responses are currently empty result responses, as 0ad shows nasty error messages in the chat window for error responses.
For all requests made from clients to XpartaMuPP, which get forwarded to EcheLOn, return the correct stanza id to the client. This is implemented using a size limited dictionary for the mapping of ids between clients and XpartaMuPP and ids between XpartaMuPP and EcheLOn so ids without a response from EcheLOn get evicted over time while avoiding an ever growing memory consumption of XpartaMuPP in such a case.
- Process events in the main event loop
- Use the send queue when sending messages
- Separate usage of JIDs and nicks
- Let the bots answer with meaningful messages
As we have two separate bots with clearly defined responsibilities now, it adds a nice touch to let them tell what their purpose is, when they're asked. As part of that "Ratings" got a nick change to "RatingsBot" to not trigger the message accidentally when talking about ratings and "WFGbot" got a nick change to "WFGBot" to be in line with "RatingsBot" in terms of capitalization.
Using callbacks has the advantages, that only expected iq results will be processed and not all iq results which might happen to find its way to XpartaMuPP.
Remove the checks if player is in the room, when sending a response, as there is proper error handling for the case that the player isn't online anymore.
Previously XpartaMuPP relayed information of players coming online to EcheLOn. As EcheLOn has to be part of the lobby-MUC as well (e.g. to post game results) it can watch and handle this information without having to rely on XpartaMuPP.
The gamelist stanza is now created only once when sending it to all players instead of once per player.
Major Fixes:
At some point sleekxmpp seemed to have changed the data structure or iq.plugins, making it incompatible with xpartamupp. This commit removes the affected pieces, which weren't necessary anyway.
Minor Fixes:
- Remove unused attribute
- Fix stanza ids when broadcasting gamelists
- Limit the max amount of open games to 128
XpartaMuPP was vulnerable to DoS, by opening an infinitive amount of games, which causes the responses to occupy all available memory and make the lobby unusable, because the responses become too big and to compute intensive. Even 128 might be still too high for a useable lobby, but at least it's not possible anymore for clients to use it to fill up all available memory. The proper fix for all related performance problems is PubSub, which will be introduced later.
- Fix a small log formatting bug
- Prevent use of non-0ad resources
- [[ https://github.com/Dunedan/XpartaMuPP/commit/398eda1706893e6db5bc26cd2e81deaf1019d41f | Remove deprecated threaded-parameter ]]
The threaded-parameter for SleekXMPP got deprecated several years ago, probably because it got replaced by the block-parameter. In any case its name was misleading, as all it does is to control whether the event dispatcher will run in a separate thread or not and not if threads are used at all or not (hint: they are).
- Use XMPP Pings instead of "whitespace pings"
- Fix module for sleekxmpp.jid.JID
- Provide proper variable types for calling methods
- Nicer variable interpolation for log statements
Code Style/Smells:
- Make it a proper Python package
- First round of Python style fixes
- Move common stanzas into a separate module
- Move PlayerXmppPlugin to stanzas
Also remove PlayerXmppPlugin from echelon, as it apparently wasn't used there.
- Fix some smaller smells
- Some code style improvements
- Some style fixes
- Clean up iq handling
- Mark internal methods as internal
- Split iq handler functions
Reduces the complexity of the iq handling code.
Following "explicit is better than implicit". Also uses loggings functionality for printing stacktraces instead of using the traceback module.
- Improve structure of relay methods
- Rename GamesList to Games
- Rename LeaderboardList to Leaderboard
- Inline _relay_rating_list_request
Database:
- Make database initialization non-global
- Add cmd script for database operations
- Use thread-local sessions for SQLAlchemy
- Add the option to specify a database url
Allows the use of alternative file locations for the SQLite database as well as completely
Tests:
Docstrings:
- Add module level docstrings
- Improve some docstrings
- Improved docstrings for XpartaMuPP
- Improve docstrings
- Improve docstrings
Ensure all methods have proper docstrings including the notion of arguments and returned values, although those aren't perfectly documented yet.
Textual:
- double quotes for every human readable message - single quotes for everything else - exception if something in the quoted string would need quoting