Page MenuHomeWildfire Games

Fix entvalidate.py
ClosedPublic

Authored by baco on Jun 12 2022, 5:42 PM.

Details

Summary

entvalidate.py, companion to checkrefs.py, is currently broken in several ways:

  • It doesn't properly handle mixins / special templates
  • On python 3.8+, XML elements are no longer sorted and this fails validation against entity.rng
  • mixins are not properly applied because the | split is incorrect (3 max items instead of 2)
  • merge="" is not supported
  • Parents aren't always passed when they should be.

This fixes all these issues.

Test Plan

Run entvalidate.py

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

wraitii created this revision.Jun 12 2022, 5:42 PM

Successful build - Chance fights ever on the side of the prudent.

builderr-debug-macos.txt
ld: warning: text-based stub file /System/Library/Frameworks//CoreAudio.framework/CoreAudio.tbd and library file /System/Library/Frameworks//CoreAudio.framework/CoreAudio are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//AudioToolbox.framework/AudioToolbox.tbd and library file /System/Library/Frameworks//AudioToolbox.framework/AudioToolbox are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ForceFeedback.framework/ForceFeedback.tbd and library file /System/Library/Frameworks//ForceFeedback.framework/ForceFeedback are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreVideo.framework/CoreVideo.tbd and library file /System/Library/Frameworks//CoreVideo.framework/CoreVideo are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Cocoa.framework/Cocoa.tbd and library file /System/Library/Frameworks//Cocoa.framework/Cocoa are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//IOKit.framework/IOKit.tbd and library file /System/Library/Frameworks//IOKit.framework/IOKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//QuartzCore.framework/QuartzCore.tbd and library file /System/Library/Frameworks//QuartzCore.framework/QuartzCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Metal.framework/Metal.tbd and library file /System/Library/Frameworks//Metal.framework/Metal are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Security.framework/Security.tbd and library file /System/Library/Frameworks//Security.framework/Security are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//OpenAL.framework/OpenAL.tbd and library file /System/Library/Frameworks//OpenAL.framework/OpenAL are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//AppKit.framework/Versions/C/AppKit.tbd and library file /System/Library/Frameworks//AppKit.framework/Versions/C/AppKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreData.framework/Versions/A/CoreData.tbd and library file /System/Library/Frameworks//CoreData.framework/Versions/A/CoreData are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ApplicationServices.framework/Versions/A/ApplicationServices.tbd and library file /System/Library/Frameworks//ApplicationServices.framework/Versions/A/ApplicationServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreImage.framework/Versions/A/CoreImage.tbd and library file /System/Library/Frameworks//CoreImage.framework/Versions/A/CoreImage are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation.tbd and library file /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Foundation.framework/Versions/C/Foundation.tbd and library file /System/Library/Frameworks//Foundation.framework/Versions/C/Foundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreGraphics.framework/Versions/A/CoreGraphics.tbd and library file /System/Library/Frameworks//CoreGraphics.framework/Versions/A/CoreGraphics are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreText.framework/Versions/A/CoreText.tbd and library file /System/Library/Frameworks//CoreText.framework/Versions/A/CoreText are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ImageIO.framework/Versions/A/ImageIO.tbd and library file /System/Library/Frameworks//ImageIO.framework/Versions/A/ImageIO are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ColorSync.framework/Versions/A/ColorSync.tbd and library file /System/Library/Frameworks//ColorSync.framework/Versions/A/ColorSync are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSyncLegacy.framework/Versions/A/ColorSyncLegacy.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSyncLegacy.framework/Versions/A/ColorSyncLegacy are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreServices.framework/Versions/A/CoreServices.tbd and library file /System/Library/Frameworks//CoreServices.framework/Versions/A/CoreServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/Versions/A/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/Versions/A/CoreFoundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CFNetwork.framework/Versions/A/CFNetwork.tbd and library file /System/Library/Frameworks//CFNetwork.framework/Versions/A/CFNetwork are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList are out of sync. Falling back to library file for linking.
ld: warning: object file (../../../libraries/osx/fmt/lib/libfmt.a(format.cc.o)) was built for newer OSX version (10.13) than being linked (10.12)
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation are out of sync. Falling back to library file for linking.
builderr-release-macos.txt
ld: warning: text-based stub file /System/Library/Frameworks//CoreAudio.framework/CoreAudio.tbd and library file /System/Library/Frameworks//CoreAudio.framework/CoreAudio are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//AudioToolbox.framework/AudioToolbox.tbd and library file /System/Library/Frameworks//AudioToolbox.framework/AudioToolbox are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ForceFeedback.framework/ForceFeedback.tbd and library file /System/Library/Frameworks//ForceFeedback.framework/ForceFeedback are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreVideo.framework/CoreVideo.tbd and library file /System/Library/Frameworks//CoreVideo.framework/CoreVideo are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Cocoa.framework/Cocoa.tbd and library file /System/Library/Frameworks//Cocoa.framework/Cocoa are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//IOKit.framework/IOKit.tbd and library file /System/Library/Frameworks//IOKit.framework/IOKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//QuartzCore.framework/QuartzCore.tbd and library file /System/Library/Frameworks//QuartzCore.framework/QuartzCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Metal.framework/Metal.tbd and library file /System/Library/Frameworks//Metal.framework/Metal are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Security.framework/Security.tbd and library file /System/Library/Frameworks//Security.framework/Security are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//OpenAL.framework/OpenAL.tbd and library file /System/Library/Frameworks//OpenAL.framework/OpenAL are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//AppKit.framework/Versions/C/AppKit.tbd and library file /System/Library/Frameworks//AppKit.framework/Versions/C/AppKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreData.framework/Versions/A/CoreData.tbd and library file /System/Library/Frameworks//CoreData.framework/Versions/A/CoreData are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ApplicationServices.framework/Versions/A/ApplicationServices.tbd and library file /System/Library/Frameworks//ApplicationServices.framework/Versions/A/ApplicationServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition.tbd and library file /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreImage.framework/Versions/A/CoreImage.tbd and library file /System/Library/Frameworks//CoreImage.framework/Versions/A/CoreImage are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation.tbd and library file /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//Foundation.framework/Versions/C/Foundation.tbd and library file /System/Library/Frameworks//Foundation.framework/Versions/C/Foundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreGraphics.framework/Versions/A/CoreGraphics.tbd and library file /System/Library/Frameworks//CoreGraphics.framework/Versions/A/CoreGraphics are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreText.framework/Versions/A/CoreText.tbd and library file /System/Library/Frameworks//CoreText.framework/Versions/A/CoreText are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ImageIO.framework/Versions/A/ImageIO.tbd and library file /System/Library/Frameworks//ImageIO.framework/Versions/A/ImageIO are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//ColorSync.framework/Versions/A/ColorSync.tbd and library file /System/Library/Frameworks//ColorSync.framework/Versions/A/ColorSync are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSyncLegacy.framework/Versions/A/ColorSyncLegacy.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSyncLegacy.framework/Versions/A/ColorSyncLegacy are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreServices.framework/Versions/A/CoreServices.tbd and library file /System/Library/Frameworks//CoreServices.framework/Versions/A/CoreServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis.tbd and library file /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CFNetwork.framework/Versions/A/CFNetwork.tbd and library file /System/Library/Frameworks//CFNetwork.framework/Versions/A/CFNetwork are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/Versions/A/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/Versions/A/CoreFoundation are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices are out of sync. Falling back to library file for linking.
ld: warning: text-based stub file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList.tbd and library file /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList are out of sync. Falling back to library file for linking.
ld: warning: object file (../../../libraries/osx/fmt/lib/libfmt.a(format.cc.o)) was built for newer OSX version (10.13) than being linked (10.12)
ld: warning: text-based stub file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation.tbd and library file /System/Library/Frameworks//CoreFoundation.framework/CoreFoundation are out of sync. Falling back to library file for linking.

Link to build: https://jenkins.wildfiregames.com/job/macos-differential/6148/display/redirect

wraitii published this revision for review.Jun 13 2022, 9:51 AM
Stan added a subscriber: Stan.Jun 13 2022, 10:22 AM
Stan added inline comments.
source/tools/entity/entvalidate.py
22

This was necessary to print to STDErr.

Run entvalidate.py

When removing the dot in front of scriptlib it works, but fails a bit later...

entvalidate.py
from scriptlib import SimulTemplateEntity, find_files
Terminal Output
cd binaries/system/
❯ ./pyrogenesis -dumpSchema
TIMER| InitVfs: 1.706 ms
FILES| Main log written to '/Users/paria/Library/Application Support/0ad/logs/mainlog.html'
FILES| Interesting log written to '/Users/paria/Library/Application Support/0ad/logs/interestinglog.html'
TIMER| CONFIG_Init: 5.076 ms
Generated entity.rng
❯ cd ~/0ad/source/tools/entity/
❯ python3 entvalidate.py
Relax NG schema non existant.
Please create the file scripts/entity.rng
You can do that by running 'pyrogenesis -dumpSchema' in the 'system' directory
❯ git status --short
 M entvalidate.py
?? ../../../binaries/system/entity.rng

❯ bat ../../../binaries/system/entity.rng
───────┬────────────────────────────────────────────────────────────────────────
       │ File: ../../../binaries/system/entity.rng
───────┼────────────────────────────────────────────────────────────────────────
   1   │ <grammar xmlns='http://relaxng.org/ns/structure/1.0' xmlns:a='http://ns
       │ .wildfiregames.com/entity' datatypeLibrary='http://www.w3.org/2001/XMLS
       │ chema-datatypes'><define name='decimal'><data type='decimal'/></define>
       │ <define name='nonNegativeDecimal'><data type='decimal'><param name='min
       │ Inclusive'>0</param></data></define><define name='positiveDecimal'><dat
       │ a type='decimal'><param name='minExclusive'>0</param></data></define><d
       │ efine name='anything'><zeroOrMore><choice> 
and so on...
wraitii added inline comments.Jun 14 2022, 9:34 AM
source/tools/entity/entvalidate.py
22

warn is no longer in scriptlib though. I'll update

Stan added a comment.EditedJun 26 2022, 6:06 PM
#!/usr/bin/env python3
# -*- mode: python-mode; python-indent-offset: 4; -*-
# SPDX-License-Identifier: GPL-2.0-or-later
# SPDX-FileCopyrightText: © 2022 Wildfire Games

from argparse import ArgumentParser
from logging import WARNING, getLogger, StreamHandler, INFO, Formatter, Filter
from os import chdir
from sys import stdout, stderr
from pathlib import Path
from scriptlib import SimulTemplateEntity, find_files
from subprocess import run, CalledProcessError
from xml.etree import ElementTree

class SingleLevelFilter(Filter):
    def __init__(self, passlevel, reject):
        self.passlevel = passlevel
        self.reject = reject

    def filter(self, record):
        if self.reject:
            return (record.levelno != self.passlevel)
        else:
            return (record.levelno == self.passlevel)

class EntityValidator:
    def __init__(self, vfs_root, rng_root, mods=None, paths = None, verbose=False):
        if mods is None:
            mods = ['public']

        self.mods = mods
        self.verbose = verbose
        self.paths = paths
        self.__init_logger
        self.vfs_root = Path(vfs_root)
        self.relaxng_schema = Path(rng_root)
        self.simul_templates_path = Path('simulation/templates')
        if not self.relaxng_schema.exists():
            self.logger.error(f"""Relax NG schema non existant.
                        Please create the file {self.relaxng_schema.relative_to(self.root)}
                        You can do that by running 'pyrogenesis -dumpSchema' in the 'system' directory""")
            return
        if run(['xmllint', '--version'], capture_output=True).returncode != 0:
            self.logger.error("xmllint not found in your PATH, please install it (usually in libxml2 package)")
            return


    @property
    def __init_logger(self):
        logger = getLogger(__name__)
        logger.setLevel(INFO)
        # create a console handler, seems nicer to Windows and for future uses
        ch = StreamHandler(stdout)
        ch.setLevel(INFO)
        ch.setFormatter(Formatter('%(levelname)s - %(message)s'))
        f1 = SingleLevelFilter(INFO, False)
        ch.addFilter(f1)
        logger.addHandler(ch)
        errorch = StreamHandler(stderr)
        errorch.setLevel(WARNING)
        errorch.setFormatter(Formatter('%(levelname)s - %(message)s'))
        logger.addHandler(errorch)
        self.logger = logger


    def process_templates(self, sorted_files, simul_template_entity, mod):
        count = 0
        failed = 0
        for fp, _  in sorted_files:
            if fp.stem.startswith('template_'):
                continue
            path = fp.as_posix()
            if path.startswith('simulation/templates/mixins/') or path.startswith("simulation/templates/special/"):
                continue

            if self.verbose:
                self.logger.info(f"{fp}...")

            count += 1
            entity = simul_template_entity.load_inherited(self.simul_templates_path, str(fp.relative_to(self.simul_templates_path)), [mod])
            xmlcontent = ElementTree.tostring(entity, encoding='utf-8').decode("utf-8")
            relax_ng_path = str(self.relaxng_schema.resolve())
            try:
                run(['xmllint', '--relaxng', relax_ng_path , '-'], input=xmlcontent, encoding='utf-8', capture_output=True, text=True, check=True)
            except CalledProcessError as e:
                failed += 1
                self.logger.error(e.stderr)
                if self.verbose:
                    self.logger.info(e.stdout)

        if self.verbose:
            self.logger.info(f"\nTotal: {count}; failed: {failed}")

    def run(self):
        simul_template_entity = SimulTemplateEntity(self.vfs_root, self.logger)
        if self.paths is not None:
            self.process_templates(sorted([(Path(p), None) for p in self.paths]), simul_template_entity, self.mods[0])
        else:
            for mod in self.mods:
                templates = find_files(self.vfs_root, [mod], self.simul_templates_path, 'xml')
                self.process_templates(templates, simul_template_entity, mod)


if __name__ == '__main__':
    ap = ArgumentParser(description="Checks the game's files invalid templates.")
    ap.add_argument('-r', '--root', action='store', dest='root', default=Path(__file__).resolve().parents[3]  / 'binaries' / 'data' / 'mods')
    ap.add_argument('-s', '--schema', action='store', dest='schema', default=Path(__file__).resolve().parents[3] / 'binaries' / 'system' / 'entity.rng')
    ap.add_argument('-m', '--mods', action='store', dest='mods', default='public')
    ap.add_argument('-p', '--path', nargs='*', help='Optionally, a list of templates to validate.')
    ap.add_argument('-v', '--verbose', action='store_true', default=False)
    args = ap.parse_args()
    validator = EntityValidator(args.root, args.schema, args.mods.split(','), args.path, args.verbose)
    validator.run()

Made an alternative version to match more the other versions we have. it works fine for the public mod, but not for other mods

EDIT: Also have to run

pyrogenesis -dumpSchema -mod=mod -mod=public

Else you get other errors.

baco requested changes to this revision.Sep 4 2022, 11:24 PM
baco added a subscriber: baco.

On lines 23 and 25 of the patched file, there is a call to the sys' module exit() function. That usage is discouraged because redefines the exit built-in in Python's interpreter when importing the module containing such import from the interactive Python interpreter. The usual way to handle exit statuses is to make the main function to return those values (def main() -> int) as integer values and handle them in the calling of the main function like so:

if __name__ == "__main__":
    raise SystemExit(main())

note that SystemExit is a handler for those cases and doesn't need an import statement. When SystemExit receives an integer value other than 0 it exists the Python interpreter and elevates the return value. That way you could import main in other modules (for instanance in testing) and call it without causing the interpreter to exit in case of failure.

As @Langbart pointed out, having relative imports in python works if the module is called as such, (with the -m flag to the Python interpreter). For instance it would work if calling the module as $ python3 -m entity.entvalidate, but will fail if trying to call it as a script (./entvalidate.py). The way of using it should be documented, as it is for checkrefs.py.

There is another module as well that was imported and never used, os.chdir, you could remove that import.

There are some PEP8 infringements. I don't know what's the policy of the project on Python modules, but a call to falke8 or pylint could list them all. Moreover on style, there are mixed single-quoted strings and double-quoted strings; the recommendation here is to only change the quotation when it is needed for readability. According to the pre-existing code in the module, single-quoted strings was adopted (with the exception of multi-line strings which are always quoted in double-quotes)

source/tools/entity/entvalidate.py
22

This could be done calling print with an extra argument, file:

print(f"""...""",
      file=sys.stderr)
This revision now requires changes to proceed.Sep 4 2022, 11:24 PM
baco commandeered this revision.Sep 11 2022, 8:35 PM
baco edited reviewers, added: wraitii; removed: baco.
This revision now requires review to proceed.Sep 11 2022, 8:35 PM
baco updated this revision to Diff 20901.Sep 11 2022, 8:36 PM
baco updated this revision to Diff 20902.Sep 11 2022, 8:58 PM

Added a new diff with the flag -U3000 as suggested on IRC.

I haven't changed anything regarding to coding style because it is not defined anything yet for the project, and would have been a lot of changes in just one diff. That work remains pending for future interventions in Python modules.

baco updated this revision to Diff 20904.Sep 13 2022, 12:41 AM

It's been a complete overhaul of the file, but it is working. Now it receives the modules' location and name as parameters.

@Stan I attach all the commits separated (14 of them) in a zipfile if you wish to apply the patches one-by-one, and leave me as author of the commits as well ;) :P

Stan added a comment.EditedSep 13 2022, 8:28 AM

~~ Sadly I'm not sure I can set you as author. There seem to be a way but I dunno if trac and phab support it https://stackoverflow.com/questions/2107719/how-to-set-author-for-svn#2107898 ~~

EDIT: Seems it's only for files.

source/tools/entity/entvalidate.py
30

Why scripts ?

baco added inline comments.Sep 13 2022, 1:22 PM
source/tools/entity/entvalidate.py
30

It was that way before and I preserved that. I thought the intention there was to keep files external to the repo, in some way, isolated.

Do you prefer another location or no directory at all?

Stan added inline comments.Sep 13 2022, 1:29 PM
source/tools/entity/entvalidate.py
30

Sorry I missed that. I'd prefer nothing at all, or a way to set the location.

baco added a comment.Sep 13 2022, 1:29 PM
In D4698#203928, @Stan wrote:

~~ Sadly I'm not sure I can set you as author. There seem to be a way but I dunno if trac and phab support it https://stackoverflow.com/questions/2107719/how-to-set-author-for-svn#2107898 ~~

EDIT: Seems it's only for files.

It doesn't seems fair to steal credit for the whole file, other people has put their effort, there, also. I am too accustomed the Git-ways this days, hahaha.

baco updated this revision to Diff 20905.Sep 13 2022, 1:39 PM

Deleted the middle directory scripts in entity.rng's location. Didn't made it eligible, by argument, to avoid directory creation in case of not existence.

Stan added a comment.Sep 13 2022, 1:43 PM

Just realised I'm an idiot and you need a way to specify the location because you don't want to put the rng in the mod folder (since you're probably gonna zip it afterwards) and it won't be there on the CI either.
Sorry, that's what happens when I do too much things at once.

Another question, how do you generate that zip?

baco added a comment.Sep 13 2022, 2:15 PM
In D4698#203936, @Stan wrote:

Just realised I'm an idiot and you need a way to specify the location because you don't want to put the rng in the mod folder (since you're probably gonna zip it afterwards) and it won't be there on the CI either.

Ok, I will have it during the day.

Sorry, that's what happens when I do too much things at once.

No prob, it happens :-)

Another question, how do you generate that zip?

git format-patch, is the counter part of git am (am stands for Apply Mail). It is the usual way for Git repos where you have not write permissions over the repo and a list of patches has to be reviewed by an official reviewer (for instance the Linux Kernel).

format-patch generates a bunch of files in email format, one per commit, stating the author as the From in the email, the subject is the first line of the commit message and the rest of the commit message as part of the email body; ending the email body with the diff for that commit.

This way, you can send the patch to the reviewer in the form of the separated commits that gave way to the final patch, so the reviewer (with write access over the repo) can apply all the diffs sequentially preserving the commit-history.

In this case, the generation was done by:

$ git format-patch master..HEAD -o./D4698-fix-entity-validation-module/

and then zipping the resulting directory

and the way to re-apply it, using Git, would be:

$ git am ./D4698-fix-entity-validation-module/*.patch

I know that the main 0AD repo is not Git, just it's mirrors, so it is difficult to do the git-format-patch/git-am dance. But anyways, having changed almost the whole file, it seemed fair to make the transformation visible step-by-step instead of the overwhelming final result.

baco updated this revision to Diff 20906.Sep 13 2022, 10:03 PM

Made Relese NG entity file an optional argument

Stan accepted this revision.EditedSep 13 2022, 10:40 PM

Looks good to go. Wonder if they should have a copyright header. According to https://trac.wildfiregames.com/browser/ps/trunk/source/tools/LICENSE.txt it's MIT.
Won't commit because we reached CF, but will be nice after the release.

This revision is now accepted and ready to land.Sep 13 2022, 10:40 PM
Stan removed a reviewer: wraitii.Sep 13 2022, 10:40 PM
This revision was automatically updated to reflect the committed changes.