Since floating point can also be negative, and in this case they are the computations needs to be done before casting to u8 because else it messes the result of clamp.
This patch maintains the warning fix, while fixing the computation.
Details
- Reviewers
elexis vladislavbelov
Notice the Danubius map lighting is currently broken (Nice map btw !)
Notice it's not anymore when using this patch.
Diff Detail
- Repository
- rP 0 A.D. Public Repository
- Lint
Lint Skipped - Unit
Unit Tests Skipped - Build Status
Buildable 5558 Build 9368: Vulcan Build Jenkins
Event Timeline
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/differential/248/display/redirect
because else it messes the result of clamp
I recommend to qualify that statement by explaining why the current code yields a wrong result, so that you don't have to rely on testing anymore but can read similar code and find errors by interpreting the code.
Minimal test case to understand the issue.
// ClampTest.cpp : Defines the entry point for the console application. // #include "stdafx.h" #include <algorithm> #include <limits> #include <iostream> #include <functional> #include <cassert> template<class T, class Compare> constexpr const T& clamp(const T& v, const T& lo, const T& hi, Compare comp) { return assert(!comp(hi, lo)), comp(v, lo) ? lo : comp(hi, v) ? hi : v; } template<class T> constexpr const T& clamp(const T& v, const T& lo, const T& hi) { return clamp(v, lo, hi, std::less<T>()); } int main() { float maxColorValueFloat = std::numeric_limits<uint8_t>::max(); float minColorValueInt = std::numeric_limits<uint8_t>::min(); uint8_t maxColorValueU8 = std::numeric_limits<uint8_t>::max(); uint8_t minColorValueU8 = std::numeric_limits<uint8_t>::min(); float dotProduct = -0.5; // Fix uint8_t colorComponentValueFromFloat = static_cast<uint8_t>(clamp(dotProduct * maxColorValueFloat, minColorValueInt, maxColorValueFloat)); // Current uint8_t colorComponentValueFromU8 = clamp(static_cast<uint8_t>(dotProduct * maxColorValueU8), minColorValueU8, maxColorValueU8); std::cout << +colorComponentValueFromFloat << std::endl; // 0 std::cout << +colorComponentValueFromU8 << std::endl; // 129 std::getchar(); return 0; }
I'm sorry I guess I should have explained better in the documentation. Basically what happens is this.
You get a float that's let's -0.5 * 255 = - 127.5 that get casted into a unsigned char (u8, uint8_t, whatever)
so that's 255 - 127 = 129.
0 < 129 < 255 so it takes 129 .
While if you cast later:
-127.5 < 0 < 255 so it takes 0
hence the breakage.
-127.5 that get casted into a unsigned char (u8, uint8_t, whatever)
so that's 255 - 127 = 129.
How do we get to the latter equation?
That's just unsigned signed difference. Just like 1111 1111 = 255 in unsigned mode and - 127 in signed mode
source/graphics/LightEnv.h | ||
---|---|---|
131 | Add .f to the 255: clamp(dot * 255.f, 0.f, 255.f) To avoid warnings/errors in case when the dot type will be changed. |
so that's 255 - 127 = 129
If those are decimal numbers, then 255 - 127 = 128
I don't think casting results in a subtraction.
Depends on the cast. It's not really a substraction it's bit reinterpretation.
@Vladislav sure :)
reinterpretation
That sounds better.
And how do we know it reinterprets the bits rather than chosing the smallest possible number (which was likely expected when writing the original line)?
It's this?
http://www.open-std.org/jtc1/sc22/open/n2356/conv.html
4.7 Integral conversions [conv.integral]
2 If the destination type is unsigned, the resulting value is the leastunsigned integer congruent to the source integer (modulo 2n where n is the number of bits used to represent the unsigned type). [Note: In a two's complement representation, this conversion is conceptual and there is no change in the bit pattern (if there is no truncation). ]
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/differential/249/display/redirect
4.9 Floating-integral conversions [conv.fpint]
1 An rvalue of a floating point type can be converted to an rvalue of an integer type. The conversion truncates; that is, the fractional part
is discarded. The behavior is undefined if the truncated value cannot be represented in the destination type.
Maybe this.
I haven't done that stuff in some time