Page MenuHomeWildfire Games

Fix rP21480 Shadow problems
ClosedPublic

Authored by Stan on Mar 15 2018, 10:04 AM.

Details

Summary

Since floating point can also be negative, and in this case they are the computations needs to be done before casting to u8 because else it messes the result of clamp.
This patch maintains the warning fix, while fixing the computation.

Test Plan

Notice the Danubius map lighting is currently broken (Nice map btw !)
Notice it's not anymore when using this patch.

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 5558
Build 9368: Vulcan BuildJenkins

Event Timeline

Stan created this revision.Mar 15 2018, 10:04 AM
Vulcan added a subscriber: Vulcan.Mar 15 2018, 10:57 AM

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/248/display/redirect

because else it messes the result of clamp

I recommend to qualify that statement by explaining why the current code yields a wrong result, so that you don't have to rely on testing anymore but can read similar code and find errors by interpreting the code.

Stan added a comment.EditedMar 15 2018, 2:40 PM

Minimal test case to understand the issue.

// ClampTest.cpp : Defines the entry point for the console application.
//

#include "stdafx.h"
#include <algorithm>
#include <limits>
#include <iostream>
#include <functional>
#include <cassert>

template<class T, class Compare>
constexpr const T& clamp(const T& v, const T& lo, const T& hi, Compare comp)
{
	return assert(!comp(hi, lo)), comp(v, lo) ? lo : comp(hi, v) ? hi : v;
}

template<class T>
constexpr const T& clamp(const T& v, const T& lo, const T& hi)
{
	return clamp(v, lo, hi, std::less<T>());
}

int main()
{
	float maxColorValueFloat = std::numeric_limits<uint8_t>::max();
	float minColorValueInt = std::numeric_limits<uint8_t>::min();
	uint8_t maxColorValueU8 = std::numeric_limits<uint8_t>::max();
	uint8_t minColorValueU8 = std::numeric_limits<uint8_t>::min();
	float dotProduct = -0.5;
	// Fix
	uint8_t colorComponentValueFromFloat = static_cast<uint8_t>(clamp(dotProduct * maxColorValueFloat, minColorValueInt, maxColorValueFloat));
	// Current
	uint8_t colorComponentValueFromU8 = clamp(static_cast<uint8_t>(dotProduct * maxColorValueU8), minColorValueU8, maxColorValueU8);

	std::cout << +colorComponentValueFromFloat << std::endl; // 0
	std::cout << +colorComponentValueFromU8 << std::endl; // 129

	std::getchar();
    return 0;
}

I'm sorry I guess I should have explained better in the documentation. Basically what happens is this.
You get a float that's let's -0.5 * 255 = - 127.5 that get casted into a unsigned char (u8, uint8_t, whatever)
so that's 255 - 127 = 129.

0 < 129 < 255 so it takes 129 .

While if you cast later:

-127.5 < 0 < 255 so it takes 0

hence the breakage.

-127.5 that get casted into a unsigned char (u8, uint8_t, whatever)
so that's 255 - 127 = 129.

How do we get to the latter equation?

Stan added a comment.EditedMar 15 2018, 5:44 PM

That's just unsigned signed difference. Just like 1111 1111 = 255 in unsigned mode and - 127 in signed mode

vladislavbelov accepted this revision.Mar 15 2018, 5:47 PM
vladislavbelov added a subscriber: vladislavbelov.
vladislavbelov added inline comments.
source/graphics/LightEnv.h
131

Add .f to the 255:

clamp(dot * 255.f, 0.f, 255.f)

To avoid warnings/errors in case when the dot type will be changed.

This revision is now accepted and ready to land.Mar 15 2018, 5:47 PM

so that's 255 - 127 = 129

If those are decimal numbers, then 255 - 127 = 128
I don't think casting results in a subtraction.

Stan added a comment.Mar 15 2018, 6:07 PM

Depends on the cast. It's not really a substraction it's bit reinterpretation.

@Vladislav sure :)

Stan updated this revision to Diff 6175.Mar 15 2018, 6:24 PM

Add a float suffix.

reinterpretation

That sounds better.

And how do we know it reinterprets the bits rather than chosing the smallest possible number (which was likely expected when writing the original line)?

It's this?
http://www.open-std.org/jtc1/sc22/open/n2356/conv.html

4.7 Integral conversions [conv.integral]
2 If the destination type is unsigned, the resulting value is the least

unsigned integer congruent to the source integer (modulo 2n where n is
the  number of bits used to represent the unsigned type).  [Note: In a
two's complement representation, this  conversion  is  conceptual  and
there is no change in the bit pattern (if there is no truncation).  ]

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/249/display/redirect

Stan added a comment.EditedMar 15 2018, 6:41 PM

4.9 Floating-integral conversions [conv.fpint]

1 An rvalue of a floating point type can be converted to an rvalue of an integer type. The conversion truncates; that is, the fractional part

is discarded.  The behavior is undefined if the truncated value cannot
be represented in the destination type.

Maybe this.
I haven't done that stuff in some time

elexis accepted this revision.Mar 15 2018, 7:14 PM

Patch reads good, tested and works as expected.

@Stan please close this (rP21557)
(This time it failed because it says "Revisions" instead of "Revision")

Stan closed this revision.Mar 18 2018, 7:06 PM

Fixed by rP21557