Page MenuHomeWildfire Games

Grand Unified Postproc Shader
AbandonedPublic

Authored by aeonios on Apr 16 2018, 5:18 PM.

Details

Summary

This merges the hdr shader and the dof shader into one. Applying both effects is actually not significantly more expensive than applying just one or the other, and there are no conflicts that would otherwise prevent both from being used together.

This also significantly improves both effects, making dof much better and more usable in general and making bloom smoother and more selective in blooming bright areas without drowning out dark areas. It also replaces the 'bloom' box blur with a proper linear sampled gaussian blur, and adds a new setting giving it specific modes for the bloom effect and dof.

Finally this patch will make the postproc shader always be enabled if the user enables postproc in options rather than requiring the map maker to enable it in the specific map being played. Map makers will of course still have artistic control over bloom and color balancing, but there is no good reason to have map makers be able to dictate what effects a user gets when they enable an option under their own client. I haven't removed the postproc setting under atlas because I don't really know my way around that code, but if this patch is merged then that option should also be removed.

Test Plan

Patch and enable postproc. Check to ensure that dof and bloom are working properly.

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Lint Skipped
Unit
Unit Tests Skipped
Build Status
Buildable 6011
Build 10012: Vulcan BuildJenkins

Event Timeline

aeonios created this revision.Apr 16 2018, 5:18 PM

Screenshots!

This is what the current dof shader looks like:


Someone apparently fixed the broken blur shader, but as you can see it's not exactly something that anyone would want to use on purpose.

This is what the current bloom shader looks like:


It's not as bad as the dof shader but it just kind of whitewashes everything indiscriminately.

Here is the new dof in action:



And here's a demonstration of bloom. As you can see it looks much more like what you would expect bloom to look like, properly selecting bright areas and softly expanding them without drowning everything else. DOF is of course also visible and you can see that the effects do not clash at all.


And here you can see the smooth transition to blur with the new dof on a more contiguous map.


Vulcan added a subscriber: Vulcan.Apr 16 2018, 5:40 PM

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/397/display/redirect

Stan added a subscriber: Stan.Apr 16 2018, 6:42 PM
Stan added inline comments.
binaries/data/mods/public/shaders/glsl/bloom.fs
1

Is that the Shader version or the openGl version ?

9

5 sounds like a good constant candidate.

30

I hate to be picky like this but I believe it's missing spaces :)

aeonios updated this revision to Diff 6414.Apr 16 2018, 7:29 PM

Added spaces to the for blocks, converted '5' into a constant.

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/399/display/redirect

vladislavbelov requested changes to this revision.Apr 16 2018, 10:16 PM
vladislavbelov added a subscriber: vladislavbelov.

I like this at all, but I have important notes:

  • Screenshots should be before/after, not just after. Also they all have the pretty same angle, that's not good for any comparison.
  • Blur has a noticeable artefacts (because a simple kernel, I suppose), that didn't present for old blur. So it should be fixed, because we try to not make a regression.
  • It'd be good to have a performance comparison tests for it.
  • Merge HDR & DOF isn't a good idea, because modders/artists may want to enable them one by one, not together. Also it'd be good to rename the HDR to a more correct name.
  • Why it's called big bloom? And why a param is called bigBlur?
  • Fix CC.

Thank you for the patch and for the work on graphic improvements!

binaries/data/mods/public/shaders/glsl/bloom.fs
1
binaries/data/mods/public/shaders/glsl/hdr.fs
25 ↗(On Diff #6414)

We need to minimise a number of add/sub, look at the linearizeDepth in dof.fs.

source/graphics/MapReader.cpp
1430

See a comment below.

source/renderer/PostprocManager.cpp
40

It's not good to change a default behaviour.

This revision now requires changes to proceed.Apr 16 2018, 10:16 PM
aeonios marked 4 inline comments as done.Apr 16 2018, 11:10 PM

I like this at all, but I have important notes:

  • Screenshots should be before/after, not just after. Also they all have the pretty same angle, that's not good for any comparison.

I did post some before screenshots, but getting exact before/after shots just isn't practical here because the two versions aren't compatible and the new version has a lot of changes in the cpp code. The 'old' 0ad version I have is kept clean and used only for making patches, and thus also doesn't have any of the same replay files or map edits that I use in my active development build.

  • Blur has a noticeable artefacts (because a simple kernel, I suppose), that didn't present for old blur. So it should be fixed, because we try to not make a regression.

What do you mean? The old blur was so ridiculously extreme that you couldn't see anything at all, and that had plenty of artifacts. Virtually all of the artifacts I've seen have either been aliasing or bad art being exacerbated and made more obvious (none of which are caused by my shaders). If you're seeing issues that I'm not due to some gfx card difference then I'll need at least a screen shot to try to diagnose it.

  • It'd be good to have a performance comparison tests for it.

Again, difficult to do any direct tests due to the inherent incompatibility in the versions. I could probably do that later but atm it's a pain because I don't want to revert my svn copy right now while I'm working on this and there's a lot of files to copy which is work I don't want to have to do over again :P

  • Merge HDR & DOF isn't a good idea, because modders/artists may want to enable them one by one, not together.

Allowing them to be enabled independently isn't really a problem. That could be done just by adding some #ifs and separate user options. I would have done that already but user options aren't exactly my area of expertise. Also again, I don't think artists should be able to tell me whether I can use dof or not, especially since it has negligible effect on art, and they still have full control of bloom through the bloom slider in atlas. Theoretically we could even detect if map bloom is set to zero and disable all the bloom-related components automatically.

Also it'd be good to rename the HDR to a more correct name.

That's a fair point but I'm not entirely sure of what all needs to be updated to make that not break stuff, which is the main reason that I didn't. I'll look into it.

  • Why it's called big bloom? And why a param is called bigBlur?

Bloom is split up into two stages, downscaling and blur. "BLOOM_BIG" affects both the downscaling process and the blur, but that particular part of the postproc manager is only concerned with blur. The other function's boolean is called 'filter' because it's only concerned with downscaling and filtering the image. They're basically different things that use the same setting in the shader, and the arg name is descriptive of what those booleans actually do.

  • Fix CC.

Do what?

binaries/data/mods/public/shaders/glsl/bloom.fs
1

shader version, equivalent to openGL 2.1

1

That's what I meant. :P

aeonios updated this revision to Diff 6419.Apr 18 2018, 10:58 AM
aeonios marked 4 inline comments as done.

Ok so I got around to dealing with compatibility issues. The way maps interact with postproc is kind of a pain but I made it work as sensibly as I could.

  • The shader is now called "postproc", but the effect is called HDR. This is to reflect intuitively what it actually does to mapmakers.
  • The "default" null effect has been renamed to "None" to better reflect what it actually does. Technically it sets DOF-only mode, which will hopefully become a user-side option in the future.
  • I've implemented backwards compatibility for existing maps. Maps using "HDR" or "hdr" will have HDR enabled, while anything else forces DOF-only mode.
  • If HDR is disabled then the entire HDR part of the shader is disabled, as well as all of the blurring work that would normally be performed to support bloom. Additionally, if bloom is set to "zero" in map settings it automatically skips the blurring steps and the HDR shader will also skip reading the bloom textures entirely, thus saving GPU cycles/memory bandwidth that would otherwise have been wasted accomplishing nothing. This is an improvement over the existing framework, which does all the blurring whether it's needed or not.
  • I added hooks for selectively enabling/disabling HDR and DOF separately through user options. I don't know enough about the options system to do that easily, but when it's done the back end code is all set up to use it, just uncomment a few things and call the relevant function to refresh the shaders.
  • After tinkering around with bloom and trying out every imaginable combination of things, I ultimately decided to revert to the original bloom algorithm. I found that bigger blur actually tends to look worse, so I replaced the 17-tap gaussian blur with a 9-tap gaussian blur and now it looks virtually identical to the old bloom, preserving the appearance of existing maps. The gaussian blur is much smoother and more artifact free than the box blur, which may be noticeable in some cases but in general you can't really tell the difference. I also got rid of the contrast filtering as it didn't really work all that well. The old blur shader used 6 samples, but I can't tell if they were linear sampled or not because they used some nasty magic constant that they shouldn't have. Overall it should be a similar cost to the legacy shader, and easier to understand.
  • Added a few comments and cleaned up and simplified a few random things after settling on the current implementation.
aeonios updated this revision to Diff 6420.Apr 18 2018, 11:12 AM

Fixed the bit at line 690 in map reader.

aeonios updated this revision to Diff 6421.Apr 18 2018, 11:15 AM

whitespace

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/402/display/redirect

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/403/display/redirect

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/404/display/redirect

Stan added a comment.Apr 18 2018, 2:38 PM

See D1370 for an exemple of Atlas options.

binaries/data/mods/public/shaders/effects/postproc/HDR.xml
4

Can you remove those whitespaces ?

binaries/data/mods/public/shaders/glsl/bloom.fs
1

So we are using GLSL 1.2 ?

vladislavbelov requested changes to this revision.Apr 18 2018, 3:23 PM

I did post some before screenshots, but getting exact before/after shots just isn't practical here because the two versions aren't compatible and the new version has a lot of changes in the cpp code. The 'old' 0ad version I have is kept clean and used only for making patches, and thus also doesn't have any of the same replay files or map edits that I use in my active development build.

You don't need to be compatible. You just need the same position and the same angle of the camera. You can hardcode it, or just use a simple cinemapath.

Virtually all of the artifacts I've seen have either been aliasing or bad art being exacerbated and made more obvious (none of which are caused by my shaders). If you're seeing issues that I'm not due to some gfx card difference then I'll need at least a screen shot to try to diagnose it.

You posted screenshots with blur, hdr and bloom together. So it's harder to review and find issues them all separately.

What do you mean? The old blur was so ridiculously extreme that you couldn't see anything at all, and that had plenty of artifacts.

Yes, but it can be an artist/modder choice, so you need to give it to then, what's a power of the blur they want.

Allowing them to be enabled independently isn't really a problem. That could be done just by adding some #ifs and separate user options. I would have done that already but user options aren't exactly my area of expertise. Also again, I don't think artists should be able to tell me whether I can use dof or not, especially since it has negligible effect on art, and they still have full control of bloom through the bloom slider in atlas. Theoretically we could even detect if map bloom is set to zero and disable all the bloom-related components automatically.

I don't suggest to you to make options. I only suggest to separated DOF and HDR shaders.

Bloom is split up into two stages, downscaling and blur. "BLOOM_BIG" affects both the downscaling process and the blur, but that particular part of the postproc manager is only concerned with blur. The other function's boolean is called 'filter' because it's only concerned with downscaling and filtering the image. They're basically different things that use the same setting in the shader, and the arg name is descriptive of what those booleans actually do.

So the big word doesn't even present in your description (only as the macro name). So I suggest to call it more meaningful, i.e. downscaledBlur or actualSizeBlur.

Do what?

https://trac.wildfiregames.com/wiki/Coding_Conventions

The postproc name isn't good for the shader. Because i.e. a BW shader, a film grain are postproc shaders too.

binaries/data/mods/public/shaders/glsl/bloom.fs
1

No... was to @Stan.

This revision now requires changes to proceed.Apr 18 2018, 3:23 PM
vladislavbelov added inline comments.Apr 18 2018, 3:25 PM
binaries/data/mods/public/shaders/glsl/postproc.fs
23 ↗(On Diff #6421)

Use a linearizeDepth like in dof.fs. Less add/sub operations.

48 ↗(On Diff #6421)

It's not HDR and not something related, it's the bloom. So call it bloom or something similar.

vladislavbelov added a comment.EditedApr 18 2018, 3:31 PM

How Postprocessing should work? An artist/modder has the Atlas or a text editor. He/she adds a list of different PP-effects (i.e. bloom, BW, cartoon) to a map and it works together. Effects are like components. So you don't have to do all things in the one file. And it'd be very good to choose an order of effects.

But if you don't know how to do it or don't want to do yourself, you can wait me. I plan to make changes to the Atlas to allow multiple effects with a different order.

aeonios marked 4 inline comments as done.Apr 19 2018, 3:27 AM

You don't need to be compatible. You just need the same position and the same angle of the camera. You can hardcode it, or just use a simple cinemapath.

That'd be a lot of work. Way more than is justified for taking screenshots.

You posted screenshots with blur, hdr and bloom together. So it's harder to review and find issues them all separately.

Taking no-effect/dof-only/bloom-only/both screenshots isn't a big problem. I'll do that today.

Yes, but it can be an artist/modder choice, so you need to give it to then, what's a power of the blur they want.

If artists are going to have any control over dof then it'd be the transition start distance (ie the distance-power factor) and the distance scaling.

I don't suggest to you to make options. I only suggest to separated DOF and HDR shaders.

Combining the shaders into one saves memory write bandwidth and reduces shader/texture setup costs. Given that those things are all expensive I strongly recommend against splitting them up. Applying the effects separately on top of one another would also be a lot more expensive since the blurs would have to be applied redundantly, but would not look significantly different.

So the big word doesn't even present in your description (only as the macro name). So I suggest to call it more meaningful, i.e. downscaledBlur or actualSizeBlur.

The shader setting is now called BIG_BLUR so they're consistent. I removed filtering entirely so that part no longer applies.

The postproc name isn't good for the shader. Because i.e. a BW shader, a film grain are postproc shaders too.

Fair enough. I guess I'll call it dofhdr or something. Also FTR you can get BW just by setting saturation to zero, so that doesn't need to be a separate effect. Also the postproc manager doesn't actually support custom effects. Because everything is hardcoded and afaik there's no JS interface for openGL, there's no way to support custom postproc shaders without changing the engine code.

binaries/data/mods/public/shaders/glsl/hdr.fs
25 ↗(On Diff #6414)

I tried plugging in that algo and what I got was a wrong result. Everything got severely overblurred indicating that it thought everything was super far away. Add and subtract aren't expensive shading operations anyway. Only stuff like log, pow, exp, sqrt are really expensive, and this algorithm is correct.

edit: actually I remember now, the old dof shader tried to create a sort of arbitrary 'focal depth' and anything closer or farther away would be blurred. My version only blurs far away objects, which looks a lot better and is less intrusive.

binaries/data/mods/public/shaders/glsl/postproc.fs
23 ↗(On Diff #6421)

No. I told you before, the linearizedepth from the old dof does not do the same thing as this, and add sub operations literally don't matter in the slightest. This is correct and it cannot be done any other way.

source/graphics/MapReader.cpp
1430

It turns out this doesn't do what I thought it did, so I reverted it more or less. I'm changing the name of the "default" null effect to "None" to better reflect what it actually does though, so it'll be less confusing to map makers.

source/renderer/PostprocManager.cpp
40

I'm replacing this with something more sensible that provides backwards compatibility and more faithfully respects map settings.

aeonios marked an inline comment as done.Apr 19 2018, 3:36 AM

How Postprocessing should work? An artist/modder has the Atlas or a text editor. He/she adds a list of different PP-effects (i.e. bloom, BW, cartoon) to a map and it works together. Effects are like components. So you don't have to do all things in the one file. And it'd be very good to choose an order of effects.

But if you don't know how to do it or don't want to do yourself, you can wait me. I plan to make changes to the Atlas to allow multiple effects with a different order.

I did that for performance reasons. Also the order of applying dof and hdr has to remain as it is because they use the same blur preprocessing steps so if you apply them as hdr first then dof will cancel out the bloom. The fact that they share preprocessing steps was another motivation for combining them.

Right now custom postproc effects are not actually supported by postproc manager, but that's beyond the scope of this revision.

aeonios updated this revision to Diff 6427.Apr 19 2018, 4:47 AM
aeonios marked 2 inline comments as done.

Naming/whitespace tweaks.

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/409/display/redirect

aeonios marked 2 inline comments as done.EditedApr 19 2018, 5:23 AM

Screenshots:

no effects:


bloom-only:

dof-only:

bloom+dof:

no effects:


bloom-only:

dof-only:

bloom+dof:

bonus shot of dof+bloom with a higher camera angle:

binaries/data/mods/public/shaders/glsl/bloom.fs
1

If we want nice things, anyway. For example nice shadow filtering, nice blur filters, or reasonably cost effective anti-aliasing all require GLSL 120. 110 doesn't let you declare uniforms inline like the arrays used for the blur offsets/weights here.

binaries/data/mods/public/shaders/glsl/postproc.fs
48 ↗(On Diff #6421)

Technically no, it's not HDR, but that's the terminology that was used previously. If and when support for separately enabling bloom and color correction in atlas is added then I'll split them up and call them those more correct names, respectively. For now it doesn't make sense to, especially if we want to maintain backwards compatibility with existing maps.

vladislavbelov added a comment.EditedApr 20 2018, 7:30 PM

I did that for performance reasons. Combining the shaders into one saves memory write bandwidth and reduces shader/texture setup costs. Given that those things are all expensive I strongly recommend against splitting them up. Applying the effects separately on top of one another would also be a lot more expensive since the blurs would have to be applied redundantly, but would not look significantly different.

Expensive is the pretty relative meaning. The blur is expensive too. How much the combining expensive? We need to choose between usability and performance.

We use the javascript language for the game logic. It's expensive, but it allows many useful things. The blur is expensive too. But it looks good. Deferred rendering is expensive, but it allows to calculate PSM with a higher precision, virtual textures, etc. Raytracing is expensive, but it allows nice AO.

Right now custom postproc effects are not actually supported by postproc manager, but that's beyond the scope of this revision.

It's not actually true, you can customize effects by editing a map file manually.

My point is to allow to artists/modders to freely edit effects. Why? Because if you will profile the game, you will notice, that postproc or final rendering don't cost a lot, the most expensive thing is object submitting. As you noticed we don't use instanced rendering, so we draw a lot of tries/units per different calls. So until we have the completely different bottleneck, we don't need to do optimizations that break usability too early.

The shader setting is now called BIG_BLUR so they're consistent. I removed filtering entirely so that part no longer applies.

But the BIG word doesn't mean something exact. Why not call it as I mention above (actualSize, fullSize, etc)?

aeonios marked an inline comment as done.Apr 20 2018, 8:02 PM

Expensive is the pretty relative meaning. The blur is expensive too. How much the combining expensive? We need to choose between usability and performance.

The biggest cost, as with most postproc-type rendering, is in memory bandwidth. Both the blur process and the final combine are shared by bloom and dof. That means the blur process only needs to be performed once for both shaders (and blur can indeed be expensive, as in causes a noticeable slowdown if overused) and the combine can read the screen texture once and write it once for both shaders. Also, as I've mentioned, combining them allows the visual effect to be consistent whether applying just one of the effects or both. There's no loss of usability and you only pay for what you use. However if you use both you get a cost discount. DOF could support customization through atlas (ie distance factor, distance scale kind of like fog), but that's not immediately needed.

We use the javascript language for the game logic. It's expensive, but it allows many useful things. Deferred rendering is expensive, but it allows to calculate PSM with high precision, virtual textures, etc. Raytracing is expensive, but it allows nice AO.

True but getting the best visual effect for a given cost is always good.

Right now custom postproc effects are not actually supported by postproc manager, but that's beyond the scope of this revision.

It's not actually true, you can customize effects by editing a map file manually.

Eh? How? AFAIK you can't actually use custom shaders. You can configure effects that are already supported, but you can't set up custom textures or FBOs or manage the rendering process.

My point is to allow to artists/modders to freely edit effects. Why? Because if you will profile the game, you will notice, that postproc or final rendering don't cost a lot, the most expensive thing is object submitting. As you noticed we don't use instanced rendering, so we draw a lot of tries/units per different calls. So until we have the completely different bottleneck, we don't need to do optimizations that break usability too early.

There's nothing here that prevents map makers from customizing things just as well as they can do already. Well, at the moment they can't turn off DOF but it wouldn't be a large step to allow it and to even make it customizable. That's mostly a matter of atlas though and beyond the scope of this revision. If you give atlas the necessary capabilities I can certainly handle the backend support. This won't be live until at least a24 anyway so we've got time. The way postproc is handled with maps isn't really ideal, and only allowing one effect is rather limiting. Due to how bad DOF was before I don't think there are any maps that use DOF intentionally anyway.

But the BIG word doesn't mean something exact. Why not call it as I mention above (actualSize, fullSize, etc)?

"actualSize" and "fullSize" aren't any more meaningful either. One blur is a 3 tap gaussian blur and the other is a 9 tap gaussian blur. Both are gaussian blurs and both are "actualSize", but one is bigger than the other which is why I named it such.

aeonios updated this revision to Diff 6499.May 1 2018, 3:18 PM

I reduced the big blur from 9 taps to 5 taps, which when using linear sampling uses the same number of samples as the 3-tap small blur. This allowed me to simplify the code by reusing the same code for both 3-tap and 5-tap blurs and just changing the offsets and weights. Visually you can't really tell the difference. I also reinstated using all 3 blur textures for bloom, since the smallest blur texture increases the sharpness of the bloom and makes it look a little better. Since that texture is already being sampled by the shader as long as dof is enabled it adds exactly zero overhead one way or the other.

Vulcan added a comment.May 1 2018, 3:41 PM

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/differential/453/display/redirect

I always have problems in linux setting an anti-aliasing filtering technique (f.e. mlaa/msaa) on 0ad.

(https://en.wikipedia.org/wiki/Multisample_anti-aliasing)

(With oos drivers f.e. radeon/intel, cuz the global setting environmental variables seems having no effect or are disabled.)

Maybe we could make some option here too in 0ad code directly and give option.

(same for anisotropic filtering.)

Need to correct my statement. Anti-Aliasing support with opensource drivers (mesa) sometimes has changes or problems in its releases or/and maybe combined with some distro packages (here Ubuntu 16.04) to enable this feature. With updated mesa stack (to the newest) anti-aliasing now works via dri. Just adding this statement.

Need to correct my statement. Anti-Aliasing support with opensource drivers (mesa) sometimes has changes or problems in its releases or/and maybe combined with some distro packages (here Ubuntu 16.04) to enable this feature. With updated mesa stack (to the newest) anti-aliasing now works via dri. Just adding this statement.

Anti-aliasing is a separate issue from this revision. Eventually I'll probably get around to that, too.

wraitii added a subscriber: wraitii.

I think this might need some atlas touchups, and I'd rather backwards-compatibility be created by changing map settings rather than hardcoding it.

I think this might need some atlas touchups, and I'd rather backwards-compatibility be created by changing map settings rather than hardcoding it.

I'd prefer to have the atlas part be done with a separate revision, but that was already the plan. There are already hooks for enabling or disabling it based on an option, which could be a user option or a map setting. Right now the way map settings work doesn't allow for more than one postproc technique to be specified, so that will require a significant change to the way that works. DOF could also potentially be configurable, but basically same deal.

aeonios abandoned this revision.May 11 2018, 5:20 PM

Closing this. It's gotten so entangled with shadows that I'm just going to merge the two revisions.