Weathering and aging effects in the hand of artists

Version : 1.1 – Living blog – First version was 04 May 2014

With permission of Dontnod entertainmenhttp://www.dont-nod.com/

It is now frequent in game to have dynamic weathering and aging effects: Rain, snow, dirt, rust, dust, pollution… Most of the time these effects are drive by programmers because they require specific code like access to a custom shadow map or perform by a full screen space post process. Before leaving Dontnod I was working on a feature to allow artists to handle themselves these kind of effects. At the time we were requiring a lot of effects and I wanted to allow artists to prototype as much idea as they can. This post will provide explanation of this feature and will give implementation details under the Unreal Engine 4. It aim to work with a deferred rendering engine. These weathering and aging effects feature has been use at Dontnod but there is some pitfalls which may prevent to use it effectively in a shipping game. It all depends on the scale of the game and performance expected. I hope by exposing this idea that others could improve the system :)

I thanks Dontnod to allow me to talk about it.

 The weathering and aging effects

We know from graphic literature that a lot of weathering and aging effects can be done through surface properties modification [1]. Of course using complex multi-layered lighting model is the right way to handle it but it is way less flexible, will require a programmer to code every effects, and it will be difficult to get right regarding all supported lights type (like image based lighting or area lights).

As an example, the wet surfaces appearing when it is raining could be simulated with material properties modification: Roughness, diffuse albedo, normal… I talk a lot about this subject in other posts on this blog (see water drops series). However these modifications should be done only were it matter. When it is raining, you don’t want your interior surface to be wet. In Remember Me, we were handling this by adding some extra code in the shaders to modify the surface properties, then artists were vertex painting the part of the surfaces requiring to be dry. But this was not sufficient to handle all cases of wet surface. For example, a player walking in a wet street could, once he get into a dry interior, will let wet footprint on the ground. This could have been simulated with decals.

Taking another example, in Kill zone 4: Shadow fall [2] they perform a full screen space pass to modify the material attributes where normal are pointing up to add dust where it matter (after an explosion for example).

GBuffer modification

Thinking with a deferred renderer in mind, it is possible to identify a set of desired control for artists allowing them to modify material properties and simulating a weathering or aging effect. These controls are perform with GBuffer modification with:
- Deferred decals
- Full screen space quad. I call it material postprocess.
- Deferred effect lights: These are same as lights with shadow map or not, except they behave like deferred decal. The amount of light being use as opacity for blending operation. Soft shadow map also allow smooth transitions between effects.
- Object shaders: Properties will be modified directly at GBuffer generation time in the shader of the object. So this require specific code in each shaders. User could use vertex painting to bring information for an effect.

Every tools will perform some material properties modification to simulate an effect, like darkening the diffuse and boosting the smoothness for wet surfaces. All the GBuffer modification must be done before the lighting pass to be taken into account.

Applying the material property modification for the GBuffer modification control could be done in two ways.  Most common case is to use hardware blending, but it could be too restrictive. The other case is to use read/write ability into the same textures (a.k.a prorgammable blending). Sadly this ability is not widely supported. The PS4 and Mantle support it for example, DX11 doesn’t (I don’t talk about intel’s pixel synchronization but simple read/writing the same pixel).

Delaying the GBuffer modification

To support every platform and to be able to do any customization of material properties, I perform the modification of material properties in an extra full screen pass. Effectively delaying the GBuffer modification. Rather than modifying the GBuffer, the different tools simply output an effect weight inside the GBuffer. This effect weight is read later in the extra pass to apply the effect. There is multiple benefit to do that:
- Applying only one time the effect could save performance for heavy effect
- Accumulating effect weights could allow to clamp it in the delayed pass in order to limit the strength of an effect
- Centralized place to deal with the effect. Easier to author.
Sadly it will require to store the effect weight in the GBuffer.
Read more of this post

DONTNOD Physically based rendering chart for Unreal Engine 4

With permission of Dontnod entertainmenhttp://www.dont-nod.com/

I get the permission from Dontnod entertainment to release the graphic chart I, Sophie Van de Velde (Lead environment) and Laurent Harduin (Lighter) made for the artists to help them with the physically based rendering (PBR) workflow present in the Unreal engine 4. This chart is an updated version of the previous chart I post on this blog DONTNOT specular and glossiness chart.

To save the chart, just right clik and chose save link target as… The chart is a PNG file.

DontnodGraphicChartForUnrealEngine4

(Real world pictures courtesy of Andrea Weidlich from “Exploring the potential of layerer BRDF models” siggraph asia 2009)

The chart has been design for Dontnod team and with Unreal engine 4 convention for textures based on the Disney “principled” BRDF use in the Unreal engine 4 [1]. The chart is use as predefined value that the artist color pick and use when creating textures. For the different meaning of the parameters except the Porosity you can refer to the Unreal engine 4 documentation: https://docs.unrealengine.com/latest/INT/Engine/Rendering/Materials/PhysicallyBased/index.html

For the details of some values, refer to the other article from this blog like feedding a physically based shading model or GDCEurope 2013 talk : The art and rendering of Remember Me.

BaseColor

The color is provide in sRGB 0-255.

The diffuse part of the base color (the one use by the non-metallic) must be in the range of the first gradient 50-243. There is some sample values of real world material in sRGB below the gradient. Some of these values are base on real world measured material (from misc sources, not done by us) and other are have been generated by Laurent Harduin. He take calibrated raw picture of representative material, take the luminance histogram in Photoshop and use the value of the medium axis for the luminance. Then he blur the picture and take one pixel inside the blurred region and use that as the color value. This explain why in few case like the clean cement the color and the luminance doesn’t match perfectly. We also lower a bit the value to take into account the invevitable specular present during the capture.

The reflectance part (the one use by metallic) must be in the range 186-255 (not present in the chart). Some example are provided below the grey square. Most of the time the metallic color of material match what the eye see.

Metallic

Simple monochrome linear parameter. Range is 0-1 but the gradient is from 0-255. The yellow sphere below represent in-engine capture of a sphere and a cube.

Roughness

The specular part of Disney “principled” BRDF is a GGX BRDF. It use a roughness parameter. This roughness is the “Disney roughness”, not the real GGX roughness. Disney Roughness = sqrt(Roughness). When use at runtime this Disney Roughness is transform to the GGX roughness with roughness = Disney Roughness * Disney Roughness.

The gradient display roughness from 0 for smooth (left) material to 1 for rough material (right).
The grey gradient are from 0 to 255 and red segments are displayed  every 1/10 with a sphere like object below to show the in-game result of the designated value.
The first row of real world image above represent no metallic object, the second row represent metallic object. Goal is to give artist a better feeling of what is roughness.
The first row of sphere like object represent metallic object, the second row represent non-metallic object.

Note: The roughness here is coupled with the BRDF used by the Unreal engine 4, it may not be compatible with other engine or offline renderer.

Porosity

The Dontnod chart include an unusual parameter named Porosity. This parameter is the “open porosity” of a material. It can be used for driving weathering and aging effect (Pollution, rain, aging…). More details on its usage can be found in previous blog post: Water drop 3a – Physically based wet surfaces and Water drop 3b – Physically based wet surfaces. In practice Dontnod use it mainly with the dynamic wet formula provided in the  mentionned previous post.

The range is remapped from 0-1 to 0-70% of open porosity. There is real worl image to try to give a feeling of what the value mean. An extremely porous material is the clay (70%), but open porosity can vary a lot for same material, clay could also be only 50%.

Porosite

From Kevin Hnat thesis [2]. French translation from top to bottom. Clay, chalk, sand, limestone, granite.

Note: The porosity parameter is not used in the BRDF formulation. More research need to be done on this topic and currently there is only one porosity BRDF paper available which doesn’t fit with the current need.

Reflectance

There is no chart for the reflectance value of the Unreal engine 4, just let the value by default and apply a cavity map on it.

Difference with the old chart

The previous chart was based on modfied Unreal engine 3 where I introduced a new physically based renderer based on Blinn-Phong BRDF. This new chart is based on the Disney “principled” BRDF use in the Unreal engine 4 which is based on the GGX BRDF. The glossiness of the previous chart and the roughness of this one do not match. Moreover, the range is inversed. Previous specular value for metalic can be reuse for Unreal engine 4 reflectance parameter. UE4 use a mapped range of 0-1 to 0.08, so you need to do the conversion. But in practice there is so little difference that’s it is not necessary to use the precise values.

Reference

[1] Burley, “Physically based shading at Disney”, http://disney-animation.s3.amazonaws.com/library/s2012_pbs_disney_brdf_notes_v2.pdf
[2] Hnat,  thesis “Influence of surfaces micro-geometry on realistic rendering” in French, http://epublications.unilim.fr/theses/index.php?id=4820

Water drop 4a – Reflecting wet world

Version : 1.0 – Living blog – First version was 08 September 2013

This is the fourth post of a series about simulating rain and its effect on the world in game. But it could be read without reading the previous post. The subject is “the reflection”.  The post is split in two parts A and B:

Water drop 1 – Observe rainy world
Water drop 2a – Dynamic rain and its effects
Water drop 2b – Dynamic rain and its effects
Water drop 3a – Physically based wet surfaces
Water drop 3b – Physically based wet surfaces
Water drop 4a – Reflecting wet world
Water drop 4b – Reflecting wet world

When a world scene is totally wet, the most striking visual cue is the reflected environment. Of course all surfaces permanently reflect their surrounding but this is more visible under rainy day. The topic of this post is “reflection”. The reflections as we see it in real world includes all the surrounding lighting. When we talk about reflection in game, too often we restrict this to water or smooth surfaces reflection. But “reflection” is just a convenient word to designate the normal lighting process. In game we separate lighting as direct, indirect and emissive. If you handle direct and indirect lighting on any kind of surfaces from smooth to rough, you have your reflections. There is no need of a particular process for it.
For Remember Me we decided to go this way. To get a good rainy mood, we were looking for having reflection everywhere on every surface. For example, we use the same process to get reflection on rocks as well as in puddles.

Reflection – Theory

The observation post already presents many pictures illustrating reflection. But I will present some others here to highlights some characteristic of reflections.

Reflection with smooth surfaces

Let’s consider à perfectly smooth surface. Most people think that the reflection of a scene in surface like calm water or mirror is the scene itself upside down.

CalmWaterReflection

But this is a really wrong assumption. The reflection depends on the distance from reflected objects and the viewer’s position.

CalmWaterReflection2

The differences become smaller, the closer we bring our eyes to the reflecting surfaces and the farther away the objects are. On the pictures below, see how Mickey Mouse is hidden by the blue cow until you reach a glazing angle with the mirror.

ReflectionMickey0ReflectionMickey1ReflectionMickey2

Read more of this post

GDCEurope 2013 talk : The art and rendering of Remember Me

Here is the slides of me and my co-worker Laurent Harduin’s talk at GDCEurope 2013 : “The art and rendering of Remember Me”

A short sum up can be seen on the official GDC Europe 2013 website : The art and rendering of Remember Me

Accompanying video of the slides are on youtube, I have modified the slides to put youtube URL directly in the slides.

Powerpoint 2007:  GDCE13_Lagarde_harduin_light.pptx
PDF: GDCE13_Lagarde_harduin_light.pdf

RememberMeFront

FXGuide Game environment series based on Remember Me

Remember Me, the new Capcom’s IP and the project I was working on has been released!
For the occasion FXGuide has publish a series of three articles about the rendering of Remember Me.

LogoRememberMe

LogoFXGuide
Mike Seymour from FXGuide, Michel Koch (Co-art director of Dontnod) and me have worked together to get these articles up. These articles overlap some posts I have done on this blog, this is what I will details here.

Game environments – Part A: rendering Remember Me

http://www.fxguide.com/featured/game-environments-parta-remember-me-rendering/

The first article talk about the PBR system of Remember Me describe in

Adopting a physically based shading model and Feeding a physically based shading model

and the reflection system describe in two parts:

Image-based Lighting approaches and parallax-corrected cubemap and GPU Pro 4 – Practical planar reflections using cubemaps and image proxies (with video)

The FXGuide article include exclusive video, concept and image from the game not present in above posts.It link the concept art with the technic develop. It also include return from the Dontnod team. The blog posts contain pseudo-code and more programming stuff not present on FXGuide.

Game environments – Part B: rain

http://www.fxguide.com/featured/game-environments-partb/

The second article talk about how we handle the rain in Remember Me, as was describe in

Water drop 2a – Dynamic rain and its effects

The FXGuide article include additional concept to show rainy mood world of the game. The blog posts contain pseudo-code and more programming stuff not present on FXGuide.

Game environments – Part C: making wet environments

http://www.fxguide.com/featured/game-environments-partc/

The third articles talk about physically based wet surfaces, as was describe in:

Water drop 1 – Observe rainy world, Water drop 3a – Physically based wet surfaces and Water drop 3b – Physically based wet surfaces

FXGuide article sum up this huge topic to be more accessible and less painful to read. People willing to go more in depth could read the above posts. The blog posts contain pseudo-code and more programming stuff not present on FXGuide.

Dead End Thrills screen of Remember Me

http://deadendthrills.com/category/remember-me/

Not related to technical stuff but as the subject of this post is the rendering of Remember Me, don’t forget to take a look at the awesome high resolution screenshots done by Dead End Thrills.

GPU Pro 4 – Practical planar reflections using cubemaps and image proxies (with Video)

I have written with my co-worker Antoine Zanuttini an article in the GPU Pro 4 book:
FinalCover
I am particularly proud of the cover as this is extracted from the project I worked on “Rememer Me” from Dontnod entertainment edited by Capcom. There is a short sum up available on the GPU Pro blog : http://gpupro.blogspot.fr/2013/02/gpu-pro-4-practical-planar-reflections.html that I will duplicate here.

Rendering scenes with glossy and specular planar reflections is a challenge in video game, particularly when targeting old hardware like current console (PS3, XBOX360). Real time planar reflection is often implemented by re-rendering the scene from the point of view of a reflected camera, an expensive process. At the opposite, several game developers use a simple generic 2D environment texture which lack realism but is really cheap. Others use screen-space local reflection which has edge cases and still has a not negligible cost. 

For the game “Remember Me”,we have developed a really fast planar reflection solution usable on every ground. In our GPU Pro 4 chapter we discuss how we render an approximation of the reflected scene with the help of parallax corrected offline-generated elements: Environment map and image proxies. The planar reflection has enough quality for game and take into account the roughness of the surface.The chapter discusses about the algorithm details and follows the work we have presented at Siggraph 2012 “Local Image-based Lighting With Parallax-correctedCubemap“.

Our goal was not only to discuss implementation of our algorithm but also its usage in a context of a game development. We describe the tools we develop for our artists and the best practices they discover. Artists are creative peoples and they have pushed the boundary of our tools where we not expected them. Want to look at the result in action? See this video which accompanying the article

Memo on Fresnel equations

Version : 1.3 – Living blog – First version was 29 April 2013

This post is a memo for me about misc thing I found related to the Fresnel equation as there is plenty of formula into the wild. I spend some times to gather these information and I was thinking it can interest others. This will not be really relevant to game rendering but still good to know. This could be useful when doing reference or when dealing with total internal reflection (frequent with multi layered BRDF). This is a memo, not a tutorial, so I won’t give basic explanation of many concepts like reflection/refraction, Snell’s law, index of refraction (IOR), total internal reflection (TIR), etc… At the end of the post I provide a Mathematica file with all the equations and graphs.

Notation for this post:

Conductor mean Metal and dielectric mean no-Metal material.

Light moves from a medium of a given IOR n_i (incoming) into a second medium with IOR n_t (transmitted).

Conductor have complex IOR with an imaginary part k_t (note that’s t “transmitted” is a bad choice for conductor but I found it more identifiable than n_1 and n_2, k_2).

Fresnel Equation basis

All equations using an index of refraction n_t can be replace with the same equation using a complex index of refraction n_t -\mathbf{i} k_t.

Snell’s law dielectric-conductor interface : \frac{\sin\theta_i}{\sin\theta_t}=\frac{n_t -\mathbf{i} k_t}{n_i}

Snell’s law dielectric-dielectric interface : \frac{\sin\theta_i}{\sin\theta_t}=\frac{n_t}{n_i}

The calculations of the reflectance R (What we are looking for when we want to calculate the percentage of reflection and transmission) depend on p- and s-polarization of the incident ray. Rs and Rp are the reflectivity for the two planes of polarization. Rs is perpendicular (s = German senkrecht) and Rp is parallel. The reflectance R for unpolarized light is the average of Rs and Rp:

R=\frac{(R_s+R_p)}{2}=\frac{(r_\perp^2+r_\parallel^2)}{2}

Following expressions use R_s and R_t or r_\perp and r_\parallel depend on cases to simplify notation.
Read more of this post