GPU Pro 4 – Practical planar reflections using cubemaps and image proxies (with Video)
May 7, 2013 4 Comments
I have written with my co-worker Antoine Zanuttini an article in the GPU Pro 4 book:
I am particularly proud of the cover as this is extracted from the project I worked on “Rememer Me” from Dontnod entertainment edited by Capcom. There is a short sum up available on the GPU Pro blog : http://gpupro.blogspot.fr/2013/02/gpu-pro-4-practical-planar-reflections.html that I will duplicate here.
Rendering scenes with glossy and specular planar reflections is a challenge in video game, particularly when targeting old hardware like current console (PS3, XBOX360). Real time planar reflection is often implemented by re-rendering the scene from the point of view of a reflected camera, an expensive process. At the opposite, several game developers use a simple generic 2D environment texture which lack realism but is really cheap. Others use screen-space local reflection which has edge cases and still has a not negligible cost.
For the game “Remember Me”,we have developed a really fast planar reflection solution usable on every ground. In our GPU Pro 4 chapter we discuss how we render an approximation of the reflected scene with the help of parallax corrected offline-generated elements: Environment map and image proxies. The planar reflection has enough quality for game and take into account the roughness of the surface.The chapter discusses about the algorithm details and follows the work we have presented at Siggraph 2012 “Local Image-based Lighting With Parallax-correctedCubemap“.
Our goal was not only to discuss implementation of our algorithm but also its usage in a context of a game development. We describe the tools we develop for our artists and the best practices they discover. Artists are creative peoples and they have pushed the boundary of our tools where we not expected them. Want to look at the result in action? See this video which accompanying the article
Sébastien Lagarde 
I update the post with the article’s accompanying video which you can get with the book GPU Pro 4. This illustrate our tools in the Unreal engine 3 editor and the result in our game “Remember Me”.
The video is in the post. You can also see it here : https://www.youtube.com/watch?v=_jh2rDyOXGI
Pingback: GPU Pro 4の”Practical planar reflections using cubemaps and image proxies”の著者のサイト
Hi, great results with image proxies! And thank you for your blog, you have a lot of great and interesting posts here.
Your technique seems to be similiar to “billboard reflections” from Unreal Samaritan demo combined with “Box projected cubemap environment mapping” (you can find it on gamedev.net). I also have experimented with similiar techniques and found them nice for glossy reflections and really fast.
I’m interested how many separate billboards (quad intersections/texture reads in shader) do you have in a typical scene? My GTX560 can deal with about 40-50 billboards without making framerate really uncomfortable to play, but PS3 level hardware is obviously slower. I guess you have some seriously optimized math.
And how do you convert characters into billboards, are they rendered each frame as simple lod geometry from main camera angle into small textures and then reflected? Or do you use separate billboards for each movable limb of each character?
Also how do you compute glossiness? I found that sometimes it’s not enough to only prefilter billboard textures and change mip based on ray distance, because blurry billboards can actually leak some color beyond their bounds thus becoming wider than original quad. It is possible to leave a lot of empty space around texture before prefiltering, but it’s just memory wasting. Better working idea is to compute intersection of reflected cone (based on roughness) with quad (at least approximately). Then you find correct mip level by examining the intersection area and correct texture coords by using the center of that area, then weight it comparing intersection area with total cone slice area, so you can get glossy reflection when a simple ray would miss the quad. However, such approach add quite a few more instructions for each billboard.
Hey,
thank you for the feedback!
Sure Image proxies/billboard for reflection are not a new idea and the “Toy shop” demo from ATI in 2005 already use it as other before (in the movie industry). This is the usage to enhance cubemap reflection and the tools we provide for our artists which is interesting here.
About your questions, I am sorry but I will be lazy here as most of the answers, reference to articles you mention, code and performance numbers can be found in the GPU Pro 4 article (And also in the Siggraph talk).
So I will only focus on your last question.
To clarify we do not use “billboard” captured in realtime but offline image of a particular object (so no background in it).
We do not use distance to select mip level but roughness of the object. We do not have distance dependant roughness management but your way is a better approximation that could be use at the cost of added instructions (finding intersection even with simple image is not free on consoles).
The big problem from my point of view is the prefiltering of the billboard. If you want to do it correctly, you need to get the whole environment around the billboard.For a very low roughness you require the hemishere around the current texel of the billboard. What we do to approximate it is that we render the billboard inside our current cubemap (as our billboard purpose is to enhance our cubemap) then prefilter it at runtime. As this can be an expensive process we use a 2D texture and a simple gaussian filtering instead (so yes we lose the specular power filtering of offline process and for low roughness the prefiltering still not correct as the 2D texture can’t handle the whole cubemap but the visual is not too bad).
Cheers