Page 1 of 1

A Description of the Improved Lighting and Materials System

Posted: Fri Sep 14, 2018 1:05 pm
by another_commander
Warning: Long post incoming.

It has been a few weeks since the introduction of the new lighting system for Oolite and I have realized that information about it is not so easy to find or track. Whilst some general directions about it have been posted in responses spread over various topics, we do not seem to have a proper description of it or how it can be used to make Oolite's graphics better. So, I will try to do this here, in this post, in order to have a localized source of information and to use this thread as the discussion point for those who would want to look into this new lighting system further.

So without further ado, we'll start by mentioning that the aim of introducing the new lighting system is to enable modders to create materials that look better than before, more natural and more compliant with how light behaves in the real world. To achieve this, we have implemented new shaders with functions that describe the behaviour of light as it encounters different types of materials. More specifically, we are now using what is called a BRDF - which stands for Bidirectional Reflectance Distribution Function - to describe the diffuse and specular components of light, both for ship objects and planets alike. The BRDF uses the standard Lambertian diffuse equation for the diffuse light component (just like before) and the Cook-Torrance model for the specular light component. The basics of those are described in more detail here, but the general idea is that light fr is calculated as fr = kd * fLambert + fCook-Torrance, with kd being the ratio of incoming light energy that gets refracted (diffuse).

Since the diffuse light has not changed compared to the old light model, there is not really much to say about it, so we will concentrate a bit on the specular light part. The Cook-Torrance specular model is described by the equation fCook-Torrance = D*F*G / (4 * NdotV * NdotL). As you see, this model takes into account a few parameters, namely the distribution factor D, the Fresnel component F and the geometry factor G, as well as the vector angle between the normal of the surface where the light hits and the viewer and that of the normal and light direction. Distribution approximates the influence of the surface's microfacets alignment and distribution in the way light is reflected. Geometry refers to light rays occlusion by approximating the way microsurface details overshadow each other on a rough material. Both Distribution and Geometry simulate microfacets on the material's surface (see pic below), which is a very important part of any physically accurate light model and depend on material roughness.
There are more than one mathematical models for distribution and geometry. The distribution model used in Oolite is GGX, while the geometry model of choice is the Smith one. Fresnel refers to the ratio of surface reflection at different surface angles and effectively represents the specular light reflected energy, which is important in its own right for light energy conservation purposes. Oolite uses the Fresnel-Schlick approximation for calculating this component.

The above summary is very brief and just tries to establish the physical compliance of the new light model. I fully recommend that those interested further read the theory part of the link a couple of paragraphs above. For now, we want to keep that the new lighting model has two characteristics: 1) it takes into account material microfacets behaviour and it uses material roughness for this and 2) it implements light energy conservation. (1) necessitated the implementation of a way to describe the material roughness to Oolite. This is where the newly introduced gloss material/shader property comes in. Gloss is described as a floating point number from 0.0 (no gloss, material is extremely rough) to 1.0 (100% glossy material, no roughness and no microfacets whatsoever, material is fully reflective). Its relation to roughness is gloss = 1.0 - roughness. (2) means that the energy of the radiated light (reflected + refracted) cannot be more than the energy of the light that hits the surface. This will weigh in later, when we discuss the importance of the specular light material parameter.

All this sounds great, but how do we use that to make better graphics in the game? Well, the new system means that the workflow for creating materials will have to be slightly different to what we had before. We now have to think about our materials in a more physically oriented way, not necessarily more complicated than before, just different. Note that those who don't want to change their ways of designing materials don't have to. The game still renders old materials just fine, but if you want to get the best result, this is something that you will need to consider.

Most game engines out there that support physically accurate lighting generally follow one (or both) of two types of workflows for designing materials: roughness / metalness and/or gloss / specular. Oolite uses the second one. We chose this workflow because we already have most of it implemented in our materials system (just needed to add the gloss handling), but also because this workflow allows the creation of materials that are not present in nature. In fact, many consider this a flaw of the gloss / specular system, but I choose to call it a feature, since we are talking about a sci-fi game with aliens.

So, gloss / specular it is, then. Specular color refers to the brightness of the specular reflection and gloss refers to its sharpness. For visualizing the relationship between those two parameters, refer to the image showing specular reflection over changing angle below. Specular color represents the height of the peak in the graph, while gloss represents the width of the peak:
Although all other material parameters are still present, those two are now becoming the primary and most important ones. The specular_color property is now not only a nice-to-have in shipdata, but it is fully recommended for all materials. You should always consider putting in a value for specular_color in any material you create if your material relies on the game's default shaders. If you don't, Oolite will put there its default specular_color of (0.2, 0.2, 0.2, 1.0) and materials with this specular color will generally have low specular highlights. Specular maps can still be used and the specular values from the map are multiplied with the defined base specular_color of the material. Gloss maps can be utilized too, and Oolite will be looking at the alpha channel of the specular map for those. Again, the values in the gloss map will be multiplied by the base gloss value of the material. In the absense of a base gloss definition for the material, the value of 0.375 will be used.

With specular and gloss defined, we can already emulate a very wide range of materials. Just to showcase this, see the below image. Don't worry about the values shown for gloss in the image, just concentrate at what happens to the red sphere as gloss increases or as specular color increases.
Non-metallic materials appearing in nature generally have very low specular color values, i.e. they do not have intense specular highlights. Increasing gloss makes them look shinier and more plasticky at high gloss values. Metallic materials have high specular colors and usually reasonably high gloss values. Note something very important here: See how increasing specular color "takes away" the original diffuse color from the object? This is a direct result of the energy conservation we mentioned earlier. If an object has a high specular color, it means that it reflects a big amount of incoming light hitting it. That in turn means that the diffuse color of the object must be reduced, otherwise the total of the refracted and the reflected light energies would exceed the initial incoming light energy. In Oolite you will see the exact same behaviour. Increasing the specular color to metallic levels will result in reduction in diffuse and, for a specular color of (1.0, 1.0, 1.0, 1.0), no diffuse light will be rendered at all, just like you see in this image. For this reason, designing metallic materials in the gloss / specular workflow is simply a case of remembering to use black (or near black) diffuse colors and ensure that the specular color is set to the characteristic color of that metal (e.g. yellow for gold, lighter yellow for brass, bright grey for most other metals). Another way to design metals is to just pass a very high specular color value in the material definition. Any diffuse color will be automatically reduced accordingly to conserve energy (so be prepared to see results slightly different to what you would exepct if you use both a diffuse and a high specular). Similarly, designing non-metallic materials is all about giving the material a diffuse color and assigning it a very dark specular - see the bottom image on this page for more). In practice, all materials should have some specular, even the non-shiny ones. Not going below RGB specular values of 0.02 is probably a good rule of thumb. Finally, it is noted that only metals would normally give colored specular highlights, all other materials should normally have some variation of gray as their specular color. Oolite will allow you to create a material with a diffuse color and a tinted specular_color at the same time, but this would represent an "impossible" material and the results can be interesting. For those who want an example, in the current 1.87 core game the Thargons are set to have such a material. So be careful with how you use your diffuse and specular colors in the materials if you want to be sure that you get something that can actually exist.

As for gloss, OXP creators need to be aware that it is the most important material property from an artistic point of view. While specular maps do provide certain aspects of the visuals on a model, it is the gloss maps or gloss values that really define a material's look. Gloss maps is where artists have the most freedom to let their imaginations run wild, so do make good use of them if you want to create something truly impressive.

As a general hint when creating materials, remember that you must design based on what will actually be seen rather than what the material you have in mind is. Example to make this clear: You have a brand new ship made of titanium alloy. You want to make materials for it. Before going ahead to create a metallic looking material, think how your ship will appear in game. Do you have a diffuse map with a paintjob already made? Or you want to show bare metal on the hull? If you go for the first option, then your material must not be designed as metal, it should be designed as paint applied on metal instead. This means low specular and, if you want to show some shiny, higher gloss maybe. But if you are going to show bare metal, then by all means, produce colored spec maps and black diffuses all you want. The top visible layer of your material is what should decide what the material should be made to look like.

This has been a very long post and there is a lot of information to digest. But I hope it can serve as a basic description and general guideline for using Oolite's new lighting system. The system could be improved further to make it fully physically based with things like:
- Shadows
- Different type light sources (we only have directional light right now, it would be nice to have point and tube lights for things like light from explosions, floodlight effects etc.).
- Environment mapping, which would boost visuals even more. At high gloss values, parts of the environment should be seen as visible reflections on objects, but currently we have no such thing. Interestingly, we do have an entire file in the source code called OOEnvironmentCubeMap.m, which apparently tries to deal with exactly this. Unfortunately this file is not part of the build at this time, which means that an attempt was started at some point but was not finished. It would be really great if it did get finished.

Re: A Description of the Improved Lighting and Materials System

Posted: Mon Sep 24, 2018 10:22 am
by another_commander
A quick note for texture creators and latest rendering changes:

Mind your colorspaces!

Colorspaces are a somewhat tricky subject to explain fully here. Check out this link if you want to get a brief explanation of what this is about. The summary of this is that image storage is not a simple thing. Due to the way early CRT screens displayed stuff, gamma corrections are typically made in stored images, so that when displayed on a screen they look as intended. This makes transitions from dark to lit parts of the image smoother and more pleasing to the eye. But such transitions are not linear, i.e. a 50% gray is not half of 100% white. These images are stored in what is called the sRGB color space.

The problem is that the human eye perceives light in a linear way and that the shaders used for physically accurate reflected light calculations rely on colors being in linear colorspace. As a result, when the shader starts working with its texture color data, it has to remove the gamma corrections from sRGB images (which is what is typically authored by apps such as Photoshop, Gimp etc.) in order to convert them to linear colorspace. At the end of its run, it re-applies the gamma correction to display the reults on a screen. The complexity increases from the fact that data textures used in shader calculations (i.e. gloss, normal, parallax maps etc.) must be in linear colorspace, because we cannot afford to apply gamma correction on them - it would distort data that the shader needs to process, resulting in bad looking normals or unexpected reflectivities etc.

Texture creators need to be aware therefore, what colorspace their textures are made in. Oolite expects textures in the following colorspaces:
- Albedo (diffuse), specular, emission and illumination maps (i.e. color maps): sRGB. The shader will convert such textures to linear colorspace automatically before proceeding. If you pass a linear colorspace diffuse texture to the shader you will get a bad result, because the shader will try to remove the gamma correction when there is really nothing to remove.
- Normal, parallax, gloss maps (i.e. data maps): Linear. The shader will take those textures and use them directly in its calculations. Passing an sRGB map here will result in bad or weird looking output for the opposite to the above reason.

Alpha channels in images are typically stored as linear, which is good. This enables us to use a colored spec sRGB map and still have an embedded gloss map in the alpha channel saved in the correct and expected linear colorspace. Still, whenever possible, it would be recommended to double check your formats when saving textures. If you have made a texture set for a brand new ship and it looks weird in the game, then probably the first thing you would want to check is colorspace for every channel of every texture.

Color values passed in materials, such as specular_color, diffuse_color etc. are expected to be in linear colorspace.

Re: A Description of the Improved Lighting and Materials System

Posted: Mon Oct 01, 2018 11:26 am
by another_commander
Further to the latest post about colorspaces, with build 97f3e070 Oolite has been made a bit more flexible on how texture colorspaces are processed by the default shader.

The materials / shader materials key gamma_correct has now been introduced and regulates whether the shader will apply gamma correction on the diffuse, specular, emission and illumination textures as mentioned in the previous post. The default for all materials is YES and a gamma_correct entry can be made for each material. The setting can also be changed globally to NO if required, using the boolean .GNUStepDefaults key no-gamma-correct accordingly.

Why this change? Well, apart from making Oolite more flexible with its rules, the real reason is that I cannot bring myself to like 100% of the gamma corrected texture results compared to the previous non-gamma corrected state. Although the proper way to render is to correct for gamma, there are occasions where I find the non-gamma corrected results better looking or more interesting, case in point being the pure metal materials. With this method, the best possible result can be now chosen and used. At the end of the day, adhering to physical properties of light and emulating them is fine and all, but the primary criterion for use is how great it looks in-game. So, until I have discovered a mistake or something in the main shader, I would like to keep both options available. Just keep in mind that, theoretically, gamma correction should happen and should be examined as a first option and switch only if the results look unacceptable. Some older OXPs that use the rgb values of the diffuse maps as inputs for effects may be particularly susceptible to this and may have to be switched to the non-gamma corrected method.

For a visual example of what effect gamma correction (or lack thereof) has on lighting, see the below image. The light source color on the first and third pics is yellow, while for the middle row screenshots it is red. Note how more "concentrated"the light reflections appear on the left and how reflection colors change hue abruptly and very noticeably. Lighting appears darker overall and diffuse colors as well as material shininess seem off without gamma correction. All these are flaws that are not seen in the gamma corrected images on the right.

The general guideline remains: Diffuse (albedo), specular, emission and illumination maps should normally be sRGB, all others linear colorspace.