Computer Graphics: Colors, Photons and the Human Eye
27.Sep.2022, Yuriy Georgiev
Intro
The importance of colors in computer graphics is probably more vital than you think.
The reason is that we use them not just to colorize things but also to deliver extra information for the human brain to process, and to “fool” the person about what he sees and how he sees it.
Is it far in the distance, is it shiny or not, at what angle the light illuminates it, etc. — this is all simulated by orchestrating different colors and their shades accordingly.
The computer screen is flat — this is a big enough limitation already if you want to display a 3D object to a person. You need all the help you can get from the knowledge of physics in order to make the technology simulate the look and feel of depth.
To accomplish that optimally, we need to have a solid, basic understanding of how the human eyes work and how they perceive colors and light in real life. What is the physics behind the colors and how does the human eye processes them.
The Human Eye
There are two photoreceptor types in the human eye. Rods and Cones.
They are placed at the back of the eye in the retina and give us color and light sensitivity.
RODS
The Rods are responsible for the perception of the light and the shapes of the objects. They are primarily used in dim and dark environments and are very sensitive to low light levels. They outnumber the Cones by order of magnitude.
Around 95% of the photoreceptors are Rods and they are spread in the outer area of the retina.
CONES
Those are the ones we are interested in when we talk about computer graphics.
The Cones are responsible for the color vision. There are Cones for each base color — Red, Green, Blue. The Cones are far less than the Rods — ~5% of all photoreceptors.
They are concentrated in the center of the retina around the optic nerve.
It’s good noting that the count of Cones is not split equally.
Red-sensing cones are ~60%; Green-sensing cones are ~30 and Blue-sensing cones are ~10% of the total amount of cones.
The brain uses the input from those cone receptors to determine color perception.
This is actually how color blindness appears to happen. It’s caused by the absence or defect in certain color-sensing cones.
There is a lot more we can tell about the human eye but it would be beyond what we care about when we talk about computer graphics.
Light
The light is the source of all colors. But that’s too general. Let’s dissect.
Whenever you have a light emitter what actually happens is that this emitter spreads photons like crazy.
Light travels in wavelengths carried by photons. The word “photon” literally means “light”.
The photons are massless particles that hold a certain amount of electromagnetic radiation.
If you are interested — you can learn more about photons in quantum mechanics.
To put things into perspective let’s give an example.
If we take the sun for example, which is the biggest source of light, essentially it emits gazillions of photons. The warmer the emitter the whiter the photon. Those photons vibrate based on the amount of energy they hold. Those vibrations form the wavelength. The more the energy the shorter the wavelength.
White color in the visible light range means all colors are at their maximum amount.
At a certain point, the photons hit the Earth’s atmosphere, some of them get their direction distorted, others don’t, and they continue to travel until they hit a surface.
At the point a photon hits a surface two things happen. Part of the energy of the photon gets absorbed by the surface, if there is still any energy left in the photon, it ricochets and continues to travel until it eventually hits your eye or gets absorbed completely by another surface. You need a very small amount of photons to get to your eye in order to see. Our eyes are very sensitive.
The interesting part here in the context of computer graphics is the kind of surface the photons hit before they get to your eyes.
Surfaces
Each surface has several properties. Roughness, color, angle, etc.
The energy amount absorbed by the surface and the angle a photon will be directed after a ricochet depends on the surface color and its roughness.
If the surface is very polished (less rough) it appears shiny. That’s because it absorbs a very small amount of the photon’s energy. The surface roughness contributes to the direction angle the photon will continue to travel after the ricochet. If the surface is white chalk, the photons will ricochet in all kinds of directions. If it’s a perfect mirror (and by perfect I don’t mean on an atomic level but just on a visual level) it will reflect the photon at the same degree based on the surface normal vector (the normal vector is a perpendicular line to the surface).
If the surface is rough like chalk photons will ricochet in all kinds of directions.
In the case of the mirror, the exact angle is actually cos(theta) from the normal vector of the surface.
The surface has some color. Whatever color it has, it reflects it, the rest gets absorbed by it.
For example, if you have a red object, whenever a white photon hits its surface, the surface will absorb all the colors but the red one. The red will be reflected with the photon. Therefore when the photon hit your eye it will be absorbed by the red Cones and you will see a red object.
This is exactly the reason why our display pixels are created by red, green and blue micro-diodes. They simply change their intensity to generate a mix of those colors. If only red and green are lit — you get yellow.
Colors Visible Spectrum
Our eyes, as great as they are, are very limited to what colors they can see. The color depends on the wavelength. The shorter the wavelength the bluish the color, the longer the wavelength the reddish the color.
The visible wavelength spectrum is just between the ultraviolet waves and the infrared waves. This is a very narrow fragment of all the kinds of wavelengths we know about.
What we can perceive with our eyes is wavelengths with a magnitude between ~400nm up to ~700nm.
There are animals that have more than 3 color receptors in their eyes. There is a shrimp with 12 receptors (there is debate on how many receptors it has — some say 16, others 12…).
This bad boy can see ultraviolet.
Utilization in computer graphics
Knowledge is great. But it’s worthless if not put into practice. Just like any unrealized idea. It — worth nothing.
So how do we use this knowledge in computer graphics?
First of all, we intuitively do some things without even thinking about them. Painters do that constantly. Shades, shadows, light — everything is simply a mix of colors or lack of them (in the case of black). Black is exactly that — no photons, no wavelengths, no colors.
As I mentioned at the beginning our job in computer graphics is to fool the human brain that it sees something that has depth in it. We use colors to do that.
If I simply show you a cube it’s not enough.
Have a look at the following picture and try to recognize what is it.
Sure, you know it’s a cube because I told you. But if you disregard what I told you it’s just a flat shape over a background.
Our brain needs more information in order to put what we see into perspective.
So we need to add another layer of information — shades.
If we add shades to the colors we already used we will get what we are looking for even in a very primitive form.
As you can see we still use blue color but in different shades. This is enough information for our brain to start recognizing basic shapes and add depth to the way we perceive them.
Of course, colors are not everything but they are an extremely important part of computer graphics. You can recognize a cube drawn simply with lines with no colors but that’s just another, a different layer of information that we give to our brain to work with. If you provide only the vertices (essentially the building points) you will be still confused about what you are watching at.
Surprise
You now literally know how Ray Tracing works. Except it works in reverse order. It tracks the photons from the human eye back to the light emitter. This is required in order to track only the visible photons back to their source, due to the enormous amount of performance waste otherwise.
In real-time computer graphics, you always care about only what you see and nothing else.
The information is not completely exhausting as the topic of Ray Tracing is a big one. However the physics the simulation is based on are exactly those.
Conclusion
Basically, the ultimate path to a physically accurate photo-realistic picture is Ray Tracing.
It simulates the life cycle of the photons precisely.
If you are interested in code example — you can have a look at one of my example projects which is a simple real-time CPU-based ray tracer.
It simulates a synthetic environment with moving objects, lights and shadows, tracing rays in reverse order — from the eye to the light source instead of the opposite.
That’s the end of this article. There is a whole lot more about the subject of colors, lighting and reflections. Some materials like glass or water do not only reflect light but also refract it.