Radiance An interactive monograph on ray tracing

Radiance

How a computer turns a description of a three-dimensional scene into a picture, by following the paths along which light could travel between the scene and the eye.

An interactive monograph in nine sections, with six live figures. The figures are operable by mouse, touch and keyboard, and the document conforms to WCAG 2.2 AA. Built as a single static page in vanilla HTML, CSS and JavaScript, with no framework and no build step.

Abstract

Ray tracing is the family of rendering methods that compute an image by simulating, geometrically, the way light travels through a scene. This document develops the subject from its starting point, the observation that a picture is the record of the light arriving at a camera, through to the integral equation that governs all of light transport and the statistical method used to solve it. It treats the pinhole camera and the primary ray, the intersection of a ray with a surface, the shading of a hit point by the surface and the light, the shadow ray, recursive reflection, and finally the rendering equation and Monte-Carlo path tracing. A literature review, placed early, traces how each method was introduced to remove a specific failure of the one before it.

Every figure is interactive and runs a genuine ray tracer over a synthetic scene of circles. The two-dimensional setting is a deliberate choice: it keeps every ray, every intersection and every bounce visible on the page, and it allows the geometry under discussion to be computed exactly rather than approximated by a trained model or hidden inside a library. Where the full method is beyond the reach of a short interactive figure, the figure models the honest core of it and the surrounding text says so plainly.

1Light, and which way to trace it

A rendered image is a grid of pixels, and the task of a renderer is to decide the colour of each one. A scene given to the renderer consists of geometry, the shapes of the objects; materials, which describe how each surface interacts with light; one or more sources of light; and a camera, which fixes the point from which the scene is observed. The colour of a pixel is the colour of the light that would pass through that small piece of the image on its way into the camera. Rendering, stated at this level, is the problem of working out what light arrives where the camera is looking.

In the physical world light originates at sources, travels outward, strikes surfaces, is partly absorbed and partly scattered onward, and repeats this until it is extinguished or happens to enter an eye. One could imagine simulating that process directly, emitting rays from every light and following each until it either reaches the camera or dies. The difficulty is one of arithmetic rather than principle. Almost none of the light that leaves a source ever reaches a particular small camera, so the overwhelming majority of the simulated effort would be spent on rays that contribute nothing to the final image. The first figure of this document makes that waste visible.

The method that makes rendering tractable is to follow light in the opposite direction. Rather than starting at the lights, the renderer starts at the camera and traces rays outward into the scene, one or more for every pixel. Each such ray corresponds to a piece of the image that must be coloured, so no ray is wasted. This is valid because the physics of light transport is reciprocal: a path from a light to the camera is equally a path from the camera to the light, and the quantity carried along it does not depend on the direction in which the path is read. Tracing backward from the eye therefore computes the same image as tracing forward from the lights, at a tiny fraction of the cost. The whole of this document concerns that backward process, which is what the term ray tracing denotes.

Radiance
The radiometric quantity that a renderer ultimately computes for each ray: the power of light travelling along the ray, measured per unit of the area it crosses and per unit of the range of directions it occupies. Radiance has the useful property of being constant along a ray through empty space, which is exactly what allows the colour seen through a pixel to be found by following a single ray outward. The quantity gives this monograph its name.

2The camera and the ray

To trace rays from the camera the renderer needs a model of the camera precise enough to say where each ray begins and in which direction it points. The model almost universally used is the pinhole camera. It consists of a single point, the centre of projection, through which all light entering the camera is taken to pass, together with an image plane, a flat rectangle on which the image is formed. The pinhole has no aperture and no lens, so everything it images is in perfect focus, which is a simplification the later history of the subject had to undo.

Each pixel occupies a small region of the image plane. The primary ray for a pixel begins at the centre of projection and passes through that pixel's position on the plane, continuing into the scene. Its origin is the camera point and its direction is fixed by the geometry of the camera: a position in space, a direction of view, and a field of view, which is the angular width the image plane subtends and therefore how much of the scene is captured. A wide field of view gathers a broad, distorted view; a narrow one gives a tighter, flatter image. A ray is written compactly as a point moving along a direction, the origin plus a distance parameter multiplied by the unit direction vector, and the distance parameter will reappear throughout this document as the natural measure of how far along a ray something lies.

The figure above draws the camera, its image plane and the fan of primary rays, and it also offers the contrast described in the first section. Switching it to trace from the light casts rays in every direction from a source and counts how few of them pass through the image plane at all. The count is small, and it is the visible form of the argument for tracing backward.

Casting one ray through the centre of each pixel treats the pixel as though it were a single point, when it is in fact a small area that may straddle the edge of an object. The consequence is aliasing, the jagged staircasing of edges and the shimmering of fine detail. The remedy is to cast several rays per pixel, at different positions within it, and to average their results; this is supersampling, and the question of how best to choose those positions is one the literature review will reach.

3A history of rendering: a literature review

The components assembled in the rest of this document arrived over several decades, and each was introduced to repair a specific, identified weakness in the methods that preceded it. The account that follows is deliberately causal: in each case the question is not only what a method achieved but which earlier failure it was built to answer.

The earliest interactive rendering did not trace rays at all. Scanline methods processed the image one row at a time, and the central difficulty they faced was the hidden-surface problem, that of deciding which of several overlapping surfaces is the visible one at each pixel. The depth buffer, which records the distance to the nearest surface seen so far at every pixel, gave an efficient and durable answer, and it remains the basis of real-time rasterised graphics. What these methods could not do was treat light that travels between surfaces. Each surface was shaded in isolation, so a reflection in a mirror, a shadow cast by one object onto another, or light bending through glass had no natural expression. They were effects that required following light beyond its first encounter with a surface.

Ray casting, introduced by Appel in 1968 [2], took the first step toward following light. Appel cast a ray from the eye through each point of the image to determine the visible surface, and from the visible point cast a further ray toward the light to decide whether the point stood in shadow. The method established the ray as the unit of the computation, but it stopped at the first surface and a single shadow test; it did not follow light onward after a reflection or through a refraction.

The recursive ray tracing of Whitted in 1980 [4] was developed because ray casting could not produce reflection, refraction or the sharp shadows of a complex scene from one consistent mechanism. Whitted's contribution was to spawn new rays at each surface the eye ray struck: a shadow ray toward each light, a reflected ray in the mirror direction, and, for transparent materials, a refracted ray bent through the surface, with the same procedure applied recursively to every ray so generated. Mirrored and glass surfaces became possible for the first time. The method's limitation was a direct consequence of its design. Every ray was a single, infinitely thin sample sent in one exactly determined direction, so the only phenomena it could render were perfectly sharp ones: hard-edged shadows, mirror-perfect reflections, and an everywhere-focused image. Photographs and the world contain soft shadows, glossy rather than mirror reflections, and the selective focus of a real lens.

Distributed ray tracing, presented by Cook, Porter and Carpenter in 1984 [5], was introduced precisely to produce the soft phenomena that Whitted's point samples could not. Its idea was to replace each single ray with a small bundle of rays distributed stochastically over an extent: over the area of a light source, which yields the soft penumbra of a real shadow; over a range of directions around the mirror direction, which yields glossy reflection; over the area of a lens aperture, which yields depth of field; and over an interval of time, which yields motion blur. Cook set out the statistical reasoning behind this two years later [6], showing why sampling at jittered, irregular positions converts the structured error of aliasing into unstructured noise that the eye tolerates far better, and how that noise diminishes as more samples are taken.

The unifying statement of the problem came from Kajiya in 1986 with the rendering equation [7]. Kajiya expressed the light leaving any surface point in any direction as a single integral equation, and in doing so revealed every earlier method as a partial and approximate solution of it. The equation made explicit something Whitted-style tracing had quietly omitted: indirect light, the illumination a surface receives not from a light source directly but from other surfaces that are themselves lit. A white floor beside a red wall is tinted red by light that has bounced once; recursive ray tracing, following only mirror and light directions, could not account for it. Kajiya also proposed the solution method that the eighth section of this document describes, path tracing, which estimates the integral by following light paths chosen at random.

Path tracing is general but converges slowly for certain kinds of light path. Photon mapping, introduced by Jensen in 1996 [8], was a response to one such weakness. Caustics, the bright focused patterns cast when light is concentrated through glass or off a curved mirror onto a diffuse surface, correspond to paths that random sampling from the eye finds only rarely, so a path tracer renders them with stubborn noise. Jensen's two-pass method first traces particles of light from the sources and records where they land, then uses that stored record during rendering, which captures caustics and indirect light efficiently. A further difficulty, that of scenes in which the important light paths are not merely rare but hard to discover at all, such as a room lit only through a door left slightly open, was addressed by Veach and Guibas in 1997 with Metropolis light transport [9], which adapts a path that has been found into nearby paths so that effort concentrates where light actually flows.

For most of this history ray tracing was an offline method, used for film and for still images where minutes or hours per frame were acceptable. Its migration to interactive rates came with the general-purpose programmability of graphics hardware; the OptiX engine of Parker and colleagues in 2010 [10] made the graphics processor a practical ray-tracing machine, and dedicated ray-tracing circuitry in consumer hardware followed at the end of that decade. Real-time renderers trace only a few paths per pixel and accept the resulting noise, then remove it with a denoising step; the modern engineering problem is as much the reconstruction of a clean image from sparse noisy samples as it is the tracing itself. The synthesis of the field into a single physically grounded framework is set out at length by Pharr, Jakob and Humphreys [11], and rests on radiometric foundations and the diffuse reflectance law that Lambert had described as early as 1760 [1].

Two threads run through this history and are worth stating plainly. The first is that rendering moved from computing only what is directly visible to computing the full transport of light between surfaces. The second is that it moved from deterministic point samples to statistical estimation, accepting noise as the price of a method general enough to handle any light path. The sections that follow build the subject up in the order of its logic rather than its history, but each piece can be located on those two threads.

Return to contents

4Ray and object intersection

Every operation in a ray tracer rests on one geometric question: given a ray, which surface does it strike first, and where. A ray is the set of points reached by starting at its origin and moving a non-negative distance along its unit direction. To intersect it with an object is to find the values of that distance at which the moving point lies on the object's surface.

For a circle, or in three dimensions a sphere, the surface is the set of points lying exactly one radius from the centre. Substituting the moving point of the ray into that condition gives an equation in the distance parameter, and because the direction vector has unit length the equation is an ordinary quadratic. Its discriminant settles the geometry at once. A negative discriminant means the ray misses the object entirely. A zero discriminant means the ray grazes it tangentially at a single point. A positive discriminant gives two solutions, the nearer being where the ray enters the object and the farther where it leaves. A flat surface such as a plane yields instead a linear equation with a single solution, found by asking how far along the ray its origin must travel for the component of motion toward the plane to close the initial gap.

A scene contains many objects, and the renderer must intersect the ray with each, discard any solution lying behind the origin or beyond a surface already found, and keep the nearest. The figure above shows this: a ray aimed across several circles meets them at a number of points, and only the nearest of those points is the surface the camera would actually see. The crossings behind it are real intersections, but they lie on surfaces hidden from view.

This nearest-hit search, repeated for every primary ray and every secondary ray, is the dominant cost of rendering. A scene of many objects tested exhaustively against every ray is prohibitively slow, which is why practical renderers organise the geometry into an acceleration structure, most often a bounding volume hierarchy, that lets a ray reject large groups of objects without testing each one. The structure changes the speed of the search but not its meaning, and this document keeps to the exhaustive test so that the geometry stays plain.

A secondary ray begins on a surface, and a naive intersection test will report that surface itself as the ray's nearest hit, at a distance of essentially zero. The resulting speckled artefact is known as shadow acne. The standard cure is to ignore intersections closer than a small threshold, so that a ray is never caught by the surface it was launched from. The figures in this document apply exactly that threshold.

5Shading: the surface and the light

Finding the point a ray strikes answers where, but not what colour. Shading is the computation that turns a hit point into the quantity of light leaving it toward the eye. It depends on the geometry at the point, on the light sources, and on the material of the surface.

The geometric quantity that shading needs is the surface normal, the direction perpendicular to the surface at the hit point. For a circle or a sphere the normal is simply the direction from the centre to the hit point. The normal matters because the brightness a surface receives from a light depends on the angle at which the light arrives. A surface facing a light directly intercepts the full strength of its illumination; the same surface turned obliquely spreads that illumination across a larger area and so is dimmer; turned past ninety degrees it faces away and receives nothing at all.

For a matte, or diffuse, surface this relationship is Lambert's cosine law, described by Lambert in his Photometria of 1760 [1]. A diffuse surface scatters the light it receives equally in all directions, so its apparent brightness does not depend on where the eye is, only on how much light it receives; and the light it receives from a source is proportional to the cosine of the angle between the surface normal and the direction to the source. That cosine is computed directly as the dot product of the two unit vectors.

Many surfaces are not perfectly matte. A polished or wet surface reflects more light in directions near the mirror direction, producing a bright highlight that does move as the eye moves. The model used in the figure for that highlight is the one introduced by Phong in 1975 [3]: the strength of the specular term is the cosine of the angle between the direction to the eye and the mirror reflection of the direction to the light, raised to a power. That power, the shininess exponent, controls the tightness of the highlight; a large exponent confines it to a small, sharp spot, and a small one spreads it into a broad sheen.

The figure shades a circular object lit by a movable light, colouring each point of the disc according to the normal in its radial direction, and drawing at a probe point the three vectors that shading depends on: the normal, the direction to the light and the direction to the eye. Two terms are still missing from this account. One is the contribution of light that has bounced off other surfaces, which a single light-and-surface calculation cannot include; the figure stands in for it with a small constant ambient term, an admitted placeholder that the eighth section replaces with something principled. The other is the effect of objects that block the light, which is the subject of the next section.

6Shadows

The shading of the previous section asks whether a surface is turned toward a light, but not whether anything stands between them. A point may face a light squarely and still be dark, because another object intercepts the light before it arrives. Determining this is the role of the shadow ray.

A shadow ray is cast from the hit point toward a light source. If it reaches the light without meeting any object, the point is lit by that light and the shading of the previous section applies. If instead the shadow ray strikes some object before it covers the distance to the light, the point lies in shadow with respect to that light, which contributes nothing to it. A point may be lit by some lights and shadowed from others, and its final shade is the sum over only the lights it can see.

The shadow test differs in a small but worthwhile way from the intersection search of the fourth section. For visibility the renderer needs the nearest surface, but for a shadow it needs only to know whether any surface at all lies on the segment between the point and the light. The search can therefore stop at the first intersection it finds rather than continuing to identify the closest, an optimisation known as an any-hit query as opposed to a closest-hit one.

The figure casts shadow rays from points along a floor toward a movable light and shades each point of the floor by whether its shadow ray is clear or blocked. Because the light is treated as a single point, every floor point is either fully lit or fully dark and the shadow has a hard edge. A real light source has area, and a point near the edge of a shadow can see part of the source and not the rest, which produces the soft gradation of a penumbra. Rendering that softness requires casting many shadow rays toward different points on the area of the light and averaging the fraction that arrive, which is the distributed sampling that the literature review attributed to Cook, Porter and Carpenter [5] and which the eighth section treats as one case of a general method.

7Reflection and recursion

A mirror does not have a colour of its own in the way a matte surface does. What the eye sees in a mirror is whatever lies along the direction in which the mirror redirects the eye's line of sight. To render a mirrored surface, the renderer must therefore find that redirected direction and discover what is along it.

The redirected direction is the reflection of the incoming ray about the surface normal, given by a short vector formula: the incoming direction, less twice its component along the normal. Once that reflected direction is known, the question of what lies along it is the very question a ray tracer already answers. The renderer casts a new ray, the reflected ray, from the hit point in the reflected direction, and traces it exactly as it traced the primary ray. This is Whitted's central observation [4]: rendering a mirror is rendering the scene again, seen from the mirror. A transparent surface is handled in the same spirit, with a refracted ray bent across the surface according to Snell's law in place of, or alongside, the reflected one.

Because the reflected ray is traced like any other, it may itself strike a mirror and spawn a further reflected ray, and so the process is recursive. The recursion needs a stopping rule. Two facing mirrors would reflect a ray between them without end, and even in less extreme scenes each successive bounce contributes a progressively smaller amount of light to the final pixel, so continuing indefinitely buys nothing. Renderers therefore impose a maximum recursion depth, a limit on the number of bounces, and accept the small error of discarding the contribution of any path longer than the limit in exchange for a guarantee that the computation terminates.

The figure traces a single ray through a scene of mirrored circles and draws its entire bounce path, with the recursion depth set by a slider. Lowering the depth cuts the path short; raising it lets the ray explore further, until it either escapes the scene, meets a matte surface that absorbs it, or reaches the limit. Each ray here remains a single deterministic sample sent in one exact direction, so the reflections it produces are mirror-sharp. A glossy surface, which reflects a slightly blurred image, would require many reflected rays spread around the mirror direction, which returns once more to the theme of sampling that the next section makes central.

8The rendering equation and global illumination

Each of the preceding sections computed one part of the light at a surface point: the direct contribution of a light source, the presence or absence of a shadow, the contribution of a mirror reflection. Kajiya's rendering equation [7] states the whole of it at once. In words, the light leaving a surface point in a particular direction equals the light the surface emits in that direction, which is zero unless the surface is itself a light, plus the sum over every direction from which light could arrive of the incoming light from that direction multiplied by the fraction of it the surface scatters into the outgoing direction.

That scattering fraction is a property of the material, called the bidirectional reflectance distribution function. It is what distinguishes a matte surface, whose function is nearly flat, from a mirror, whose function is sharply concentrated about the reflection direction. The equation is an integral equation, and its essential difficulty is that the unknown quantity appears on both sides: the light arriving at a point is the light leaving other points, which is what the equation is trying to determine. Light is globally coupled, every surface potentially illuminating every other, and this mutual coupling is what the phrase global illumination names.

Indirect illumination
Light that reaches a surface only after having scattered off one or more other surfaces, rather than arriving from a source directly. It is the term that recursive ray tracing omitted: the soft fill light in a room no source reaches directly, and the tinting of one surface by the colour of a nearby one. The rendering equation includes it as a matter of course, because its integral is taken over light arriving from every direction, not only from the directions of the sources.

An integral over all directions has no closed-form answer for any scene of interest, so it must be estimated numerically. The method Kajiya proposed, and the one that dominates high-quality rendering today, is Monte-Carlo path tracing. To estimate the integral at a point, the renderer chooses directions at random, follows a ray along each, finds what it strikes, and recursively estimates the light arriving from there, then averages the results. Averaged over enough random paths, the estimate converges to the true value of the integral; the method is unbiased, meaning that it is correct on average rather than merely close.

The price of this generality is noise. A small number of random paths gives an estimate that is correct in expectation but visibly speckled, and the speckling diminishes only as the square root of the number of paths: reducing the noise by half requires four times as many samples, and reducing it to a tenth requires a hundred times as many. The figure above makes that law visible, shading a floor by Monte-Carlo estimation and reporting how the measured noise falls as the sample count is raised. Cook's analysis of stochastic sampling [6] is the basis for choosing samples well, and the methods of Jensen [8] and of Veach and Guibas [9] are, in their different ways, attempts to reach a given level of noise with fewer samples.

The figure is honest about its scope. It computes the light arriving at each floor point directly from the source together with a single further bounce off the other objects, which is the honest core of the rendering equation rather than its full recursive depth; a production path tracer continues the recursion for many bounces. The figure does, however, use the genuine Monte-Carlo estimator, so the noise it shows and the rate at which that noise falls are real. The same square-root law explains why real-time renderers, which can afford only a few samples per pixel, produce an image too noisy to show directly and must pass it through a denoiser, so that the modern problem is partly one of reconstructing a clean image from a sparse and noisy estimate.

9Conclusion

A renderer answers, for every pixel, a single question: what light arrives at the camera along the line of sight through that pixel. This document has assembled the answer one piece at a time. The light is found by tracing a ray backward from the eye, because tracing forward from the sources would waste almost all of its effort. The ray is intersected with the geometry of the scene to find the nearest visible surface. That surface is shaded by combining its orientation, through the surface normal, with the directions to the lights, using the cosine law for the matte component and the Phong model for the highlight. A shadow ray decides which lights actually reach the point. A reflected ray, traced recursively under a depth limit, brings in what is seen in mirrors. And all of these computations are recognised, finally, as partial evaluations of one integral equation, the rendering equation, whose general solution is the statistical averaging of random light paths.

The literature review placed that assembly in time and argued that it was not arbitrary. Each method removed a definite failure of the one before it: ray casting followed light past the image plane where scanline methods could not; recursive tracing followed it past the first surface where ray casting stopped; distributed tracing replaced single samples with bundles where recursive tracing could render only sharp phenomena; the rendering equation named the indirect light that all of them had omitted; and the methods that followed it sought to estimate that equation with less noise. The two long movements of the subject, from local visibility toward global transport and from deterministic samples toward statistical estimation, are visible in every step of that sequence.

A reader who has followed the argument is equipped to read further into the subject, because the topics that remain are refinements of the pieces built here. The study of reflectance functions refines the shading model; importance sampling refines the choice of random directions so that effort follows the light; acceleration structures refine the intersection search; and denoising refines the sparse, noisy estimate into a finished image. Each is an answer to a measurable shortcoming of what this document has described, which is the same pattern the literature review traced through the history of the field. The quantity carried along every ray, and computed in the end for every pixel, is the radiance that gives the method, and this monograph, its name.

Return to contents

RReferences

  1. Lambert, J. H. (1760). Photometria, sive de mensura et gradibus luminis, colorum et umbrae. Augsburg: Eberhard Klett.
  2. Appel, A. (1968). Some techniques for shading machine renderings of solids. Proceedings of the AFIPS Spring Joint Computer Conference, 32.
  3. Phong, B. T. (1975). Illumination for computer generated pictures. Communications of the ACM, 18(6).
  4. Whitted, T. (1980). An improved illumination model for shaded display. Communications of the ACM, 23(6).
  5. Cook, R. L., Porter, T. and Carpenter, L. (1984). Distributed ray tracing. Computer Graphics (Proceedings of SIGGRAPH), 18(3).
  6. Cook, R. L. (1986). Stochastic sampling in computer graphics. ACM Transactions on Graphics, 5(1).
  7. Kajiya, J. T. (1986). The rendering equation. Computer Graphics (Proceedings of SIGGRAPH), 20(4).
  8. Jensen, H. W. (1996). Global illumination using photon maps. Rendering Techniques (Proceedings of the Eurographics Workshop on Rendering).
  9. Veach, E. and Guibas, L. J. (1997). Metropolis light transport. Proceedings of SIGGRAPH.
  10. Parker, S. G., Bigler, J., Dietrich, A., Friedrich, H., Hoberock, J., Luebke, D., McAllister, D., McGuire, M., Morley, K., Robison, A. and Stich, M. (2010). OptiX: a general purpose ray tracing engine. ACM Transactions on Graphics, 29(4).
  11. Pharr, M., Jakob, W. and Humphreys, G. (2016). Physically Based Rendering: From Theory to Implementation (3rd ed.). Cambridge, Massachusetts: Morgan Kaufmann.