Tags: CG

  • 波动光学渲染:全波参考模拟-学习笔记-4

    Wave Optics Rendering: Full Wave Reference Simulation - Study Notes - 4

    Today, let’s take a quick look at the Sig23 paper on a full-wave reference simulator for calculating surface reflections.

    Keywords: Introduction to graphics, wave optics rendering, BEM, AIM, BRDF
    Wave Optics Render, Full Wave Reference Simulator

    original:https://zhuanlan.zhihu.com/p/1471147574

    Preface

    Let me briefly summarize my current understanding.

    Computer graphics based on wave optics is undoubtedly a huge challenge for graphics researchers. It not only requires computational processing of electromagnetic waves, but also involves some theories of quantum mechanics. It is not easy to bridge the gap between physics and graphics. In graphics based on wave optics, the propagation of light in a medium is no longer just a straight line, but will exhibit various characteristics such as spin, deflection and diffraction due to different wavelengths. From the optical rotation caused by soap bubbles, polarizers, oil film gloss or sugar media to how radio radiation propagates between urban buildings, all are inseparable from the theory of wave optics.

    There are also many experts on Zhihu who have shared very detailed basic teaching on wave optics rendering theory.

    However, the reading threshold is high and it is too far from practical application. Wave optics rendering involves too much mathematical and physical knowledge, and even if you read the paper of Professor Yan and his doctoral students line by line, it is difficult to make any progress.

    The holy grail of graphics is ray tracing, and global wave optics rendering is the holy grail of graphics holy grails.

    At Sig21, Shlomi proposed combining path tracing with physical optics to achieve a realistic simulation of electromagnetic radiation propagation. There are many methods for calculating electromagnetic wave transmission, from high-precision but computationally intensive wave solvers to fast but inaccurate geometric optics methods. The following figure shows the current calculation methods for electromagnetic wave transmission. The left one is the most accurate, and the right one is the fastest.

    Wave Solvers focus on finding exact solutions to Maxwell's equations, but are not practical for large scenes, which are usually done with FDTD, BEM or FEM.

    PO is based on high-frequency approximation of electromagnetic wave calculations, but it is barely sufficient for the frequency of visible light. PS: [Xia 2023]'s black dog hair belongs to the Physical Optics method. The full-wave reference in this article should still belong to the Wave Solvers method, but it uses some ideas of PO (equivalent current, etc.) on the basis of BEM.

    In addition to PO, there is another method between Wave Solvers and Geometrical Optics, called Hybrid GO-PO. I personally think it should be called a hybrid method of geometric optics and physical optics. The Uniform Theory of Diffraction (UTD) incorporates the diffraction effect into geometric optics to calculate the transmission of electromagnetic waves under high-frequency conditions. In my opinion, UTD compensates for the deficiencies of the boundary conditions of geometric optical rays by calculating the diffraction coefficient, which means that the rays of geometric optics can also turn. This operation is very practical in the field of radar detection antenna design. In addition to UTD, Hybrid GO-PO also involves a technology called Shooting and Bouncing Rays (SBR). This technology simulates multiple reflections of rays on the surface of an object, which is also based on geometric optics.

    I personally think that understanding light as a plane sine wave is somewhat limited. For example, the 3b1b optics series video considers the electric field of light as a plane sine wave.

    Although it can explain most phenomena and is very suitable for introductory learning, for the study of wave optics rendering, simply describing the electric field of light as a plane sine wave cannot further explain, for example, the Gaussian beam mentioned in this paper.

    But I still strongly recommend readers who are not familiar with wave optics to watch this series of videos first.

    The video shows the special wave optical phenomenon that light "rotates" when passing through a right-handed chiral medium, and popularizes a series of interesting phenomena in wave optics and the principles behind them.

    Very thought-provoking.

    Finally, it even talks about how to use matter waves to construct holographic images.

    In classical electromagnetic field theory, we usually usePlane WaveExpand the electromagnetic field, and the photons of each mode are spatiallyNonlocalWith infinite spatial extension,A photon is in a mode with a well-defined frequency and wave vector., so it "fills" the entire area of the plane wave in space. This is very common for electromagnetic field patterns in free space, but this plane wave is not suitable for describing localization.

    Another way to expand isLocal wave packetThis way of understanding allows us to arrange the electromagnetic field in space, that is, to form a series of wave packets with a certain width.

    img

    Simply understanding light as a sine wave actually violates the uncertainty principle. If light is regarded as a sine wave, it means that the light is completely monochromatic, but monochromatic light is impossible in reality. The effect of optical signals is basically in the frequency domain, and Fourier transform is required to convert the spatial domain to the frequency domain. The bandwidth-time uncertainty principle states that the narrower the bandwidth, the longer the signal will be in time. Conversely, the wider the bandwidth, the shorter the time length of the signal.

    In classical electromagnetic theory, a wave packet can correspond to a region where energy is concentrated.

    But please note that photons cannot be simply understood as a wave packet, but as a probability wave.Probability wave functionDescribes the probability of a photon's existence. The more concentrated the wave packet, the more obvious the particle nature. This is the wave-particle duality explained by quantum electrodynamics. A photon is not necessarily in only one wave packet, it can also be described as a superposition of multiple wave packets. Because the photon state is essentially the excitation of the quantum field,Allows superposition of wave packets at different locations.

    That is to say,A photon can "span" multiple wave packets, that is, its wave function can exist in the form of multiple wave packets in space, rather than being confined to a specific location. When a wave packet mode has a photon, this wave packet can be regarded asLowest excited state; If there are multiple photons on the wave packet (higher-order excited states), higher energy will be reflected.

    An electromagnetic beam mentioned in this articleGaussian beamIt is a specific electromagnetic wave solution and can be regarded as a wave packet.Gaussian beamIt is a structure with stable amplitude and phase.Single wave packet, which shows a Gaussian distribution on the cross section. This is different from the plane wave solution we usually use in classical electromagnetic theory, because the wavefront of the Gaussian beam has a changing curvature and is not an infinitely extended plane.

    Ensemble of wavesRefers to a collection of multiple waves that may have different frequencies, phases, or propagation directions. If we consider a system in which multiple independent wave packets (such as multiple pulsed laser beams) are superimposed on each other, these wave packets can be understood as aEnsemble of wavesIn other words, if multiple uncorrelated wave packets, especially those with random phases, are superimposed on each other, they can statistically constitute a wave ensemble.

    When we expand the electromagnetic field into a series of wave packets, we can regard each wave packet as a random event, and their arrival time and phase are random variables. For a collection of multiple wave packets, we can observe the characteristics of these wave packets at different times and locations. In a statistical sense, usingEnsemble meanTo analyze the energy distribution and wave behavior of light.


    $$
    \langle U(\vec{r}, t) \rangle = \lim_{T \to \infty} \frac{1}{2T} \int_{-T}^{T} U(\vec{r}, t) \, dt
    $$

    Since I am not a physicist, I do not approve of writing the differential before the integral formula (

    As mentioned above, light can be viewed as a collection of multiple wave packets. There may be phase and time differences between different wave packets, so it is necessary to introduceCross-correlation functionandInterference Functionto describe the coherence and relative phase between wave packets at two different locations, providing aSimilarity in time and spaceIf we sayEnsemble meanUsed to describe the average behavior of a signal in a statistical sense, thenCross-correlation functionIt describes the time averageThe correlation between different locations and different timesandVolatility Correlation.

    Specifically, the cross-correlation function describes the coherence of the waves at positions $\vec{r}_1$ and $\vec{r}_2$. If the two waves are coherent, then the Gamma value will be large:


    $$
    \Gamma(\vec{r}_1, \vec{r}_2, \tau) = \langle U(\vec{r}_1, t) U^*(\vec{r}_2, t + \tau) \rangle
    $$


    The cross spectral density (CSD) matrix $W(\vec{r}_1, \vec{r}_2, \omega)$ is the cross-correlation functionFourier Transform, expressed in the frequency domainThe correlation between two positions.

    Remember the autocorrelation function we discussed in Xia2023's article Black Dog Hair? The cross-correlation function requires two signals to describe, while the autocorrelation function only has one signal. From another perspective, the autocorrelation function (ACF) is a special case of the cross-correlation function. The autocorrelation function describesThe correlation between the signal and itself at different times or locations, and the cross-correlation function isCorrelation between two different signals or the same signal at different locationsThe cross-correlation function and the cross-spectral density are a pair of Fourier transform pairs, and the autocorrelation function and the power spectral density are a pair of Fourier transform pairs.

    The cross spectral density (CSD) and the mutual coherence function describe the coherence of the fluctuations between different locations.Radiance Cross-Spectral Density (RCSD)It is a generalization of CSD, describing the correlation between light radiation (i.e., energy density) at different positions and directions. It can be understood that RCSD provides a coherence description similar to CSD based on radiation measurement. In the formula, $L(\vec{r}_1, \vec{r}_2, \omega)$ represents the radiation intensity coherence between positions $\vec{r}_1$ and $\vec{r}_2$ at frequency $\omega$.

    Radiative cross-spectral density transfer equation (SDTE)With classicLight Transport Equation (LTE)Similar, but more suitable for wave optics. LTE describes the transmission of optical radiance from point to point, while SDTE uses the RCSD function to describe the propagation of optical radiation, which is equivalent to treating the transmission as coherent transmission between regions.

    The RCSD in SDTE expresses the propagation of radiation in the form of an integral between regions, which can be understood as replacing the traditional reflection and scattering with the RCSD matrix and diffraction operator. And note that SDTE is based onRCSD function, rather than the specific light intensity value, which is quite different from LTE.

    In further discussion of the useBoundary Element Method (BEM)andAdaptive Integral Method (AIM)Before we discuss the accelerated full-wave reference simulator, let's first briefly review the previous article. The previous article introduced the PO method and SDTE/RCSD theory. These methods are used for different scattering calculation needs, but their basic theories and scope of application are different. This article will discuss a method that provides high-precision surface scattering simulation by combining BEM and AIM.

    To sum up in the original author's words, Wave optics is a very new branch in physically based rendering. Although wave optics phenomena can be seen everywhere in life, their impact on the picture is not very big. There is actually a lot of room for improvement in this direction.

    Other related references:


    A Full-Wave Reference Simulator for Computing Surface Reflectance

    Paper Homepage:

    https://blaire9989.github.io/assets/1_BEMsim3D/project.html

    ACM Home Page:

    https://dl.acm.org/doi/10.1145/3592414

    ACM Citations:

    Yunchen Yu, Mengqi Xia, Bruce Walter, Eric Michielssen, and Steve Marschner. 2023. A Full-Wave Reference Simulator for Computing Surface Reflectance. ACM Trans. Graph. 42, 4, Article 109 (August 2023), 17 pages. https: //doi.org/10.1145/3592414

    Speech Report

    Compared to the more commonly used ray tracing techniques, wavelight simulations are more time-consuming but also more accurate.

    For example, the microscopic scratches on the metal surface and the hair fiber structure cannot be rendered by traditional optical models, and the colorful color effects we observe in real life cannot be rendered.

    Wave optics-based rendering is a difficult problem because solving Maxwell's equations requires a lot of complex calculations. Existing wave-based appearance models usually adopt some approximation methods.

    One approximation is the scalar field approximation.

    In the wave scattering problem, the electric field and the magnetic field are different vector field quantities, and light with different polarizations consists of field quantities pointing in different directions.

    Some approximate models replace these two vector field quantities with a single scalar function, thus being able to calculate the intensity of the light energy but giving up modeling polarization.

    Another approximation is the first-order approximation. This assumes that the light reflects only once from each part of the model structure, ignoring multiple reflections. However, there are many cases where these approximations are not valid.

    For example, Yu et al., in collaboration with Dr. Lawrence’s group at Penn State University, created surfaces with cylindrical cross-sections that cause multiple light reflections and produce structural colors, phenomena that cannot be well understood or predicted using approximate models.

    The authors wanted to characterize surface scattering as accurately as possible by calculating the bidirectional reflectance distribution function (BRDF).

    Existing models all use various approximations, such as ray-based, scalar, or first-order approximation models. Without a reference quality BRDF, it is difficult to see what each reflection model is missing or what scenarios it is suitable for.

    The authors' solution is to build a 3D 4-way simulation to compute the BRDF of surfaces with well-defined microgeometry.

    Claims to compute a reference-quality BRDF for surface samples without using any approximations.

    In terms of speed, dddd.

    The authors next describe how their simulation works and how it relates to the BRDF.

    First, in the left picture, we input a surface sample (defined as a height field) and an incident direction (expressed on the projected hemisphere). We define an incident field propagating toward the surface and calculate a scattered field from the surface.

    In the middle picture, the beam is the incident field, and the scattered field in the background is also shown in this picture.

    The output is the BRDF model for a given incident direction. Each point in the hemisphere represents an outgoing direction, and the color represents the color of the reflected light in that direction. The BRDF is expressed in RGB colors, which are converted from spectral data.

    For many surfaces with low roughness, the reflection pattern is symmetric around the specular direction and changes as the incident light direction moves.

    The following is a discussion on how to use the boundary element method to solve the problem of signal scattering only on the surface, thereby reducing the dimensionality of the problem.

    The surface signal is the surface current solved from Maxwell's equations. After discretization, the problem is constructed as a linear system, and the surface current and scattered field are solved.

    To make the computation feasible, the authors symmetrize the linear system and use a minimum residual solver suitable for symmetric matrices.

    In addition, the matrix-vector multiplication is accelerated using the adaptive integration method, which is an acceleration method based on the fast Fourier transform and was originally used in radar calculations.

    Most of the code uses the Cuda C++ package for acceleration.

    Next, some results are shown, illustrating how the BRDF it computes compares to the BRDF derived from previous methods.

    [Yan 2018] use scalar field approximation BRDF models that only consider one refraction.

    [Xia 2023] This paper uses vector field quantities but only considers one refraction.

    The most accurate method is ours, which not only uses vector field quantities, but also takes into account reflections of all orders.

    In the above figure, each incident direction corresponds to five BRDFs, representing different calculation methods.

    A relatively smooth material covered with a bunch of isotropic bombs.

    The first row shows the normal instance, and the reflection pattern is pretty much centered.

    In the second row, the pattern shifts to the left because the incident light comes from a certain oblique direction.

    Since the surface is not too rough, the results of the five methods are very similar.

    Another material has some corners (corner cubes). The three faces of each corner cube will reflect the light multiple times, making the reflected light return in the direction of incidence. This is called retroreflection.

    Our simulator can also simulate this phenomenon. The four methods on the left all failed.

    The reason is that if only one reflection is considered, when the light hits one face of the corner cubes, it will be predicted to go down into the lower hemisphere.

    The final example is a surface covered with spherical pits.

    Multiple reflections occur due to the high slopes of the surface at the edges of the pits.

    Different methods show obvious differences.

    You can see the extra lobe predicting. (The part slightly to the right in the middle)

    In addition, it is brighter overall.

    These differences arise from interference between reflections of different orders.

    Finally, a technique for efficiently computing the BRDF of a very large number of densely sampled directions is briefly introduced.

    If the surface to be simulated is large, this will be slow and require a lot of GPU memory.

    But the calculation is linear and can decompose a large surface into multiple smaller sub-areas.

    The incident field is projected onto each sub-area, the smaller sub-area simulation is performed first, and then the scattered fields are integrated to obtain the BRDF of the entire large-area surface.

    By applying different complex value scale factors to the scattered fields of different sub-areas, the BRDF of a large surface corresponding to different incident directions can be synthesized.

    This is because applying an appropriate phase shift to the local incident field in each sub-region produces a total incident field with a different net direction on the surface. In this figure, the incident field propagates vertically. If the same field with five foci at different spatial locations is superimposed, a spatially wider field is obtained, still propagating in the vertical direction. If these fields are linearly combined and an appropriate complex-valued scaling factor is applied to the field in each sub-region, the overall field can be made to propagate in a slightly tilted direction.

    Here we explain why different complex value scale factors can produce different incident directions.

    • This factor can adjust the amplitude and phase of the wave. For example, if two stones are thrown into the water at the same time, the waves on the water will be stronger. If one stone is thrown a little later, the ripples may cancel each other out (destructive interference). This factor controls the time of throwing the stones. By adjusting the phase to control the superposition of waves, the waves can be "guided" to propagate in different directions. Search for "Beamforming" in detail. It is widely used in radar, wireless communication, sonar and other fields.

    These three pictures represent the wavefront of the light wave, i.e. the wave crest, which can be understood as the waveform cross section of the light during propagation.

    In the upper figure, the incident field distribution is concentrated in the center when the light is incident perpendicular to the surface. The light field is concentrated in the center and propagates in the vertical direction (i.e. the direction of the yellow line in the middle).

    At the bottom left, the effect of superposition of multiple identical incident fields, but with the same phase (i.e. no phase difference) when superimposed. The addition of multiple incident fields makes the entire field wider in space, but the propagation direction remains vertical.

    In the lower right corner, the superposition effect of multiple incident fields is shown, but a phase difference is added during the superposition, which is equivalent to "offsetting" the direction of the incident field, showing an inclined direction.

    In the demo, two videos show the movement of BRDF patterns. (I'm too lazy to make GIFs)

    Finally, the big guys took a photo with their hands making a heart shape.

    paper

    mineFirst articleHaving briefly introduced the content and results of this work, we will now move directly to the theoretical derivation (Section 3-5).

    3 FULL-WAVE SIMULATION

    The holistic approach starts with a surface model, described by a height field and its material properties (such as the complex refractive index), and specifies a target point.

    To compute the BRDF for a given incident direction, an incident field is defined that propagates from that specific direction to the target point.

    This incident field is used as input and processed through a surface scattering simulation to solve for the corresponding scattered electromagnetic field.

    In this section (FULL-WAVE SIMULATION), we will focus on the principles of BEM in application scenarios. The next section will explain how to efficiently implement the BEM algorithm, and the last section will explain how to combine multiple simulation results to synthesize a BRDF that is densely sampled in the incident and scattering directions.

    Let me start with a symbol table to scare you.

    3.1 Boundary Element Method: The Basics

    The boundary element method (BEM) mainly solves the scattering problem of a single frequency, that is, how electromagnetic waves (including electric and magnetic fields) of a specific frequency are reflected and scattered at the boundaries of different media. The boundary here divides the space into two uniform regions, and the material properties of the two regions (medium parameters at the incident site) are represented by ($\epsilon_1, \mu_1$) and ($\epsilon_2, \mu_2$). Among them, $\epsilon$ represents the dielectric constant (dielectric constant), and $\mu$ represents the magnetic permeability (magnetic permeability coefficient).

    In this approach, we deal with complex-valued fields that contain both amplitude and phase information (i.e., the propagation state of the wave). To simplify the formula, we assume that all waves are "time-harmonic" - that is, the waves change with a specific period over time. Throughout the text, the $e^{j\omega t}$ term is omitted to simplify the presentation.

    3.1.1 Maxwell's Equations and Surface Currents

    First, Maxwell's equations describe how electric fields (E) and magnetic fields (H) interact with each other, determining how light waves propagate and scatter between different materials. For simplicity, this is expressed in a "time-harmonic" form:

    $$ \begin{align} \nabla \times \mathbf{E} &= -\mathbf{M} – j \omega \mu \mathbf{H} \\ \nabla \times \mathbf{H} &= \mathbf{ J} + j \omega \epsilon \mathbf{E} \tag{1} \end{align} $$


    The left side of the equal sign describes the degree of "rotation" of the electric and magnetic fields in space. $M$ and $J$ areSurface current density(imaginary current), respectively representing the density of magnetic flow and current (electric and magnetic current densities). This formula can be understood as that when the electric field "rotates" near the boundary, changes in magnetic flow and magnetic field occur; the rotation of the magnetic field also causes changes in the electric field and current.

    The core idea of the boundary element method is:Introducing surface currents on boundaries, using these currents to indirectly describe the field distribution without having to calculate all points in each region. The three-dimensional problem is reduced to a two-dimensional problem on the boundary.

    3.1.2 Source-Field Relationships

    How does an imaginary electric current on the surface (the "source" of the electromagnetic waves) produce the scattered electromagnetic field (the "field")?

    As shown in Fig. 2., in the region $R_1$, the total field (incident field and scattered field) are represented as $E_1$ and $H_1$ respectively;

    $$ \begin{align} &\mathbf{E}_1 = \mathbf{E}_i + \mathbf{E}_s \\ &\mathbf{H}_1 = \mathbf{H}_i + \mathbf{H}_s \tag{2} \end{align} $$


    In the region $R_2$ , the total field is denoted by $E_2$ and $H_2$ . The upper scattered field is generated by the upper electromagnetic current; the lower scattered field is generated by the lower electromagnetic current.

    In a homogeneous medium, Maxwell's equations can be written in integral form to describe how electric and magnetic fields are generated.

    \begin{align} \mathbf{E}(\mathbf{r}) &= -j \omega \mu (\mathcal{L} \mathbf{J})(\mathbf{r}) – (\mathcal{K } \mathbf{M})(\mathbf{r}) \\ \mathbf{H}(\mathbf{r}) &= -j \omega \epsilon (\mathcal{L} \mathbf{M})(\mathbf{r}) + (\mathcal{K} \mathbf{J})(\mathbf{r}) \tag{3} \end{align }


    The left side of the equal sign represents the electromagnetic field strength at $r$, that is, it describes the "effect" of the field. $\mathcal{L}$ and $\mathcal{K}$ are integral operators that represent how the field is generated from the surface current and magnetization. These two operators are defined as:

    $$ \begin{aligned} & (\mathcal{L} \mathbf{X})(\mathbf{r})=\left[1+\frac{1}{k^2} \nabla \nabla \cdot\right ] \int_V G\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \mathbf{X}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} \\ & (\mathcal{K} \mathbf{X})(\mathbf {r})=\nabla \times \int_V G\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \mathbf{X}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} \end{aligned} \tag{4} $$


    $G(r, r{\prime})$ is the three-dimensional Green's function for the scalar Helmholtz equation, defined as:

    $$
    G(\mathbf{r}, \mathbf{r}^{\prime}) = \frac{e^{-jkr}}{4 \pi r} \quad \text{where } r = |\mathbf{r } – \mathbf{r}^{\prime}|
    \tag{5}
    $$


    This function converts the source field of the scattering surface into the distribution of the electromagnetic field in the scattering area.

    The formula (11) in this paper is actually the same as that in [Xia 2023], but this paper implicitly incorporates the Green's function in the operator and the integration domain is wider. In essence, both describe how to generate the scattered electric field $E(r)$ from the current density $\mathbf{J}$ and the magnetic flux density $\mathbf{M}$.

    When solving Maxwell's equations, the Green's function is used to integrate the influence of various sources (such as current and charge) on the electromagnetic field in space. Assuming that the electromagnetic field changes with time in the form of $e^{j\omega t}$ (single frequency), a form similar to the Helmholtz equation can be obtained: $(\nabla^2 + k^2) \mathbf{E} = -j \omega \mu \mathbf{J}$ , which is actually a "frequency domain" form of Maxwell's equations. The Green's function is introduced to establish a source-field relationship, that is, the current $\mathbf{J}$ and the charge $\rho$ are used as "sources" to calculate the distribution of the electromagnetic field $\mathbf{E}$ and $\mathbf{H}$. Then the Green’s function satisfies the following equation: $(\nabla^2 + k^2) G(\mathbf{r}, \mathbf{r}{\prime}) = -\delta(\mathbf{r} – \mathbf{r}{\prime})$ , $\delta$ is the Dirac delta function, which represents the standard waveform generated by a “point source” in space, and describes the wave field excited by a “point source” in space. Through the Green’s function, the influence of any current distribution $\mathbf{J}$ on the field point $\mathbf{r}$ in space can be expressed! Then, the “current” or “charge” at each source point is diffused to the entire space through the Green’s function, producing a cumulative effect on each field point. $\mathbf{E}(\mathbf{r}) = \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{J}(\mathbf{r}{\prime}) d \mathbf{r}{\prime}$ This formula is the electric field expressed as an integral superposition of source currents. Finally, combine the integral form of Maxwell's equations with the idea of Green's function. For example, the integral form of the electric field can be expressed as: $\mathbf{E}(\mathbf{r}) = -j \omega \mu \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{J}(\mathbf{r}{\prime}) d \mathbf{r}{\prime} – \nabla \times \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{M}(\mathbf{r}{\prime}) d \mathbf{r}{\prime}$ . Through convolution, we can obtain the cumulative influence of the tiny current distribution at each source point on the field point, which is the result of the propagation and superposition of the integral of the Green’s function in space. The Green function is used as a convolution kernel to propagate the current distribution at each point in space to the target field point through integration, realizing the cumulative impact of the current at each source point in space on the entire field.

    The electric field in the dielectric regions $R_1$ and $R_2$ is expressed as formula (6) and (7) respectively, which include two operators.

    In area $R_2$:

    $$ \begin{align} & \mathbf{E}_1(\mathbf{r}) = -j \omega \mu_1 (\mathcal{L}_1 \mathbf{J}_1)(\mathbf{r}) – ( \mathcal{K}_1 \mathbf{M}_1)(\mathbf{r}) \\ & \mathbf{H}_1(\mathbf{r}) = -j \omega \epsilon_1 (\mathcal{L}_1 \mathbf{M}_1)(\mathbf{r}) + (\mathcal{K}_1 \ mathbf{J}_1)(\mathbf{r}) \end{align} \tag{6} $$


    In area $R_2$:

    \begin{align} \mathbf{E}_2(\mathbf{r}) &= -j \omega \mu_2 (\mathcal{L}_2 \mathbf{J}_2)(\mathbf{r}) – (\ mathcal{K}_2 \mathbf{M}_2)(\mathbf{r}) \\ \mathbf{H}_2(\mathbf{r}) &= -j \omega \epsilon_2 (\mathcal{L}_2 \mathbf{M}_2)(\mathbf{r}) + (\mathcal{K}_2 \mathbf{J}_2)(\mathbf{r}) \tag{7} \end{align}


    This gives the manifestations of electric and magnetic fields in different dielectric regions.

    In summary, this section transforms the Maxwell equations into integral expressions by generating scattered electromagnetic fields through hypothetical surface currents. Green's function transforms current and charge distribution into integral superposition of electromagnetic fields, showing the specific implementation of the source-field relationship. Finally, the expressions of the electric and magnetic fields in the regions $R_1$ and $R_2$ are given, showing the influence of different medium parameters on the field.

    3.1.3 Boundary Conditions

    When electromagnetic waves propagate at the boundary of two different media, reflection and refraction will occur. At this time, the energy of the wave cannot disappear out of thin air, but transitions smoothly at the interface. If the electric field or magnetic field is discontinuous at the boundary, an unrealistic energy jump will occur (that is, the energy suddenly disappears or increases), which violates the law of conservation of energy.

    Specifically, you can search: "Interface conditions for electromagnetic fields".

    Therefore, certain boundary conditions must be met at the boundary of the medium to ensure the continuity of the electromagnetic field and the conservation of energy. Specifically, when an electromagnetic wave propagates at the interface of two different media, the tangential components of the electric and magnetic fields need to maintain continuity at the boundary:


    $$
    \begin{aligned}
    & \mathbf{n} \times (\mathbf{E}_1 – \mathbf{E}_2) = 0 \\
    & \mathbf{n} \times (\mathbf{H}_1 – \mathbf{H}_2) = 0
    \end{aligned}
    \tag{8}
    $$


    And the net electromagnetic current density on the boundary is zero, that is, the electromagnetic current densities on both sides of the boundary are opposite in direction and equal in magnitude.


    $$
    \begin{aligned}
    & \mathbf{J} = \mathbf{J}_1 = -\mathbf{J}_2 \\
    & \mathbf{M} = \mathbf{M}_1 = -\mathbf{M}_2
    \end{aligned}
    \tag{9}
    $$


    These two conditions must be met at the same time to avoid violating the laws of physical conservation.

    3.1.4 Integral Equations

    Combining the above formulas (6), (7), (8) and (9), we get the integral equations for the electric and magnetic fields, which are called the PMCHWT (Poggio-Miller-Chang-Harrington-Wu-Tsai) equations:

    $$
    \begin{aligned}
    & {\left[j \omega \mu_1\left(\mathcal{L}1 \mathbf{J}\right)(\mathbf{r})+j \omega \mu_2\left(\mathcal{L}_2 \ mathbf{J}\right)(\mathbf{r})+\left(\mathcal{K}_1 \mathbf{M}\right)(\mathbf{r})+\right.}\left.\left(\mathcal{K}_2 \mathbf{M}\right)(\mathbf{r})\right] {\tan } \\
    &=\left[\mathbf{E}i(\mathbf{r})\right]{\tan } \\
    & {\left[\left(\mathcal{K}1 \mathbf{J}\right)(\mathbf{r})+\left(\mathcal{K}_2 \mathbf{J}\right)(\mathbf {r})-j \omega \varepsilon_1\left(\mathcal{L}_1 \mathbf{M}\right)(\mathbf{r})-j \omega \varepsilon_2\left(\mathcal{L}_2 \mathbf{M}\right)(\mathbf{r})\right]{\ tan } } \\
    &=-\left[\mathbf{H}i(\mathbf{r})\right]{\tan }
    \end{aligned}
    \tag{10}
    $$


    These two equations also have names:

    • EFIE(Electric Field Integral Equation), electric field integral equation. $j \omega \mu_1 (\mathcal{L}_1 \mathbf{J})(\mathbf{r})$ and $j \omega \mu_2 (\mathcal{L}_2 \mathbf{J})(\mathbf{r})$ represent the contribution of current density $\mathbf{J}$ to the electric field in medium 1 and medium 2. $\mathcal{K}_1 \mathbf{M}$ and $\mathcal{K}_2 \mathbf{M}$ represent the contribution of magnetic current density $\mathbf{M}$ to the electric field in medium 1 and medium 2.
    • MFIE(Magnetic Field Integral Equation), magnetic field integral equation.

    In general, this is a boundary integral equation specifically used to solve electromagnetic scattering problems caused by dielectric objects. With this PMCHWT equation, the distribution of electromagnetic fields can be accurately calculated.

    3.1.5 Solving for Current Densities

    In this section, we need to use the PMCHWT equation above to calculate the distribution of "electric current" and "magnetic current" on the surface of the object. These distributions determine how the electromagnetic wave will be "reflected" or "refracted" when it hits the object.

    The surface current density $\mathbf{J}$ and magnetic flux density $\mathbf{M}$ are solved by discretizing the boundary element. A basis function $f_m(\mathbf{r})$ is defined for the discrete element, and the current density and magnetic flux density distribution are expressed by the basis function expansion method.


    $$
    \mathbf{J}(\mathbf{r}) = \sum_{m=1}^{N} I_{J_m} f_m(\mathbf{r}); \quad \mathbf{M}(\mathbf{r} ) = \sum_{n=1}^{N} I_{M_n} f_n(\mathbf{r})
    \tag{11}
    $$


    N is the total number of basis functions; $ I_{J_m}$ and $I_{M_n}$ are the unknown coefficients of the corresponding basis functions, representing the current and magnetic flux intensities on each unit.

    Through this basis function expansion, the continuous surface current density and magnetic current density are decomposed into a linear combination of a series of basis functions.

    In order to solve the unknown coefficients of the corresponding basis functions, the electric field integral equation (EFIE) and the magnetic field integral equation (MFIE) are transformed into a linear equation system. This is done using the Galerkin Method. The basic idea of this method is to apply the integral equation to each basis function and perform weighted averaging so that the integral equation holds true in the projection direction of each basis function. Simply put, this method involves discretization, finding the basis, and calculating the coefficients. A high-dimensional linear equation system can be simplified using linear algebra methods.

    In this way, the EFIE part of the original continuous form of the PMCHWT equation can be converted into a finite number of linear equations, and the problem can be transformed into solving the following matrix equation.


    $$
    \begin{bmatrix} A_{EJ} & A_{EM} \ A_{HJ} & A_{HM} \end{bmatrix} \begin{bmatrix} I_J \ I_M \end{bmatrix} = \begin{bmatrix} V_E \ V_H \end{bmatrix}
    \tag{12}
    $$


    in


    $$
    A_{\mathrm{EJ}}^{mn} =\int_S \mathbf{f}_m(\mathbf{r}) \cdot\left[j \omega \mu_1\left(\mathcal{L}_1 \mathbf{ f}_n\right)(\mathbf{r})+j \omega \mu_2\left(\mathcal{L}_2 \mathbf{f}_n\right)(\mathbf{r})\right] d \mathbf{r} \tag{13}
    $$

    $$
    A_{\mathrm{EM}}^{mn} =\int_S \mathbf{f}_m(\mathbf{r}) \cdot\left[\left(\mathcal{K}_1 \mathbf{f}_n\right )(\mathbf{r})+\left(\mathcal{K}_2 \mathbf{f}_n\right)(\mathbf{r})\right] d \mathbf{r} \tag{14}
    $$

    $$
    A_{\mathrm{HJ}}^{mn} =\int_S \mathbf{f}_m(\mathbf{r}) \cdot\left[\left(\mathcal{K}_1 \mathbf{f}_n\right )(\mathbf{r})+\left(\mathcal{K}_2 \mathbf{f}_n\right)(\mathbf{r})\right] d \mathbf{r} \tag{15}
    $$

    $$
    A_{\mathrm{HM}}^{mn} =-\int_S \mathbf{f}_m(\mathbf{r}) \cdot\left[j \omega \varepsilon_1\left(\mathcal{L}_1 \mathbf {f}_n\right)(\mathbf{r})+j \omega \varepsilon_2\left(\mathcal{L}_2 \mathbf{f}_n\right)(\mathbf{r})\right] d \mathbf{r} \tag{16}
    $$

    and


    $$
    V_{\mathrm{E}}^m =\int_S \mathbf{f}_m(\mathbf{r}) \cdot \mathbf{E}_i(\mathbf{r}) d \mathbf{r}\tag{ 17}
    $$

    $$
    V_{\mathrm{H}}^m =-\int_S \mathbf{f}_m(\mathbf{r}) \cdot \mathbf{H}_i(\mathbf{r}) d \mathbf{r}\tag {18}
    $$

    In formula (12), $I_J$ and $I_M$ need to be calculated.

    In a loose sense, formulas (13)-(16) respectively represent the contribution of each small block's current density to the electric field, the contribution of each small block's magnetic flux density to the electric field, the contribution of each small block's current density to the magnetic field, and the contribution of each small block's magnetic flux density to the magnetic field. Formulas (17)(18) respectively represent the "driving force" of the external incident electric field on the small block's current and the "driving force" of the external incident magnetic field on the small block's magnetic flux. Emphasize the elements in the matrix, such as $A_{EJ}^{mn}$, which is actually a double integral. Since the integration of the source point $\mathbf{r}{\prime}$ has been completed in $\mathcal{L}_1$ and $\mathcal{L}_2$, the original paper looks like a single integral.

    Although it is not written in the original paper, it is recommended that smart readers derive it by themselves. Try to expand formula (13) according to formula (4) mentioned above. I have tried to derive it here. Please correct me if there are any mistakes. First, substitute the two operators and pay attention to the position of the gradient operator here:


    $$
    \begin{aligned}
    A_{\mathrm{EJ}}^{mn} &= j \omega \mu_1 \int_S \mathbf{f}_m(\mathbf{r}) \cdot \left\{ \int_V G_1(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{\prime} + \frac{1}{k_1^2} \nabla \left[ \nabla \cdot \int_V G_1(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{\prime} \right] \right\} d\mathbf{r} \\
    &\quad + j \omega \mu_2 \int_S \mathbf{f}_m(\mathbf{r}) \cdot \left\{ \int_V G_2(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{\prime} + \frac{1}{k_2^2} \nabla \left[ \nabla \cdot \int_V G_2(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r }{\prime}) d\mathbf{r}{\prime} \right] \right\} d\mathbf{r}
    \end{aligned}
    $$

    Consider first one of the gradient terms:

    $$
    \int_S \mathbf{f}_m(\mathbf{r}) \cdot \nabla \left[ \nabla \cdot \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f }_n(\mathbf{r}{\prime}) d\mathbf{r}{\prime} \right] d\mathbf{r}
    $$

    Expand using vector distribution integral:

    $$
    \int_S \mathbf{f}m \cdot \nabla B \, dr = -\int_S B (\nabla \cdot \mathbf{f}_m) \, dr + \int{\partial S} B (\mathbf{A } \cdot \mathbf{n}) \, dr
    $$

    Among them, the divergence term $B$:

    $$
    B = \nabla \cdot \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{ \prime}
    $$

    But sorry, this is physics. Under the boundary conditions, the boundary terms are directly simplified to zero, and we get:

    $$
    \int_S \mathbf{f}_m \cdot \nabla B \, dS = -\int_S B (\nabla \cdot \mathbf{f}_m) \, dS
    $$

    For the divergence term $B$ , the divergence can be directly expanded:

    $$
    B = \nabla \cdot \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{ \prime} = \int_V (\nabla \cdot G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime})) d\mathbf{r}{\prime}
    $$

    One is a scalar function and the other is a matrix function, so according to the product rule of divergence:

    $$
    \nabla \cdot (G \mathbf{f}_n) = (\nabla G) \cdot \mathbf{f}_n + G (\nabla \cdot \mathbf{f}_n)
    $$

    But sorry, this is physics. Since the basis functions satisfy the divergence-free condition, this is a straightforward simplification:

    $$
    \nabla \cdot (G \mathbf{f}_n) = (\nabla G) \cdot \mathbf{f}_n
    $$

    At the same time, we note the symmetry of the Green's function $\nabla G(\mathbf{r}, \mathbf{r}{\prime}) = -\nabla{\prime} G(\mathbf{r}, \mathbf{r}{\prime})$, so:

    $$
    B = \nabla \cdot \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{ \prime} = -\int_V \nabla{\prime} G(\mathbf{r}, \mathbf{r}{\prime}) \cdot \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{\prime}
    $$

    Perform the distribution integral on this term as well:

    $$
    \int_V \nabla{\prime} G(\mathbf{r}, \mathbf{r}{\prime}) \cdot \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r }{\prime} = \int_V \nabla{\prime} \cdot (G \mathbf{f}_n) d\mathbf{r}{\prime} – \int_V G (\nabla{\prime} \cdot \mathbf{f}_n) d\mathbf{r}{\prime}
    $$

    But sorry, this is physics. The boundary terms are simplified again:

    $$
    \int_V \nabla{\prime} G(\mathbf{r}, \mathbf{r}{\prime}) \cdot \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r }{\prime} = -\int_V G(\mathbf{r}, \mathbf{r}{\prime}) (\nabla{\prime} \cdot \mathbf{f}_n(\mathbf{r}{\prime})) d\mathbf{r}{\prime}
    $$

    Finally, we get:

    $$
    B = \nabla \cdot \int_V G(\mathbf{r}, \mathbf{r}{\prime}) \mathbf{f}_n(\mathbf{r}{\prime}) d\mathbf{r}{ \prime} = \int_V G(\mathbf{r}, \mathbf{r}{\prime}) (\nabla{\prime} \cdot \mathbf{f}_n(\mathbf{r}{\prime})) d\mathbf{r}{\prime}
    $$

    The relationship between wave number $k_i$ and medium parameters:

    $$
    k_i^2 = \omega^2 \mu_i \varepsilon_i \quad \Rightarrow \quad \frac{1}{k_i^2} = \frac{1}{\omega^2 \mu_i \varepsilon_i}
    $$

    The same goes for the other gradient term, and finally we get the final expression $A_{\mathrm{EJ}}^{mn}$ :

    $$
    \begin{aligned} A_{\mathrm{EJ}}^{mn} &= j \omega \mu_1 \int_S \int_{V_1} \mathbf{f}m(\mathbf{r}) \cdot \mathbf{f }n(\mathbf{r}{\prime}) G_1(\mathbf{r}, \mathbf{r}{\prime}) d\mathbf{r}{\prime} d\mathbf{r} \\ &\quad – \frac{j}{\omega \varepsilon_1} \int_S \int{V_1} (\nabla \cdot \mathbf{f}m(\mathbf{r})) G_1(\mathbf{r}, \mathbf{r}{\prime}) (\nabla{\prime} \cdot \mathbf{f}n(\mathbf{r}{\prime})) d\mathbf{r}{\prime} d\mathbf{r} \\ &\quad + j \omega \mu_2 \int_S \ int{V_2} \mathbf{f}m(\mathbf{r}) \cdot \mathbf{f}n(\mathbf{r}{\prime}) G_2(\mathbf{r}, \mathbf{r}{\prime}) d\mathbf{r}{\prime} d\mathbf{r } \\ &\quad – \frac{j}{\omega \varepsilon_2} \int_S \int{V_2} (\nabla \cdot \mathbf{f}_m(\mathbf{r})) G_2(\mathbf{r}, \mathbf{r}{\prime}) (\nabla{\prime} \cdot \mathbf{f}_n(\mathbf{r}{\prime})) d\mathbf{r }{\prime} d\mathbf{r} \end{aligned}
    $$

    The same operation is used to obtain the remaining matrix elements. Here we directly copy the content of the additional materials of the paper:

    $$
    \begin{aligned} A_{\mathrm{EM}}^{mn}= & A_{\mathrm{HJ}}^{mn}=\int_{\mathbf{f}_m} \int_{\mathbf{f} _n} \mathbf{f}_m(\mathbf{r}) \cdot\left[\nabla G_1\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \times \mathbf{f}_n\left(\mathbf{r}^{\prime}\right)\right] d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{f}_m(\mathbf{r}) \cdot\left[\nabla G_2\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \times \mathbf{f} _n\left(\mathbf{r}^{\prime}\right)\right] d \mathbf{r}^{\prime} d \mathbf{r} \\ A_{\mathrm{HM}}^{mn} & =-j \omega \varepsilon_1 \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{ f}_m(\mathbf{r}) \cdot \mathbf{f}_n\left(\mathbf{r}^{\prime}\right) G_1\left(\mathbf{r}, \mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\frac{j}{ \omega \mu_1} \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \nabla \cdot \mathbf{f}_m(\mathbf{r}) G_1\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \nabla^{\prime} \cdot \mathbf{f} _n\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & -j \omega \varepsilon_2 \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{f}_m(\mathbf{r}) \cdot \mathbf{f}_n\left(\mathbf{r }^{\prime}\right) G_2\left(\mathbf{r}, \mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\frac{j}{\omega \mu_2} \int_{\mathbf {f}_m} \int_{\mathbf{f}_n} \nabla \cdot \mathbf{f}_m(\mathbf{r}) G_2\left(\mathbf{r}, \mathbf{r}^{\prime}\right) \nabla^{\prime} \cdot \mathbf{f}_n\left(\mathbf{r}^{\prime}\right) d \mathbf{r} ^{\prime} d \mathbf{r}\end{aligned}
    $$

    After getting each element of the matrix, we introduce shift-invariant functions to help us get the Green's function and its gradient in different coordinate systems.

    $$
    \begin{aligned}
    & g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right)=G_i\left(\mathbf{r}, \mathbf{r}^{\prime} \right)=\frac{e^{-j k_i r}}{4 \pi r} \\
    & g_{2, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right)=\hat{\mathbf{x}} \cdot \nabla G_i\left(\mathbf{ r}, \mathbf{r}^{\prime}\right)=-\left(xx^{\prime}\right)\left(\frac{1+j k_i r}{4 \pi r^3}\right) e^{-j k_i r} \\
    & g_{3, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right)=\hat{\mathbf{y}} \cdot \nabla G_i\left(\mathbf{ r}, \mathbf{r}^{\prime}\right)=-\left(yy^{\prime}\right)\left(\frac{1+j k_i r}{4 \pi r^3}\right) e^{-j k_i r} \\
    & g_{4, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right)=\hat{\mathbf{z}} \cdot \nabla G_i\left(\mathbf{ r}, \mathbf{r}^{\prime}\right)=-\left(zz^{\prime}\right)\left(\frac{1+j k_i r}{4 \pi r^3}\right) e^{-j k_i r} \quad \text { where } r=\left|\mathbf{r}-\mathbf{r}^{\prime}\ right|
    \end{aligned}
    $$

    Then the matrix elements are expanded into a combination of different components of the basis functions, which act on the translation invariant functions. All elements of the final matrix have the following form. This form can speed up the construction and solution of the boundary element matrix.

    $$
    \int_{\mathbf{f}m} \int{\mathbf{f}_n} \psi_m(\mathbf{r}) g\left(\mathbf{r}-\mathbf{r}^{\prime}\ right) \xi_n\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r}
    $$

    Finally, we get:


    Here, $x, y, z, x^{\prime}, y^{\prime}, z^{\prime}$ are the Cartesian components of $\mathbf{r}, \mathbf{r}^{\prime }$. Now we have for $i=1,2$:

    $$ \begin{aligned} A_{\mathrm{EJ}, i}^{mn} &= j \omega \mu_i \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{ f}_{mx}(\mathbf{r}) \, g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \, \mathbf{f}_{nx}\left(\mathbf{r}^{\prime}\right ) \, d \mathbf{r}^{\prime} \, d \mathbf{r} \\ & \quad + j \omega \mu_i \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{f}_{my}(\mathbf{r}) \, g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \, \mathbf {f}_{ny}\left(\mathbf{r}^{\prime}\right) \, d \mathbf{r}^{\prime} \, d \mathbf{r} \\ & \quad + j \omega \mu_i \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \mathbf{f}_{mz}(\mathbf{r }) \, g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \, \mathbf{f}_{nz}\left(\mathbf{r}^{\prime}\right) \, d \mathbf{r}^{\prime} \, d \mathbf{r} \\ & \ quad – \frac{j}{\omega \varepsilon_i} \int_{\mathbf{f}_m} \int_{\mathbf{f}_n} \nabla \cdot \mathbf{f}_m(\mathbf{r}) \, g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \, \ nabla^{\prime} \cdot \mathbf{f}_n\left(\mathbf{r}^{\prime}\right) \, d \mathbf{r}^{\prime} \, d \mathbf{r} \end{aligned} \tag{S.18} $$

    where $\mathbf{f}{mx}, \mathbf{f}{my}, \mathbf{f}_{mz}$ are the $x, y, z$ components of the vector basis function $\mathbf{f}_m$ . Similarly, we have:

    $$ \begin{aligned} & A_{\mathrm{EM}, i}^{mn}=A_{\mathrm{HJ}, i}^{mn}=\int_{\mathbf{f}m} \int{ \mathbf{f}n} \mathbf{f}{mz}(\mathbf{r}) g_{2, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{ny}\left(\mathbf{r}^{\prime}\right) d \ mathbf{r}^{\prime} d \mathbf{r} \\ & -\int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{my}(\mathbf{r}) g_{2, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{nz }\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{mx}(\mathbf{r}) g_{3, i}\left(\mathbf{r} -\mathbf{r}^{\prime}\right) \mathbf{f}{nz}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & -\int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{mz}(\ mathbf{r}) g_{3, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{nx}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\int{\mathbf {f}m} \int{\mathbf{f}n} \mathbf{f}{my}(\mathbf{r}) g_{4, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{nx}\left(\mathbf{r}^{\prime}\right) d \ mathbf{r}^{\prime} d \mathbf{r} \\ & -\int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{mx}(\mathbf{r}) g_{4, i}\left(\mathbf{r}-\mathbf{r}^{\prime} \right) \mathbf{f}_{ny}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \end{aligned} \tag{S.19} $$

    Lastly, we also have:

    $$ \begin{aligned} A_{\mathrm{HM}, i}^{mn} & =-j \omega \varepsilon_i \int_{\mathbf{f}m} \int{\mathbf{f}n} \mathbf {f}{mx}(\mathbf{r}) g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{nx}\left(\mathbf{r}^{\prime}\right) d \ mathbf{r}^{\prime} d \mathbf{r} \\ & -j \omega \varepsilon_i \int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{my}(\mathbf{r}) g_{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{ny }\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & -j \omega \varepsilon_i \int{\mathbf{f}m} \int{\mathbf{f}n} \mathbf{f}{mz}(\mathbf{r}) g_{1, i}\left( \mathbf{r}-\mathbf{r}^{\prime}\right) \mathbf{f}{nz}\left(\mathbf{r}^{\prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \\ & +\frac{j}{\omega \mu_i} \int{\mathbf{f}m} \int{\mathbf{f}n } \nabla \cdot \mathbf{f}_m(\mathbf{r}) g{1, i}\left(\mathbf{r}-\mathbf{r}^{\prime}\right) \nabla^{\prime} \cdot \mathbf{f}_n\left(\mathbf{r}^{\ prime}\right) d \mathbf{r}^{\prime} d \mathbf{r} \end{aligned} \tag{S.20} $$


    This part of the code is inMVProd ClassIt doesn’t matter if you don’t understand it, because I just copied and scribbled it randomly, which is beyond the scope of graphics research.

    In addition, the author also discussed the symmetry of EFIE and MFIE. It is this symmetry that makes the computational efficiency and space utilization lower. Note:


    $$
    A_{EJ} = A_{EJ}^T, \quad A_{HM} = A_{HM}^T \tag{19}
    $$

    $$
    A_{EM} = A_{EM}^T, \quad A_{HJ} = A_{HJ}^T, \quad A_{EM} = A_{HJ}\tag{20}
    $$

    Since the matrix is symmetric, we do not need to calculate all the matrix elements, nor do we need to store all the matrix elements.

    After solving for the surface current density, equation (6) can be used to calculate the scattered field propagating outward from the scattering surface.

    3.2 Rough Surface Scattering: The Specifics

    When simulating the interaction between electromagnetic waves and rough surfaces, the irregular geometric structure of the surface will have a significant impact on the scattering characteristics. The authors discretized the rough surface and divided the continuous surface into multiple units, each of which can be numerically calculated using the PMCHWT equation.

    3.2.1 Rough Surface Samples

    Represent the rough surface as a two-dimensionalHeight field, and then the height field is discretized and divided into multiple rectangular units.

    Only a surface sample of size $$L_x \times L_y$$ is considered for each simulation, and a step size $d$ is chosen to define the discretization grid.


    $$
    x_s = s \cdot d, \quad s = 0, 1, \ldots, N_x
    \
    y_t = t \cdot d, \quad t = 0, 1, \ldots, N_y
    \tag{21}
    $$


    Among them, $ N_x = L_x / d$ and $N_y = L_y / d$, respectively, represent the number of cells divided in the $x$ and $y$ directions. At each discrete point $(x_s, y_t)$ there is a height field $h(x_s, y_t)$ function.

    The authors concluded that the height variations of rough surfaces are very small, typically only a few micrometers, which is comparable to the wavelength of visible electromagnetic waves.

    3.2.2 Basis Elements and Functions

    Each primitive has four corners, each corner has a different height, and each corner contributes differently to the current and magnetic flux. Therefore, four basis functions are defined on each small square to approximate the situation on each small square.

    In most simulations, the step size $d$ is chosen to be around $\lambda / 16$ of the wavelength to ensure accuracy.

    Each primitive is parameterized by two parameters u and v, both in the range [-1, 1].

    The shape of the primitive is represented by a bilinear function $\mathbf{r}(u, v)$, where $(s, t)$ represents the index of the current primitive, and the primitive is determined by the coordinates of the four vertices:

    $$ \begin{aligned} \mathbf{r}(u, v) = &\frac{(1 – u)(1 – v)}{4} \mathbf{p}_{s-1, t-1} + \\ &\frac{(1 – u)(1 + v)}{4} \mathbf{p}_{s-1, t} + \\ &\frac{(1 + u)(1 – v)}{4} \mathbf{p}_{s, t-1} + \\ &\frac{(1 + u)(1 + v)}{4} \mathbf{p} _{s, t} \end{aligned} \tag{22} $$

    where $\mathbf{p}{s, t} = (x_s, y_t, z{s, t})$ are the coordinates of the four vertices of the primitive.

    Four basis functions $f_1(u, v), f_2(u, v), f_3(u, v), f_4(u, v)$ are defined on each rectangular primitive, and their forms are:


    $$
    \begin{aligned}
    & f_1(u, v) = \frac{(1 – u)}{J(u, v)} \frac{\partial \mathbf{r}(u, v)}{\partial u}, \quad f_2 (u, v) = \frac{(1 + u)}{J(u, v)} \frac{\partial \mathbf{r}(u, v)}{\partial u} \\
    & f_3(u, v) = \frac{(1 – v)}{J(u, v)} \frac{\partial \mathbf{r}(u, v)}{\partial v}, \quad f_4 (u, v) = \frac{(1 + v)}{J(u, v)} \frac{\partial \mathbf{r}(u, v)}{\partial v}
    \end{aligned}
    \tag{23}
    $$


    Here the Jacobian $J(u, v)$ is represented as follows:


    $$
    J(u, v) = \left| \frac{\partial \mathbf{r}(u, v)}{\partial u} \times \frac{\partial \mathbf{r}(u, v)}{ \partial v} \right|
    \tag{24}
    $$


    The introduction of Jacobian is used to transform the coordinate system and ensure that the basis functions have appropriate proportional relationships in different $u, v$ directions.

    3.2.3 Gaussian Beam Incidence

    Since I don't know much about optics, the following is my personal understanding. A Gaussian beam is a phenomenon in which the middle part of the traveling wave field appears to be concave inward when the laser is emitted. In other words, a Gaussian beam describes the energy distribution of light on the cross section. The focus of plane waves and spherical waves is to describe the direction of energy propagation. During the propagation process, the wavefront shape of a Gaussian beam is approximately a spherical wave.

    Gouy phase is a phase delay effect in Gaussian beam propagation. The phase of the beam will increase after passing through the focus. On the other hand, Gaussian beams satisfy a solution of Maxwell's equations under paraxial conditions and can be approximated as non-uniform spherical waves. I feel that there is no need to study it too deeply at present.

    Information about Gaussian beams:

    Back to the original paper, the advantage of Gaussian beam is that the size of the incident field can be controlled, and thus the surface induced current density of an area slightly larger than the irradiated area can be controlled to be a non-zero value.

    A Gaussian beam is an electromagnetic wave whose amplitude is distributed in a two-dimensional Gaussian pattern in a plane perpendicular to the propagation direction [Paschotta 2008]. Its energy is mainly concentrated near the center of the beam. Looking at the figure (a) above, a Gaussian beam can be described by the focal plane $P$ , the center point $o$ , and the beam waist $w$ . The field intensity decays with position as $e^{-r^2 / w^2}$ . When the distance from the center exceeds $2.5w$ , the field intensity decays to a very small value and can be considered to be almost zero.

    However, Gaussian beams also have a certain degree of divergence. The divergence angle $\theta$ is approximately proportional to the wavelength $\lambda$ and inversely proportional to the beam waist diameter $w$. Formula:


    $$
    \theta = \frac{\lambda}{\pi \eta w} \tag{25}
    $$


    When the beam is incident on the surface obliquely, the Gaussian beam has an elliptical cross section in the focal plane, as shown in the figure below. There are different beam waist sizes in two perpendicular directions: one parallel to the plane of the incident direction and the surface normal, and the other perpendicular to it.

    In order to ensure that the irradiation area of the Gaussian beam in different directions on the surface is the same, two beam waist widths in the lateral direction are introduced here:


    $$
    w_1 = w, \quad w_2 = w \cos \theta_i\tag{26}
    $$


    The illuminated area on the surface remains consistent even at different angles of incidence. This is very important for deriving the BRDF.

    4. IMPLEMENTATION AND ACCELERATION

    However, if we use the above method to calculate, there will be no result. Therefore, some acceleration methods are needed.

    4.1 The Adaptive Integral Method

    If we want to directly calculate the system of equations of formula (12) above, the amount of calculation is unacceptable.

    $$ \begin{bmatrix} A_{EJ} & A_{EM} \ A_{HJ} & A_{HM} \end{bmatrix} \begin{bmatrix} I_J \ I_M \end{bmatrix} = \begin{bmatrix} V_E \V_H \end{bmatrix} \tag{12} $$


    According to the original idea, $N$ basis functions are used to represent the current magnetic flux density, and the size of the matrix is $2N \times 2N$ . If the matrix is solved directly (LU decomposition, Cholesky decomposition, etc.), the total complexity may be $\mathcal{O}(N^3)$ . Even if the conjugate gradient method is used, the total complexity is still $\mathcal{O}(N^2)$ . In some small-scale simulations, the size of the basis function is about $960*960$ , and the storage requirement is about 29.4GB. Using the Adaptive Integral Method, AIM, the total storage requirement is about 76.8 MB based on 8 bytes. What is so magical about this method? Let me show you together!

    4.1.1 Approximating Matrix Elements

    AIM was first proposed by Bleszynski et al. [1996]. The core idea of AIM is to approximate the effect of each basis function as the effect of a set of point sources, avoiding the direct calculation of the exact interaction between each pair of basis functions, and at the same time spread the influence of each basis function through FFT to improve the calculation efficiency.

    The calculation method of matrix elements in AIM is to approximate the calculation by linear combination of some items of matrix elements. This is why the readers are asked to further deduce after the above formulas (13)-(16) to derive the final result so that it conforms to the form of AIM.

    $$
    \int_{f_m} \int_{f_n} \psi_m(r) g(r – r{\prime}) \xi_n(r{\prime}) \, dr{\prime} \, dr
    \tag{27}
    $$


    AIM first creates a global 3D Cartesian grid in the space containing the electromagnetic fields and field sources, as shown in Figure (6) below.

    In order to further simplify formula (27), the AIM algorithm approximates the original basis function as a point source on a set of grid points in this three-dimensional Cartesian coordinate system. In other words, it is a continuous-to-discrete transformation, which is convenient for subsequent FFT.

    $$ \psi_m(r) \approx \tilde{\psi}m(r) := \sum{p \in S_m} \Lambda_{mp} \delta^3(r – p) \\ \xi_n(r{\ prime}) \approx \tilde{\xi}n(r{\prime}) := \sum{q \in S_n} \Lambda{\prime}_{nq} \delta^3(r{\prime} – q) \tag{28} $$

    Substituting formula (27) into formula (28), in other words, the double integral form is converted into a double summation form.

    $$
    \sum{p \in S_m} \sum_{q \in S_n} \Lambda_{mp} g(p – q) \Lambda{\prime}_{nq}
    \tag{29}
    $$

     

    Method detailed reference:

    Kai Yang and Ali E Yilmaz. 2011. Comparison of precorrected FFT/adaptive integral method matching schemes. Microwave and Optical Technology Letters 53, 6 (2011), 1368–1372.

    4.1.2 Base and Correction Matrices

    Based on formula (29), a set of base approximation matrices $B_{EJ}, B_{EM}, B_{HJ}, B_{HM}$ are defined as approximations to specifically handle basis function pairs with a long distance. These matrices simplify the calculation by introducing $\Lambda$ matrices and convolution operations. At the same time, for basis function pairs with a short distance ($d_{near}$), correction matrices are introduced to reduce the error. $C_{EJ}, C_{EM}, C_{HJ}, C_{HM}$ is a sparse matrix defined as follows:

    $$ C_{\mathrm{X}}^{mn}=\left\{\begin{array}{ll} A_{\mathrm{X}}^{mn}-B_{\mathrm{X}}^{mn } & d_{mn} \leq d_{\text {near }} \\ 0 & \text { otherwise } \end{array} \quad \mathrm{X} \in\{\mathrm{EJ}, \mathrm{EM}, \mathrm{HJ}, \mathrm{HM}\}\right. \tag{30} $$

    $A_X^{mn}$ is the exact value of the original matrix, while $B_X^{mn}$ is the approximation of the fundamental matrix. By subtracting the approximation of the fundamental matrix, a more accurate correction term is obtained to compensate for the error of the close basis function pairs.

    In summary, the final approximate form of each matrix in the AIM method can be written as follows:

    $$ \begin{aligned} A_{\mathrm{EJ}} \approx B_{\mathrm{EJ}}+C_{\mathrm{EJ}} ; & A_{\mathrm{EM}} \approx B_{\mathrm{ EM}}+C_{\mathrm{EM}} ; \\ A_{\mathrm{HJ}} \approx B_{\mathrm{HJ}}+C_{\mathrm{HJ}} ; & A_{\mathrm{HM}} \approx B_{\mathrm{HM}}+C_{\mathrm{HM}} \end{aligned } $$


    In other words, the original matrix can be approximated by a combination of the fundamental matrix and the correction matrix.

    4.1.3 Fast Matrix-Vector Multiplication

    Fast Matrix-Vector Multiplication is the core of AIM.

    Since the modified matrix $C$ obtained above is a sparse matrix, $C$ has non-zero values only on close-range basis functions, so the multiplication operation of the matrix $C$ is very fast.

    Using the convolution property of the basic approximation matrix $B$, the product of the matrix $B$ and the vector is calculated. The calculation process is divided into three steps:


    $$
    y_1 = \Lambda_2^T x, \quad y_2 = G y_1, \quad y_3 = \Lambda_1 y_2
    \tag{32}
    $$


    The first step is to project the vector onto a sparse matrix grid.

    The second step is also the core step, which is to propagate the data of the grid points to the entire network, that is, to calculate the influence of each point on other points. The closer the two points are, the larger the propagation function in the matrix $G$ will be. FFT is used to speed up the process.


    $$
    y_2 = \mathcal{F}^{-1} { \mathcal{F}(g) \mathcal{F}(y_1) }
    \tag{33}
    $$


    The third step maps the result back to the original basis function space.

    4.2 GPU-Accelerated Iterative Solving

    The computational focus of the AIM method is shifted to fast Fourier transform (FFT) and sparse matrix operations on the GPU. In summary, the large matrix is currently divided into the basic matrix $B$ and the correction matrix $C$ to handle the long-distance and short-distance basis function pairs respectively.

    • cuFFT: Converts the underlying matrix multiplication operation to a convolution calculation in the frequency domain
    • cuSPARSE: Accelerate the correction matrix C for sparse matrix calculations

    And optimize the calculation strategy:

    • For small-scale simulation tasks, only one GPU is needed.
    • For large-scale simulation tasks, distribute the tasks to 4 GPUs
    4.2.1 Small-Scale Simulations

    For small-scale simulation tasks (e.g. $12 \mu m \times 12 \mu m$), the Fourier transform of the propagation function (i.e. the Fourier transform of the matrix $G$) must be calculated and stored in advance. In small-scale tasks, the sparse correction matrix C occupies less than 5GB of video memory and can be handled by a single GPU.

    4.2.2 Large-Scale Simulations

    For large-scale simulation tasks (e.g. $24 \mu m \times 24 \mu m$), since the video memory of a single GPU is not enough to store all the data, the author distributes the computational tasks to 4 GPUs. In this scale simulation, the number of basis functions will reach 960 × 960, and storing all non-zero elements of the correction matrix (including row and column indices and complex floating-point values) requires about 20GB of video memory. The strategy is still the same as the small scale, and each GPU is allocated about 5GB of memory to store the correction matrix $C$.

    The MINRES solver is executed on the host CPU, while the matrix-vector product $y = Ax$ is computed on the GPU. But don’t worry about the transfer time, the vector is only about 30MB.

    4.3 FFT-Accelerated Scattered Field Evaluation

    The FFT is used to accelerate the calculation of the scattered field. The field scattered from the surface is evaluated in the far field region to finally find the surface BRDF.

    After solving the BEM, the current density $\mathbf{J}$ and the magnetic flux density $\mathbf{M}$ on the surface are obtained. These density distributions define the electromagnetic sources on the surface and can be used to calculate the scattered field in the far field region. The formula is very simple, and it will weaken with distance and have a certain phase change:


    $$
    \mathbf{E_s}(r) \approx \mathbf{E}(\hat{r}) \frac{e^{-jkr}}{r}; \quad \mathbf{H_s}(r) \approx \mathbf {H}(\hat{r}) \frac{e^{-jkr}}{r}
    \tag{36}
    $$


    The right side of the formula $\mathbf{E}(\hat{r})$ and $\mathbf{H}(\hat{r})$ are the amplitudes of the specific directions $\hat{r}$ in the far field. In different directions, the intensity of the scattered field may be different.


    $$
    \begin{aligned}
    F_1(\hat{\mathbf{r}})=\int_V J_x\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime} ; & F_2(\hat{\mathbf{r}})=\int_V J_y\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime} \\
    F_3(\hat{\mathbf{r}})=\int_V J_z\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime} ; & F_4(\hat{\mathbf{r}})=\int_V M_x\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime} \\
    F_5(\hat{\mathbf{r}})=\int_V M_y\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime} ; & F_6(\hat{\mathbf{r}})=\int_V M_z\left(\mathbf{r}^{\prime}\right) e^{jk \mathbf{r}^{\prime} \cdot \hat {\mathbf{r}}} d \mathbf{r}^{\prime}
    \end{aligned}
    \tag{37}
    $$


    To avoid solving these integrals directly, the author uses the point source approximation ($\Lambda$ matrix) of $\mathbf{J}$ and $\mathbf{M}$ previously (4.1.2) to discretize each integral term $F_i(\hat{r})$ and rewrite it in Fourier transform form, as shown in formula (38):


    $$
    F_i(\hat{r}) = \sum_{p \in S} h_i(p) e^{jp \cdot k \hat{r}}
    \tag{38}
    $$


    Convert the continuous field strength calculation into a discrete sum so that it can be quickly calculated using FFT. And from the above formula, it can be observed that $F_i(\hat{r})$ is actually the Fourier component of $h_i(p)$ at the spatial frequency $-k\hat{r}$.

    The required spatial frequencies are not on the FFT grid but can be interpolated; we add zero padding prior to the FFT step, to ensure enough resolution in the frequency domain for the trilinear interpolation to be sufficiently accurate.

    5 HIGH RESOLUTION BRDF GENERATION

    After a lot of work, I finally got back to the familiar BRDF calculation. The key here is to use the linear superposition of small-scale simulations to reconstruct the far-field scattering of the large-scale incident field, rather than trying to do it all at once. A grid of $N^2$ small-scale Gaussian beams is linearly combined to approximate the large-scale incident field.

    This is done using a technique called beam steering, which significantly reduces computational costs by not having to simulate each direction.

    5.1 Basic and Derived Incident Directions

    First, $N^2$ Gaussian beams propagating in a certain direction $\mathbf{u}$ form a grid of $N \times N$ points in the receiving plane. These beams are combined to generate a large total field.

    Then, a complex scaling factor is introduced into each Gaussian beam to adjust the phase of each beam and thus adjust the propagation direction of the combined field. These directions are called desired directions.


    $$
    a_{st} = e^{jk \mathbf{p}{st} \cdot \omega_i} \tag{39}
    $$

    When the angle between the target incident direction $-\omega_i$ and the basic direction $\mathbf{u}$ is close to the divergence of the small beams, aliasing artifacts begin to appear. An example is shown in Fig. 8 (d).

    In our framework, we decide on a primary waist w and choose a collection of basic incident directions. In general, a smaller waist width means a larger divergence angle, so that more incident directions can be derived from each basic direction, thereby reducing the number of basic directions required. A larger waist width will reduce the divergence angle of each Gaussian beam, making the total field divergence after combination smaller, thus producing a more accurate incident direction.

    The center of each hexagon corresponds to a basic direction of incidence. All the directions of incidence in the entire hemisphere are divided into several territories, and each territory belongs to a basic direction of incidence. In the hemispherical projection, this proportional relationship and the cosine factor cancel each other out, so you can useHexagons of equal sizeTo express.

    For each cardinal direction, several derived directions that deviate from the cardinal direction can be generated through "beam steering".

    When the incident angle of light is very small (for example, close to the surface normal direction), the range of derived directions near the basic incident direction is very small, because the light at a small angle is more concentrated and will not diffuse greatly. Conversely, the range of derived directions will be larger. In summary, the larger the incident angle (the closer the angle is to the horizontal), the larger the coverage of the derived directions. The author of the formula also mentioned: $1/\cos \theta_i$.

    5.2 Individual Simulations and Synthesized Results

    In order to calculate the BRDF, we need to know the scattering of this large area of incident light. However, directly simulating such a large area of light will require a high computational cost. Therefore, we:

    The large-scale incident field is simulated using a superposition of small-scale simulations.

    Think of it as using many small flashlights (Gaussian beams) to cover an area, rather than one giant searchlight.

    First, determine the size of the flashlights (the size of the beam, i.e. the waist width $w$) and evenly distribute them on the surface. Written as a formula, this arrangement of flashlights is the grid point ${x_s}, {y_t}$, representing the center position of each Gaussian beam. The grid spacing is generally the same as the waist width to ensure uniform coverage of the light and keep the divergence angle low.

    Let each Gaussian beam produce the same electromagnetic field in its central region, just repeating the effect at different locations.

    Next, to get the total scattered field of a large beam, we needPhase FactorTo perform “adjustment” and “superposition”, please refer to formula (39) for details.

    Finally, Combining with Eq. 39, the scattered fields in the far field region corresponding to the pair of directions $(w_i,w_o)$ are given by:

    $$ \begin{aligned} \mathbf{E}\left(\omega_i, \omega_o\right) & =\sum_{s=1}^n \sum_{t=1}^ne^{jk \mathbf{p} {st} \cdot\left(\omega_i+\omega_o\right)} \mathbf{E}{st}\left(\omega_o\right) \\ \mathbf{H}\left(\omega_i, \omega_o\right) & =\sum_{s=1}^n \sum_{t= 1}^ne^{jk \mathbf{p}{st} \cdot\left(\omega_i+\omega_o\right)} \mathbf{H}{st}\left(\omega_o\right) \end{aligned} \tag{41} $$

    where $\mathbf{E}, \mathbf{H}$ refer to the far field quantities only associated with directions (without the $e^{-jkr} / r$ term).

    Finally, we can compute the surface BRDF value as

    $$
    f_r\left(\omega_i, \omega_o\right)=\frac{\frac{1}{2}\left|\mathbf{E}\left(\omega_i, \omega_o\right) \times \mathbf{H} \left(\omega_i, \omega_o\right)^*\right|}{\Phi_i \cos \theta_r}
    \tag{42}
    $$

    where the incident power $\Phi_i$ is computed by integrating the incident irradiance over the surface:

    $$
    \Phi_i=\frac{1}{2} \int_S\left|\left[\mathbf{E}_i\left(\mathbf{r}^{\prime}\right) \times \mathbf{H}_i\ left(\mathbf{r}^{\prime}\right)^*\right] \cdot \mathbf{n}\right| d \mathbf{r}^{\prime}
    \tag{43}
    $$

    where $\mathbf{n}$ is the surface normal at the macro scale ( $+\mathbf{z}$ ). Note that Eq. 42 and Eq. 43 can also be applied in single simulations, where $\Phi_i$ is computed from a single Gaussian beam.

  • 波动光学毛发渲染:相关论文汇总整理(一)-学习笔记-3

    Wave Optics Hair Rendering: A Summary of Related Papers (I) - Study Notes-3

    Disclaimer: This is a pure garbage article, which is a supplement to the previous two articles. I just sorted it out for my own review. All titles can be redirected to the paper homepage. Special terms are given in Chinese and English as much as possible. If there are any mistakes, please point them out. Thank you very much.

    original:https://zhuanlan.zhihu.com/p/830617613

    Table of contents

    1. [Xia 2023] A Practical Wave Optics Reflection Model for Hair and Fur
    2. [Xia 2023] Iridescent Water Droplets Beyond Mie Scattering
    3. [Aakash 2023] Accelerating Hair Rendering by Learning High-Order Scattered Radiance
    4. [Kneiphof and Klein 2024] Real-Time Rendering of Glints in the Presence of Area Lights
    5. [Huang 2024] Real-time Level-of-detail Strand-based Hair Rendering
    6. [Xing 2024] A Tiny Example-Based Procedural Model for Real-Time Glinty Appearance Rendering
    7. [Zhu 2022] Practical Level-of-Detail Aggregation of Fur Appearance
    8. [Clausen 2024] Importance of multi-modal data for predictive rendering
    9. [Shlomi 2024] A Free-Space Diffraction BSDF
    10. [Kaminaka 2024] Efficient and Accurate Physically Based Rendering of Periodic Multilayer Structures with Iridescence
    11. [Yu 2023] A Full-Wave Reference Simulator for Computing Surface Reflectance
    12. [Shlomi 2022] Towards Practical Physical-Optics Rendering
    13. [Huang 2022] A Microfacet-based Hair Scattering Model
    14. [Shlomi 2021] A Generic Framework for Physical Light Transport
    15. [Shlomi 2024] A Generalized Ray Formulation For Wave-Optics Rendering
    16. [Shlomi 2021] Physical Light-Matter Interaction in Hermite-Gauss Space
    17. [GUILLÉN 2020] A general framework for pearlescent materials
    18. [Werner 2017] Scratch iridescence: Wave-optical rendering of diffractive surface structure
    19. [Fourneau 2024] Interactive Exploration of Vivid Material Iridescence using Bragg Mirrors
    20. [Chen 2020] Rendering Near-Field Speckle Statistics in Scattering Media
    21. [Kajiya and Kay 1989] Kajiya-Kay Model
    22. [Marschner 2003] Light Scattering from Human Hair Fibers
    23. [Benamira 2021] A Combined Scattering and Diffraction Model for Elliptical Hair Rendering
    24. [Zinke 2008] Dual Scattering Approximation for Fast Multiple Scattering in Hair

    [Xia 2023] A Practical Wave Optics Reflection Model for Hair and Fur

    Wave optics, hair rendering, surface electromagnetics, far-field scattering

    Wave optics is used to render hair. The surface electromagnetic field is calculated to obtain the scattered field, and then noise is added to simulate the Glints effect.

    I found that the authors of this series are all very good-looking. (crossed out)

    img

    1. Background

    Hair rendering has been mainly based on ray tracing technology, which cannot handle wave optics effects, such as strong forward scattering and subtle color changes on the hair surface. Previous research [Xia et al. 2020] demonstrated that diffraction effects play a key role in the color and scattering direction of fibers. However, this study did not consider surface roughness and the microstructure of the fiber epidermis (such as tilted keratin scales).

    2. Motivation

    In order to make up for the lack of treatment of diffraction and forward scattering (such as Glints phenomenon) in the existing light optics model.

    Although full-wave simulations can produce very detailed scattering data, the computational effort is still too high and must be accelerated or simplified in some way to achieve hair or fur rendering in large-scale scenes.

    We wanted to develop a model that could efficiently handle various fiber geometry variations.

    3. Methods

    Hair modeling is based on scanning electron microscope (SEM) images of hair.

    Use "WAVE SIMULATION WITH 3D FIBER MICROGEOMETRY" to calculate the reflection and diffraction of rough fiber surfaces. That is, PO.

    Speckle theory is introduced to analyze the statistical characteristics of the scattering pattern, and noise is used to accelerate it.

    [Xia 2023] Iridescent Water Droplets Beyond Mie Scattering

    Wave optics, iridescence effect, Quetelet scattering model of water droplets on water surface

    Combining Mie scattering, Quetelet scattering (light interference) and dynamic changes of water droplets, the rainbow-like color effect of water droplets on the water surface and in the steam is realistically rendered, surpassing the traditional single Mie scattering model.

    img

    1. Background

    Iridescence is common in nature, especially in water droplets, fog and steam. It can generally be explained by Mie scattering. Mie scattering describes the scattering effect that occurs when light encounters spherical particles of the same wavelength. It is one of the important theories currently used to simulate natural phenomena such as water droplets, clouds, rain and fog.

    However, while Mie scattering can explain the optical properties of isolated water droplets, it cannot fully explain phenomena such as the iridescence of water droplets on the surface of water and the complex rainbow patterns in vapor. Phenomena depend not only on how individual particles scatter light, but also on surface reflections, interference effects, and dynamic changes in particle size.

    2. Motivation

    Mie scattering can only deal with isolated light scattering phenomena and cannot explain more complex optical interference effects.

    Accurately simulating these natural phenomena can greatly improve the realism and look and feel of image rendering.

    Existing computer optical models and rendering methods are mostly limited to Mie scattering and cannot explain the interaction of light in a multi-particle environment, such as light interference and reflection between water droplets or between water droplets and surfaces.

    3. Methods

    The "Quetelet scattering model on water" is used to explain the rainbow effect produced by water droplets floating on the water surface. By building an empirical model, thermal imaging technology is used to relate temperature to the size and height of water droplets. Quetelet scattering phase function and BRDF (bidirectional reflectance distribution function) are used to render particle groups and water surfaces.

    A water droplet growth and evaporation model was developed to simulate the dynamic changes of water droplets in steam. Combined with Mie scattering, water droplets of non-uniform size were used to simulate the rainbow color changes in steam. In order to improve rendering efficiency, an acceleration algorithm based on motion blur was used, which increased the calculation speed by 10 times compared with traditional methods.

    [Aakash 2023] Accelerating Hair Rendering by Learning High-Order Scattered Radiance

    Hair rendering, MLP, accelerated hair scattering

    The method of learning hair higher-order scattered radiance online combined with a small multilayer perceptron (MLP) significantly accelerates hair rendering in a path tracing framework, reducing computation time and introducing only a small amount of bias.

    img

    1. Background

    The multiple scattering of hair is very complex, especially in the path tracing process, because it is necessary to simulate the multiple scattering of light between hairs, which makes it difficult to converge.

    2. Motivation

    Develop a method to improve computational efficiency while maintaining high-quality simulation of multiple scattering effects.

    In the existing technology, some methods make simplifying assumptions about the scene or lighting. This paper hopes to propose a general method that does not make any assumptions about the scene.

    3. Methods

    A small multilayer perceptron (MLP) is used to learn higher-order scattered radiance online. This MLP network learns the scattering properties of hair in real time during the rendering process, without relying on pre-computed tables or simulations.

    The MLP is integrated into the path tracing framework to infer and compute higher-order diffuse radiation contributions.

    The renderer's bias and speedup can be adjusted in real time to find the optimal balance between computational efficiency and rendering quality.

    [Kneiphof and Klein 2024] Real-Time Rendering of Glints in the Presence of Area Lights

    Accelerated area light source Glints, microsurface models, real-time rendering

    Rendering glints under area lights is done in real time by combining Linearly Transformed Cosines (LTC) with a microsurface count model based on the binomial distribution.

    img

    1. Background

    Many real-world materials (such as metals, gemstones, etc.) have a glittering appearance, which is caused by the reflection of micro-surfaces. However, glitter is a discrete phenomenon, and the computational complexity of wave optics simulation is too large.

    Previous studies have mostly focused on using infinitesimal point light sources to render flash effects, which is a reasonable simplification for distant light sources like the sun, but in reality most light sources are essentially area light sources. Existing technologies have not been able to effectively handle flash rendering under area light sources.

    2. Motivation

    Glint rendering under area lights. Area lights (such as the light shining into a room through a window) are a common type of light, and how to efficiently render glint effects under such lights is an unsolved problem. We hope to develop a method that can accurately render glint effects under area lights while meeting the needs of real-time rendering.

    It is hoped that it can be easily integrated into existing real-time rendering frameworks without introducing significant additional overhead to existing area light shading methods.

    3. Methods

    Glint reflection probability estimation computes the probability that a microfacet is correctly oriented to reflect light from a light source to an observer, using Linearly Transformed Cosines (LTC) for large sources and a locally constant approximation for small sources.

    The number of reflective microsurfaces is counted using a binomial distribution-based counting model.

    Integration with existing frameworks.

    [Huang 2024] Real-time Level-of-detail Strand-based Hair Rendering

    Hair rendering, LoD, based on hair strands, BCSDF

    An innovative real-time strand-based hair rendering framework is proposed, which ensures the consistent appearance of hair at different view distances and achieves significant rendering acceleration through seamless level-of-detail (LoD) transition.

    img

    1. Background

    Strand-based hair rendering is becoming increasingly popular in film, television and game production for its realistic appearance, but it is computationally very expensive, especially at long viewing distances.

    The current LoD method is prone to noticeable discontinuities in the transition from hair strands to cards, resulting in inconsistent appearance.

    2. Motivation

    Solve discontinuity in dynamics and appearance. Existing solutions for converting hair strands to hair cards have significant differences in appearance and animation performance. The goal of this paper is to achieve seamless LoD transition from far to near, eliminating appearance changes during transition while maintaining computational efficiency.

    3. Methods

    Encapsulates multiple hair strands within an elliptical volume using an elliptical thick hair model. The shape and overall structure of the hair cluster is maintained at different LoDs, providing a consistent look as the view distance changes.

    The elliptical bidirectional curve scattering distribution function (BCSDF) simulates single and multiple scattering phenomena within hair clusters and is suitable for hair distribution scenarios ranging from sparse to dense and from static to dynamic.

    Dynamic LoD adjustment and hair width calculation.

    [Xing 2024] A Tiny Example-Based Procedural Model for Real-Time Glinty Appearance Rendering

    Glints, material self-similarity

    A model based on tiny example microstructures that renders glinty effects in real time, significantly reducing memory usage and computational overhead while maintaining the realism of high-frequency reflection details.

    img

    1. Background

    The shimmering details produced by complex microstructures can significantly improve the realism of renderings, especially on materials such as metals and gemstones. These details usually require high-resolution normal maps to define each micro-geometry, but such methods have high memory requirements and are not suitable for real-time rendering applications.

    2. Motivation

    Reduce memory and computational overhead.

    Leveraging material self-similarity: Many materials have independent structural features and self-similarity, and small samples are used to implicitly represent complex microstructures, thereby reducing memory requirements.

    3. Methods

    A tiny example-based procedural model based on the microstructure of a small sample can generate complex sparkle details by reusing a small number of samples based on the self-similarity of the material.

    Precomputed Normal Distribution Functions (NDFs) Precompute and store small samples of normal distribution functions (NDFs) using 4D Gaussians. Stored in multi-scale NDF maps and called by simple texture sampling at rendering time.

    A tiny example-based NDF evaluation method combines texture sampling with a small example NDF evaluation method to quickly generate the shiny appearance of complex surfaces.

    [Zhu 2022] Practical Level-of-Detail Aggregation of Fur Appearance

    Hair rendering, simplified hair count, neural networks

    A practical hair appearance aggregation model that significantly accelerates hair rendering while maintaining realistic visual effects by reducing the number of geometric hairs and combining multiple scattering of light, using neural networks to achieve real-time dynamic simplification.

    img

    1. Background

    If there are too many hairs, the light scattering and reflection of each hair will greatly increase the calculation amount, especially when simulating multiple light scattering.

    Most existing simplification methods improve rendering efficiency by reducing the number of hairs, but this method has great limitations. This method can cause the hair to look too rough or dry, and the reflection and scattering effects of light are not realistic.

    2. Motivation

    Reducing geometric complexity.

    Improving rendering efficiency.

    3. Methods

    An aggregated fur appearance model is proposed, which uses a thick cylinder to represent the optical behavior of a group of hair clusters. By analyzing the optical properties of individual hairs (such as the incident angle of light), the model can accurately reflect the aggregated appearance of hair clusters.

    A lightweight neural network is used to map the optical properties of individual hairs to parameters in the aggregate model.

    A dynamic level-of-detail scheme based on view distance and number of light bounces is proposed to dynamically simplify the geometric structure of hair.

    [Clausen 2024] Importance of multi-modal data for predictive rendering

    Predictive rendering, spectral rendering, microsurface geometry

    Multi-modal data is important for predictive rendering, especially in accurately modeling material reflection behavior. By combining spectral, spatial information and micro-geometric details, the realism and computational efficiency of reflection models can be improved.

    img

    1. Background

    The need for predictive rendering aims to accurately simulate the appearance of materials.

    Most current databases on material reflection behavior are limited to a single dimension, usually covering only the spectral domain or the spatial domain, and lack descriptions of microgeometry details.

    2. Motivation

    In order to address data limitations, multimodal data can not only better simulate the reflection of materials under different lighting conditions, but also reveal the influence of the microscopic geometry of the material surface on light scattering.

    Multimodal reflectance data can help develop more realistic and efficient reflectance models.

    3. Methods

    Building a multi-modal reflection database, including spectral data, spatial distribution data and microgeometry details of the material.

    Simulating microgeometry of the microgeometry of a material surface.

    Integrating spectral and spatial domains.

    [Shlomi 2024] A Free-Space Diffraction BSDF

    Wave optics, electromagnetic computing, free space diffraction, importance sampling, PT integration,

    A bidirectional scattering distribution function (BSDF) based on free-space diffraction can efficiently simulate the diffraction phenomenon of light around the edges of objects in complex scenes through ray tracing without the need for geometric preprocessing, and is particularly suitable for path tracing technology.

    img

    1. Background

    Free-space diffraction is an optical phenomenon in which light is diffracted when it encounters an edge or corner of an object, bending some of its energy into the shadowed area. This phenomenon is important for modeling the propagation of light waves, especially at long wavelengths, such as radar, WiFi, and cellular signals.

    The limitations of traditional methods such as the Geometric Theory of Diffraction (GTD) and the Unified Theory of Diffraction (UTD) are the extremely high computational complexity caused by the need to deal with light rays that interfere with each other, especially in complex geometric scenes. Existing methods rely on scene simplification and specific geometric structures and cannot effectively handle complex three-dimensional scenes.

    2. Motivation

    Addressing diffraction rendering in complex scenes. Existing diffraction simulation methods are difficult to scale and make compatible with path tracing techniques.

    Existing diffraction methods often rely on complex nonlinear interference calculations, while path tracing uses linear rendering equations. This paper hopes to design a free-space diffraction BSDF that works efficiently within the path tracing framework without requiring major modifications to the path tracer.

    3. Methods

    The Fraunhofer diffraction edge model is based on Fraunhofer diffraction. Near the intersection of light and geometric objects, the relevant edges are identified and the diffraction effects are calculated. When the light hits the object, the BSDF of free space diffraction is constructed through geometric analysis to quantify how the light propagates around the geometric object and how much energy is diffracted.

    The importance sampling strategy evaluates the geometric edges around the points where the ray interacts with the object and samples and traces the diffracted rays.

    Seamless integration in path tracing

    [Kaminaka 2024] Efficient and Accurate Physically Based Rendering of Periodic Multilayer Structures with Iridescence

    Multi-layer oil film rendering, iridescence effect, wave optics

    A multi-layer interference rendering method. It can express the iridescence effect of periodic multi-layer structures. By introducing the Huxley method from biology, it can achieve efficient calculation independent of the number of layers.

    img

    1. Background

    Thin-film interference is an optical phenomenon caused by the wave properties of light waves, which produces iridescence when the viewing angle or illumination angle changes. It usually appears in single-layer or multi-layer structures in nature, such as butterfly wings, beetle shells and dielectric mirrors.

    The limitations of existing methods such as recursive calculation method and transfer matrix method (TMM) are that the computational complexity increases significantly with the number of layers. Simplified methods ignore multiple reflections in thin films.

    2. Motivation

    Improving efficiency for multilayer structures.

    Applied to physical rendering of complex materials.

    3. Methods

    A multilayer interference model based on Huxley's approach is proposed. It can efficiently calculate the reflection and transmission coefficients in periodic multilayer structures and supports multiple materials and absorption effects.

    Based on BRDF implementation. Implemented as a BRDF (Bidirectional Reflectance Distribution Function), it can be integrated into traditional rendering systems such as PBRT-v3.

    [Yu 2023] A Full-Wave Reference Simulator for Computing Surface Reflectance

    Wave optics, full-wave simulation

    Full-wave simulator based on the boundary element method (BEM) that can calculate light scattering on rough surfaces with high accuracy. It is used to evaluate and improve approximate reflection models in computer graphics, especially when multiple scattering, interference and diffraction effects are significant.

    img

    1. Background

    Surface reflection models are usually based on geometric optics, which assumes that light propagates in the form of rays. For scenes where surface features are comparable to the wavelength of light, geometric optics models cannot accurately capture wave effects such as diffraction and interference.

    Based on wave optics approximations, such as Beckmann-Kirchhoff theory and Harvey-Shack model, they still produce errors under multiple scattering and complex geometric structures.

    2. Motivation

    Since existing reflection models have different accuracy in different situations, there is a lack of reliable benchmarks to verify their accuracy. The goal of this paper is to develop a simulator based on full-wave theory to minimize approximations and achieve high-precision surface reflection calculations through numerical discretization, thereby providing a reference tool that can be used to evaluate the accuracy of various reflection models.

    Addressing multiple scattering and wave effects.

    3. Methods

    Boundary Element Method (BEM), accelerated by Adaptive Integral Method (AIM).

    The simulator's full-wave simulation completely solves Maxwell's equations and can accurately simulate wave phenomena such as light propagation, interference, and scattering.

    And it can efficiently calculate BRDF (efficient BRDF computation).

    [Shlomi 2022] Towards Practical Physical-Optics Rendering

    Wave optics, PLT

    We propose an efficient Physical Light Transport (PLT) framework that exploits the principles of partially coherent light and wave optics to achieve accurate rendering of interference, diffraction, and polarization effects in complex scenes through an improved rendering algorithm, bringing its performance close to that of classic “physically based” rendering methods.

    img

    1. Background

    Most existing rendering methods ignore the wave characteristics of light, especially in complex scenes, which makes it impossible to render physical phenomena such as interference and diffraction of light, which are particularly important on certain materials (such as iridescent coatings, optical discs, etc.). To solve this problem, a rendering framework based on Maxwell's electromagnetic theory is proposed.

    Although PLT provides a theoretical full-wave model that can simulate the coherence, interference and diffraction of light, existing methods are very computationally difficult.

    2. Motivation

    Simplifying the physical light transport model.

    Introducing new coherence-aware materials and developing material models that can perceive light coherence will improve the usability of PLT in practical scenarios.

    3. Methods

    Restricting the coherence shape of light, through thermodynamic derivation, proves that this approximation is reasonable under most natural light sources.

    An extended Stokes-Mueller calculus is used to combine the radiation, polarization and coherence properties of light as new rendering primitives. The generalized Stokes parameters can fully quantify all properties of light and accurately simulate complex optical phenomena caused by these properties, such as interference and diffraction.

    Wave BSDF and importance sampling.

    New coherence-aware material models take full advantage of the coherence properties of light to expand the scope of application of PLT.

    [Huang 2022] A Microfacet-based Hair Scattering Model

    Hair rendering, scattering lobes, BCSDF

    The first hair scattering model based on microsurface theory is proposed to accurately describe the scattering behavior of hair, including non-separable scattering lobe structure, elliptical cross section, efficient importance sampling and forward scattering spot (glint-like) effect.

    img

    1. Background

    Complexity of hair rendering. Most existing hair scattering models simplify the mathematical calculations through separable scattering lobes, which are fast but not ground truth.

    Most hair scattering models are based on geometric simplification, treating hair as smooth cylinders, which leads to deviations in scattering behavior.

    2. Motivation

    Introducing a physically-plausible microfacet model more accurately describes the scattering behavior of hair: the surface microscopic roughness, the tilted scale structure, and the non-separable scattering lobe shapes.

    Improving sampling efficiency and physical accuracy.

    3. Methods

    The hair modeling is combined with microfacet theory, and GGX or Beckmann normal distribution is applied to describe the microscopic roughness of the surface. And it is non-separable lobes.

    The bidirectional curve scattering distribution function (BCSDF) describes the complex interaction of light on the hair surface.

    Support for elliptical cross-sections and efficient sampling. Support for elliptical cross-sections for hair.

    [Shlomi 2021] A Generic Framework for Physical Light Transport

    Wave optics, PLT

    The first global light transport framework based on Maxwell's electromagnetic theory that can handle partially coherent light is proposed, which accurately simulates the interference and diffraction effects of light and extends the traditional radiometric-based light transport theory to the field of wave optics.

    img

    1. Background

    Existing light transport models are usually based on geometric optics and radiometry, which ignore the wave characteristics of light and cannot simulate phenomena such as interference and diffraction. They cannot accurately reproduce wave optical effects such as rainbow effect, grating, thin film interference, light polarization, etc., which is the limitation of classical radiometric light transport models.

    Current models can only handle local treatment of wave effects, but cannot account for the transmission and coherence of light in global scenes.

    2. Motivation

    Achieving global wave-optics consistency in light transport, that is, combining Maxwell's electromagnetic theory.

    Combining wave optics with classical geometric optics (integrating wave optics with classical geometric optics) can deal with the wave effect of light and be consistent with classical geometric optics in the short wavelength limit.

    3. Methods

    Modeling partially-coherent light is divided into two parts: two-point coherence description and light source model. Different from traditional radiance, this paper introduces a "cross-spectral density function" based on the partial coherence of light, which can capture the interference characteristics of light. The physical model of natural light sources is based on the principle of spontaneous radiation in quantum mechanics.

    Generalizing the light transport equation. The spectral-density transport equation is used to calculate the interference and diffraction effects of light during propagation. This paper also proves that the framework can be simplified to classical geometric optics in the short wavelength limit, so it can be seamlessly integrated with existing light transport methods.

    Diffraction and propagation model.

    [Shlomi 2024] A Generalized Ray Formulation For Wave-Optics Rendering

    Wave optics, wave sampling theory, bidirectional path tracing

    A generalized ray formal model is proposed for wave optics rendering. By solving the sampling problem, weak locality, linearity and completeness are simultaneously established in wave optics. Bidirectional wave optics path tracing and efficient rendering are achieved in complex scenes.

    img

    1. Background

    The classical model of light transport is based on ray optics, which assumes that light propagates as a point query in a linear manner. However, ray optics cannot capture the wave nature of light and ignores interference and diffraction phenomena, such as the iridescence effect, thin film interference, and diffraction of long-wave radiation.

    Although wave optics can accurately describe the interference and diffraction effects of light, traditional sampling and path tracing techniques are difficult to apply due to its nonlinear behavior.

    2. Motivation

    Solving the sampling problem in wave optics. In order to apply wave optics in bidirectional optical transmission, it is necessary to solve the sampling problem under weak locality.

    Develop a novel formalism of wave optics that enables efficient applications in inverse path tracing and bidirectional light transport while maintaining linearity and completeness.

    Improving wave-optics rendering efficiency, making the convergence speed of wave-optics rendering close to that of classical ray optics rendering systems.

    3. Methods

    Introduction of the generalized ray. Perform weak local linear queries. Generalized rays are no longer limited to point queries at a single location, but occupy a small spatial region. They can capture the interference and diffraction effects of light.

    Weak locality and linearization. In wave optics, perfect locality and linearization cannot be achieved simultaneously. Therefore, perfect locality is abandoned. Weak locality is adopted to ensure that generalized rays can be linearly superposed.

    Backward wave-optical light transport model.

    Application in bidirectional path tracing.

    [Shlomi 2021] Physical Light-Matter Interaction in Hermite-Gauss Space

    Wave optics, PLT

    A new framework for light-matter interaction is proposed, which unifies the formulas for scattering and diffraction by decomposing partially coherent light into the Hermite-Gauss space and modeling matter as a locally stationary random process, and enables efficient calculation and description of complex optical phenomena.

    img

    1. Background

    The light observed in daily life is usually composed of many independent electromagnetic waves. Due to the complexity of partially-coherent light, the coherent properties of partially coherent light, such as reflection on microscopic geometric surfaces, the appearance of coating materials, grating effects, etc., cannot be explained by classical radiosity theory.

    The limitations of existing tools only allow for rendering of specific materials and are difficult to generalize.

    2. Motivation

    Building a general-purpose light-matter interaction framework to efficiently process partially coherent light and simplify the complexity of existing computational tools.

    Decomposing light coherence properties, the Hermite-Gauss space is introduced in the hope of decomposing and representing the coherence of light in a computationally feasible way, which is widely applicable to various optical phenomena.

    3. Methods

    Light transport in Hermite-Gauss space.

    Locally-stationary matter model.

    Analysis of light-matter interaction.

    Unifying light-matter interaction formulae.

    [GUILLÉN 2020] A general framework for pearlescent materials

    Wave optics, interference pigment optics, inverse rendering

    Simulate the optical properties of pearlescent materials, and provide a theoretical basis for the design and reverse rendering of pearlescent materials.

    img

    1. Background

    Wide Applications of Pearlescent Materials. These materials have unique gloss and color-changing effects and are widely used in packaging, ceramics, printing, cosmetics and other fields.

    The complex optical processes of pearlescence are derived from multiple scattering and wave optical interference between pigment flakes. Existing models are difficult to fully describe these complex optical behaviors.

    2. Motivation

    Building a More Comprehensive Model for Pearlescent Materials. Existing pearlescent material models do not adequately account for the complex structure of pigments and the effects of the manufacturing process. The goal is to expand the range of pearlescent appearances that can be represented by introducing new optical simulation models.

    A generic pearlescent material model can also be used in reverse rendering.

    3. Methods

    An optical model based on interference pigments is proposed, which takes into account the multilayer structure of pigment flakes, the directional correlation of particles, thickness variation and other characteristics.

    Systematic Study of Parameter Space, exploring the effects of orientation, thickness, and arrangement of pigment flakes on the material’s appearance.

    Inverse Rendering helps interpret light scattering phenomena in the real world.

    [Werner 2017] Scratch iridescence: Wave-optical rendering of diffractive surface structure

    Wave optics, non-paraxial scalar diffraction theory, iridescence effect, microscopic scratches

    A wave optics model based on non-paraxial scalar diffraction theory is used to simulate the iridescence effect on microscopic scratched surfaces, from local spots to smooth reflections at long distances.

    img

    1. Background

    Optical Effects of Scratches: Under directional lighting (such as sunlight or halogen lamps), these scratched surfaces will show complex iridescent patterns, which are caused by the diffraction of incident light by the scratch structure. This cannot be reproduced in the geometric optics model.

    Although existing analytical models are able to reproduce the iridescence effect of some microstructures (such as optical discs), simulation of the optical behavior of locally resolved scratches remains a challenge.

    2. Motivation

    Provide a Wave-Optical Scratch Rendering Framework, which can accurately simulate the optical effects caused by scratches, including light spots, iridescence and other visual phenomena.

    3. Methods

    Wave-Optical Model Based on Non-Paraxial Scalar Diffraction Theory: The method in this paper can accurately simulate the diffraction behavior of light on micro-scale surface features at large angles of incidence and reflection.

    Vector Graphics Representation of Scratch Surfaces.

    Multi-Scale BRDF Model.

    Integration and Optimization in Physically-Based Rendering Systems.

    [Fourneau 2024] Interactive Exploration of Vivid Material Iridescence using Bragg Mirrors

    Wave optics, iridescence effect, Bragg mirror, spectral approximation

    Describes the material iridescence effect of 1D photonic crystals (i.e. Bragg mirrors). Simplifies to a single bounce BRDF for fast computation under certain conditions.

    img

    1. Background

    Iridescence in nature is manifested in organisms, plants or gemstones. It is caused by specific microscopic geometric structures whose size is comparable to the wavelength of visible light. The most prominent example is photonic crystals, which produce structural colors by repeating in one-, two- or three-dimensional structures.

    1D photonic crystals, or optical properties of Bragg mirrors. Most existing works use the classic transfer matrix method to calculate the optical effects of multilayer films, but as the number of films increases, the computational complexity increases significantly.

    2. Motivation

    Simplifying the computation of Bragg mirror reflectance, introducing a more concise, closed-form reflection formula and exploring fast approximation methods in RGB spectral rendering.

    Investigating the effects of rough Bragg layers to explore the influence of surface roughness on optical performance.

    3. Methods

    Introduce the closed-form reflectance formula. Based on Yeh's formula (Yeh88 Formula), do RGB spectral approximation (RGB Spectral Approximation).

    Analyze the effect of roughness on optical transmission.

    The appearance of a rough Bragg layer is efficiently rendered using the Single-reflection BRDF Model.

    [Chen 2020] Rendering Near-Field Speckle Statistics in Scattering Media

    MC path integral, importance sampling, memory effect, speckle, biological tissue imaging

    Simulating speckle statistics under near-field imaging conditions in scattering media accelerates speckle rendering in biological tissue imaging applications and provides support for speckle-based imaging techniques.

    img

    1. Background

    When performing deep imaging in biological tissues, imaging becomes very difficult due to multiple scattering of light inside the tissue. When irradiated with coherent light (such as laser), high-frequency speckle patterns are generated inside the tissue. The statistical properties of speckle patterns, especially the memory effect, provide the basis for tissue imaging techniques (such as fluorescence imaging and adaptive optical focusing).

    The limitations of existing models are that they mainly focus on far-field imaging, while near-field conditions are ignored.

    2. Motivation

    Developing a Physically Accurate and Efficient Model for Near-Field Speckle Rendering.

    Improving Computational Efficiency of Speckle Simulations. The wave equation solver is too computationally intensive.

    3. Methods

    Monte Carlo Path Integral Rendering Framework.

    Aperture and Phase Function Approximations.

    Importance Sampling.

    [Kajiya and Kay 1989] Kajiya-Kay Model

    The originator of hair, no need to say more

    The hair is simplified as a thin and long cylinder, and the light reflection behavior of the hair surface is simulated by extending the Phong model.

    img

    1. Background

    Based on the concept of Phong lighting model, it is extended into an empirical model suitable for hair rendering.

    2. Motivation

    Hair has very unique optical properties, such as specular reflection, subsurface scattering, etc., and the emergence of these phenomena is closely related to the geometric shape and surface structure of hair.

    3. Methods

    The Kajiya-Kay model is based on the idea of the Phong model and is an extension of the Phong model.

    Cylindrical Hair Representation.

    [Marschner 2003] Light Scattering from Human Hair Fibers

    The originator of hair, no need to say more +1

    It is able to capture key visual effects that existing Kajiya-Kay models cannot describe, such as multiple highlights and scattering variations associated with fiber axis rotation.

    img

    1. Background

    Limitations of the Kajiya-Kay Model assumes that hair is only an opaque cylinder, ignoring key phenomena such as internal reflection and transmission.

    2. Motivation

    Hair is a dielectric material, and especially light-colored hair (such as blonde, brown, and red) has significant translucency. Therefore, there is a need for a More Accurate Hair Scattering Model.

    3. Methods

    The 3D full hemispherical light scattering of a single hair was measured.

    The Transparent Elliptical Cylinder Model is proposed.

    Simplified Shading Model.

    [Benamira 2021] A Combined Scattering and Diffraction Model for Elliptical Hair Rendering

    Wave optics, hair rendering, elliptical hair, diffraction scattering lobe function, no pre-calculation

    A new combined scattering and diffraction model that simulates light scattering and diffraction phenomena for hair with an elliptical cross-section without pre-calculation.

    img

    1. Background

    Still with wave optics as the background, when light interacts with objects whose size is close to the wavelength of light, interference and diffraction effects become significant.

    Rendering hair requires considering its geometric properties as well as the wave effects of light. While ray tracing can simulate most scattering phenomena, it falls short when it comes to diffraction in hair.

    2. Motivation

    Addressing Diffraction and Elliptical Cross-sections. A model combining the wave and ray properties of light is proposed to handle the light diffraction phenomenon of hair without pre-calculation. Supports hair fibers with arbitrary elliptical cross-sections.

    3. Methods

    The ray part (Ray Interaction with Elliptical Fibers) introduces a complete light transport model, continues the traditional ray model, and handles most scattering effects.

    The Wave Diffraction by Elliptical Fibers section introduces a new diffraction scattering lobe function that captures the strong forward scattering effect that occurs when light interacts with hair.

    Precomputation-free Approach.

    Integration with Modern Ray Tracers.

    [Zinke 2008] Dual Scattering Approximation for Fast Multiple Scattering in Hair

    Hair rendering, multiple scattering

    The "dual scattering" model is widely used in real-time rendering, and there is no need to explain this classic model.

    img

    1. Background

    In light-colored dense hair, multiple scattering is a key factor in determining the overall hair color.

    Existing methods based on path tracing or photon mapping are too slow to render and often ignore the circular cross-section of hair fibers.

    2. Motivation

    Need for a Physically Accurate and Efficient Multiple Scattering Model.

    3. Methods

    Dual Scattering Model, global multiscattering and local multiscattering. The global multiscattering part aims to calculate the light that passes through the hair volume and reaches the neighborhood of the target point, while the local multiscattering considers the scattering events within this neighborhood.

  • 毛发渲染研究:波动光学的毛发渲染-学习笔记-2

    Hair Rendering Research: Wave Optics Hair Rendering - Study Notes - 2

    Disclaimer: This article is mainly about my personal notes on this black dog hair paper in SIG23. There should be no threshold for reading, because I myself have not entered the graphics field. There are formulas, but not many. The formula tags are the same as the original paper. All the content is just my tinkering. If there are any misunderstandings in the formulas, please correct me. Thank you very much!

    original:https://zhuanlan.zhihu.com/p/809636731

    Keywords: Introduction to graphics, offline rendering, wave optics-based rendering, hair rendering

    Mengqi Xia, Bruce Walter, Christophe Hery, Olivier Maury, Eric Michielssen, and Steve Marschner, “A Practical Wave Optics Reflection Model for Hair and Fur,” ACM Transactions on Graphics (TOG), vol. 42, no. 4, article 39, pp. 1-15, Jul. 2023.

    1. Related Work

    • Ray-based fiber model In the study of human hair, the model of Marschner [2003] is widely used in the industry. It analyzes the light paths in dielectric cylinders and cones and separates the scattering into R, TT and TRT. Zinke [2009] added a diffuse reflection component. Sadeghi [2010] proposed an artist-controlled parameterization method. d'Eon [2014] and Huang [2022] proposed a non-separable characterization method, that is, there is a coupling effect between the azimuthal and longitudinal angles, which cannot be simply separated. Chiang [2016] further optimized the model to make it suitable for production-level rendering. In addition to human hair, Khungurn and Marschner [2017] explored the modeling of elliptical hair. Yan [2015, 2017] studied animal hair with an internal medulla. Aliaga [2017] generalized the model to textile fibers with more complex cross-sections.
    • Fiber Model Based on Wave Optics Linder [2014] resolved cylindrical fibers with perfectly circular cross-sections and investigated the scattering behavior. Xia [2020] demonstrated several important differences compared to the geometric optics model by performing two-dimensional wave simulations on cylinders with arbitrary cross-sections. However, this model is a regular circle and does not have microscopic geometric structures such as hair scales. Benamira and Pattanaik [2021] proposed a faster hybrid model that uses wave optics only for forward scattering and relies on geometric optics for the rest.
    • Plane model simulation based on physical optics Physical optics planar models use physical optics approximations to simulate the scattering behavior of light on nearly planar, rough surfaces that can be represented as height fields, including Gaussian random, periodic, precomputed, and scratched surfaces. They combine Kirchhoff scalar diffraction theory with path tracing methods to handle scattering and reflection, and calculate the diffraction of light on different rough surfaces through various models such as Beckmann-Kirchhoff and Harvey-Shack. Although these models are effective on planar surfaces, the closed nature of fiber geometry and complex light interactions require more complex treatments.
    • Computational Electromagnetics,Spickle Effect and Stylized noise This has been mentioned in the previous article, so I will skip it here.

    https://zhuanlan.zhihu.com/p/776529221

    2. Background Research

    Overview

    Hair modeling is based on scanning electron microscope (SEM) images of hair, which can accurately restore the microstructure of hair, such as hair scales and roughness.

    Next, use "WAVE SIMULATION WITH 3D FIBER MICROGEOMETRY" to calculate the reflection and diffraction of the rough fiber surface.

    On this basis, speckle theory is introduced to analyze the statistical characteristics of scattering patterns, and noise is used to describe these speckles, which greatly optimizes the model.

    Then, by comparing with the actual measured data, reasonable fiber parameters (such as size, skin angle and surface roughness) are derived and finally integrated into the rendering system.

    3. OVERVIEW

    3.1 Fiber scattering models

    I translate this into fiber scattering model. This describes the interaction between single fibers. The key here isBCSDF (Bidirectional Curvilinear Scattering Distribution Function)Unlike the bidirectional scattering distribution function (BSDF) commonly used in surface reflection and refraction, the BCSDF is designed specifically for curved fibers. The following formula states that when a given wavelength of light is irradiated on a fiber, it will be reflected or transmitted from another direction after entering from one direction.
    $$
    L_r(\omega_r, \lambda) = \int L_i(\omega_i, \lambda) S(\omega_i, \omega_r, \lambda) \cos \theta_i d\omega_i \tag{1}
    $$
    The left side of the formula represents the given wavelength $\lambda$,EmissionThe radiant brightness in the direction $\omega_r$. $ L_i(\omega_i, \lambda)$ isIncidentThe radiance in the direction $\omega_i$. $S(\omega_i, \omega_r, \lambda)$ is the bidirectional curvilinear scattering distribution function, which describes how the light is "scattered" by the fiber. $\cos \theta_i$ is to take into account the effect of the angle of incidence. If the light hits the fiber at a very flat angle, its effect will be smaller than when it hits it vertically.

    Writing the above formula in spherical coordinates, and treating each different interaction of light with the hair fiber as a different mode, and then summing up the different scattering terms:
    $$
    S(\theta_i, \theta_r, \phi_i, \phi_r, \lambda) = \sum_{p=0}^{\infty} S_p(\theta_i, \theta_r, \phi_i, \phi_r, \lambda)
    \tag{2}
    $$
    The first scattering term $S_0$ describes the surface reflection, which is often referred to as the direct reflection term $R$ in the past. This term generally represents the statistical average of the reflection from the smooth fiber or the rough fiber surface. This term can be calculated more accurately here in the paper.

    Recall that previous rendering methods (e.g. Marschner [2003]) typically decompose each scattering pattern $S_p$ into two separate functions: the longitudinal function $M_p$ and the anazimuthal function $N_p$ . This approach has been criticized as being inaccurate and should be avoided using the following separable approximation.
    $$
    S_p(\theta_i, \theta_r, \phi_i, \phi_r, \lambda) = M_p(\theta_i, \theta_r)N_p(\theta_i, \phi_i, \phi_r, \lambda)
    \tag{3}
    $$
    Therefore, XIA[2023] samples the scattering parameters of multiple rough fibers and takes the average, denoted as $S_0,avg$ . $f(\theta_h, \phi_h, \lambda)$ is the noise component, represented by two half-range vector angles and wavelength, which is used to correct the deviation of the current specific fiber instance from the mean.
    $$
    S_{0,\text{sim}}(\theta_i, \theta_r, \phi_i, \phi_r, \lambda) \approx S_{0,\text{avg}}(\theta_i, \theta_r, \phi_i, \phi_r , \lambda) f(\theta_h, \phi_h, \lambda)
    \tag{4}
    $$
    Currently, the complete fiber scattering model is as follows. The first term represents the scattering mode of reflection from the fiber surface, combined with 3D wave simulation. The subsequent summation terms are the sum of other higher-order scattering modes.
    $$
    S_{\text{prac}}(\theta_i, \theta_r, \phi_i, \phi_r, \lambda) = S_{0,\text{prac}}(\theta_i, \theta_r, \phi_i, \phi_r, \lambda ) + \sum_{p=1}^{\infty} S_p(\theta_i, \theta_r, \phi_i, \phi_r, \lambda)
    \tag{5}
    $$
    In practice, the final scattering formula is much simpler to accommodate surfaces with more complex geometry.

    3.2 Speckle theory

    Speckle theory describes the random light intensity distribution phenomenon produced when light interacts with a rough surface. The $A$ in Goodman's [2007] formula represents the superposition of all phase vectors, i.e. the resulting phase vector (Phasor):
    $$
    \mathbf{A} = \frac{1}{\sqrt{N}} \sum_{n=1}^{N} a_n = \frac{1}{\sqrt{N}} \sum_{n=1} ^{N} a_n e^{i\phi_n}
    $$
    The above formula is a comprehensive expression of the speckle intensity, in other words, it is used to express the intensity of scattered light. The reason for this phenomenon is that light reflects, refracts and interferes with each other between many tiny surfaces. The phase difference in the propagation of light leads to uneven light and dark patterns. Through this formula, it is possible to statistically calculate how light interferes with each other on the fiber surface to obtain the speckle pattern.

    4. WAVE SIMULATION WITH 3D FIBER MICROGEOMETRY

    In wave optics simulations, light is considered an electromagnetic wave. It consists of magnetic and electric fields that are perpendicular to each other. The interaction of light with hair (such as scattering) can be transformed into an analysis of how the electromagnetic field is affected by the object.

    4.1 Wave optics

    First of all, it is important to understand that when the electromagnetic field changes with a time period, it is called a time-harmonic field. In a time-harmonic field, the electric and magnetic fields can be represented by complex numbers as phase vectors (phasors). The clever thing is that the electric and magnetic fields are perpendicular to each other.
    $$
    E_{\text{inst}} = \Re(E e^{j\omega t}), \quad H_{\text{inst}} = \Re(H e^{j\omega t})
    \tag{6}
    $$
    They are the electric field and magnetic field, respectively, but the real part is taken separately. The complex part contains the amplitude and phase information of the field.

    The following two sets of equations areMaxwell's equationsIn the time-harmonic field form:
    $$
    \nabla \times \mathbf{E} = -\mathbf{M} – j\omega \mu \mathbf{H}
    \
    \nabla \times \mathbf{H} = \mathbf{J} + j\omega \varepsilon \mathbf{E}
    \tag{7}
    $$
    Among them, $\varepsilon$ is the dielectric constant (affects the electric field), and $\mu$ is the magnetic permeability (affects the magnetic field). These two terms describe how the material affects the propagation of electromagnetic fields.

    When an object (such as a fiber) is illuminated by an incident wave, the incident electric and magnetic fields are represented by $\mathbf{E}_i$ and $\mathbf{H}_i$, respectively. But the presence of the object changes these fields, so that what we observe isTotal Field.
    $$
    \mathbf{E}_1 = \mathbf{E}_i + \mathbf{E}_s, \quad \mathbf{H}_1 = \mathbf{H}_i + \mathbf{H}_s
    \tag{8}
    $$
    In short, total field = incident field + scattered field.

    The light energy propagates outward along with the scattered fields $\mathbf{E}_s$ and $\mathbf{H}_s$, so calculating the scattering function of the fiber is the key. How to calculate it?

    Full-wave simulation is the most accurate. Full-wave simulation requires discretizing the object into a grid and requires high resolution (generally, at least 10 grid cells per wavelength), which requires processingMillions of grid cells. It was also mentioned in the previous article.

    That is to say, even simulating a short fiber segment that is only tens of microns long requires processing millions of grid cells.

    Therefore, using Physical Optics Approximation (PO)In PO, the electric current and magnetic current on the surface of the object (respectively denoted as $J$ and $M$) can be regarded as the secondary source of the scattered field. The electromagnetic current generates secondary radiation, forming scattered waves. PO assumes that only a single reflection occurs on the surface of the object, ignoring multiple reflections and complex diffraction effects. After obtaining the electric current and magnetic current on the surface, the scattered waves generated by them are calculated. The properties of these far-field waves are derived from the surface electric current and magnetic current.

    Just one PO is not enough, an octree algorithm must be incorporated to accelerate far-field calculations.

    4.2 Physical Optics Approximation

    Specifically, PO makes two simplifying assumptions: single scattering and local plane assumption. This method is also general enough.

    As shown in the figure above, multiple points are sampled on the surface of an object, which is approximated as a plane. The current and magnetic field on the tangent plane are calculated to generate a scattering field. Through this scattering field, the far-field scattered waves generated by these currents and magnetic currents are calculated. At the same time, the Octree structure is used to divide it into smaller voxels. The calculation results of each leaf node are aggregated to the parent node to obtain the total radiation contribution.

    Surface current calculation.

    The calculation of surface current and magnetic current is a critical step in PO simulation. For each sampling point, its normal vector $n(r{\prime})$ and area information are stored. Next, the interaction between the incident wave and the surface current is calculated for each small plane.

    According to the direction of the incident wave $e_i$ and the surface normal $n(r{\prime})$, the incident electric field is decomposed into parallel polarization and perpendicular polarization components. The reason for the decomposition is that parallel polarization and perpendicular polarization are completely different expressions in the Fresnel equation.

    Parallel polarization component: The electric field of light is parallel to the reflection plane of the incident light.
    Vertically polarized component: The electric field of light is perpendicular to the reflection plane of the incident light.

    Therefore, the incident field $E_i$ is decomposed into a parallel polarization component $E_i^p$ and a perpendicular polarization component $E_i^s$, which looks like this: $E_i = E_i^p + E_i^s$.

    The reflected field $E_r$ is expressed as the sum of parallel and perpendicular polarization components, and the coefficients next to the incident field represent the reflection from the Fresnel equations:
    $$
    E_r = E_r^p + E_r^s = F^p E_i^p + F^s E_i^s
    \tag{9-1}
    $$
    Then, the total electric field $E_1$ is the sum of the incident and reflected fields:
    $$
    E_1 = E_i + E_r
    \tag{9-2}
    $$
    According to high school physics, we know that electricity can generate magnetism, and magnetism can also generate electricity. Therefore, we have the following expression:
    $$
    M = -n \times E_1,
    J = n \times H_1
    \tag{10}
    $$
    Therefore, the induced current and magnetic flux on the surface can be calculated based on the incident field and the reflected field. After obtaining the current and magnetic flux, the scattered wave in the far field is calculated.

    In theory, the incident field can be anything. But here the author uses Gaussian-windowed plane waves. The amplitude of this wave follows a normal distribution and is easy to calculate.

    To summarize, here we decompose the incident light into two components, and use the Fresnel equation to calculate the reflected field. Then we add the reflected field to the incident field to get the total field. Through the relationship between the electric/magnetic field and the electric/magnetic current, we calculate the induced electric/magnetic current on the surface. In this way, we can simulate the scattering of the fiber.

    That is to say,Calculation of electric and magnetic currents on the fiber surfaceThe scattering behavior of light can indeed be obtained.

    Far-field radiation in 3D

    In the previous section, we used the surface currents to obtain the surface currents and magnetic fluxes. This section uses this information to calculate the far-field scattering.

    based onHuygens's PrincipleConvert the original scattering problem into a radiation problem.

    Huygens' principle states that the electricity at each wavefront can be considered a new secondary wave source. It feels like one thing leads to another.

    The electric current $J$ and magnetic current $M$ on the surface of the hair fiber are regarded as secondary sources of light, which are then reradiated to obtain the scattered field.

    This is a bit difficult, as it involves the Method of Moments (MoM) in electromagnetism. Readers can study Gibson's [2021] book in depth. If you learn it, you must teach me well. But it doesn't matter. I found a picture from Professor Yan's 2021 paper, and I guarantee you can understand it.

    The red ball in the figure representsSecondary radiation source, each radiates outward, similar to the secondary radiation waves generated by surface currents and magnetic currents.
    Secondary radiation sourceEmit waves of equal intensity in all directionsThis is consistent with the idea that each surface point in the scattering formula in this paper contributes to scattering in all directions.
    The figure shows a small area ( $\delta \mathbf{r}$ ) far from the light source (the distance is $r$ ).Far field areaIn , the behavior of the analytical wave at different locations $\mathbf{r}_1$ and $\mathbf{r}_2$ is observed along the two directions $\hat{r}_1$ and $\hat{r}_2$ respectively. This is very similar to the behavior of the scattered electric field in the far field in the scattering formula.
    The light wave on the right side of the figure willFar field superpositionComplex interference fringes are formed, which is similar to the integral term in the scattering formula, superimposing the electromagnetic waves of the secondary radiation source at a distance.

    The formula is:
    $$
    E_s(\mathbf{r}) = j \omega \mu_0 \frac{e^{-jk_0 R}}{4\pi R} \hat{r} \times \int_\Gamma \left[ \hat{r} \times \mathbf{J}(r{\prime}) + \frac{1}{Z_0} \mathbf{M}(r{\prime}) \right] e^{jk_0 r{\prime} \cdot \hat{r}} d\mathbf{r{\prime}}
    \tag{11}
    $$
    Combined with the figure, the formula describes the performance of the scattered electric field $E_s(\mathbf{r})$ at a certain point $\mathbf{r}$ far away (far field), and decays with the distance in a relationship of $1/R$. This formula is a two-dimensional complex plane, that is, the solution of Maxwell's equations is regarded asTime-harmonic electromagnetic wavesin the form of.

    Looking at the structure of the above formula, it is very similar in form to $\mathbf{E}(\mathbf{r}, t) = \mathbf{E}_0 e^{j(\omega t – k \cdot \mathbf{r})}$, which is called the common time-harmonic solution of the electric field.

    $ j \omega \mu_0$ This imaginary term describesEffect of Magnetic Field on Electric Field, which is related to the frequency of the electromagnetic field$\omega$ and the magnetic permeability$\mu_0$ in the vacuum. The higher the frequency of the light wave, the stronger the electric field.$e^{-jk_0 R}$ is the phase factor, which represents the phase change of the light wave during propagation. The wave number$k_0 = \frac{2\pi}{\lambda}$, $\lambda$ is the wavelength of the electromagnetic wave. It means that when the wave propagates to a distance R, the phase of the electric field will change.$\hat{r}$ is the unit vector pointing from the scattering object to the observation point, indicating the direction of wave propagation. The cross product$\times$ operator indicates that the direction of the electric field is calculated. Make sure that the calculated electric field is consistent with the direction of wave propagation.

    The integral term $ \int_\Gamma \left[ \hat{r} \times \mathbf{J}(r{\prime}) + \frac{1}{Z_0} \mathbf{M}(r{\prime}) \right] e^{jk_0 r{\prime} \cdot \hat{r}} d\mathbf{r{\prime}} $ is the core of the formula. By integrating $\Gamma$ on the surface of the object, we can get the contribution of each point on the hair surface to the scattered electric field. The surface current and surface magnetic current at each point are summed, and then multiplied by the phase factor. Furthermore, $\mathbf{J}(r{\prime})$ is the surface current density, $\hat{r} \times$ ensures that the scattered electric field generated by the current is orthogonal to the wave propagation direction$\hat{r}$. $Z_0$ is the free space impedance, and the specific value$Z_0 = \sqrt{\frac{\mu_0}{\varepsilon_0}} \approx 377 \, \Omega$. I personally understand that this constant is the proportional relationship between the electric field and the magnetic field in a vacuum. The electric field and the magnetic field have different impedances, so they need to be scaled to be linearly added, that is, the magnetic current is normalized to a form similar to the current. The exponential term is also a phase factor, so I won't explain it in detail.

    Careful readers may find out why the surface current density $\mathbf{J}(r{\prime})$ has a cross product $\hat{r} \times \mathbf{J}(r{\prime})$ , while the surface magnetic current density $\mathbf{M}(r{\prime})$ does not.

    According to Maxwell's equations, the electric field and magnetic field are orthogonal to each other. The current $\mathbf{J}$ will generate a magnetic field, and the changing magnetic field will in turn generate an electric field.Electric and magnetic fields are coupled. However, let us revisit the following two parts of Maxwell's equations.
    $$
    \nabla \times \mathbf{E} = -\frac{\partial \mathbf{B}}{\partial t}
    \
    \nabla \times \mathbf{B} = \mu_0 \mathbf{J} + \mu_0 \varepsilon_0 \frac{\partial \mathbf{E}}{\partial t}
    $$
    The generation of magnetic field is achieved directly by displacing electric field, and the direction is orthogonal to the electric field. The generation of electric field is the rate of change of magnetic field, and the directionality needs to be adjusted.

    To understand it from another perspective, the left side of the integral term describesThe strength of the electromagnetic field decreases with distance, phase variation and directivity are also taken into account. The integral term describesElectromagnetic current and phase contribution at each surface point.

    In summary, formula (11) is an accurate expression that describes how the electromagnetic field on the surface of an object is scattered at a distance of $r$. With the formula for the electric field, the magnetic field is easy to calculate.

    When the scattering distance $R$ is large enough, the simplified expression of the far field is as follows, and the near-field effect can be directly ignored. In other words, in the far field, the propagation characteristics have become stable, so the integral term can be simplified to a term related to the scattering direction $\hat{r}$, that is, $E_s^{\text{far}}(\hat{r})$.
    $$
    E_s(r) = \frac{e^{-jk_0 R}}{R} E_s^{\text{far}}(\hat{r}), \quad H_s(r) = \frac{e^{- jk_0 R}}{R} H_s^{\text{far}}(\hat{r})
    \tag{12}
    $$
    In addition, if the contribution of each point is calculated directly, the time complexity is $O(MN)$, that is, the number of discrete points is $M$ $*$ and the number of scattering directions is $N$. Here, the octree is introduced to reduce the number of discrete points on the calculation surface through space partitioning, and the time complexity is reduced to $O(M+log(M)N)$. The specific implementation method is in 4.3.

    4.3 Multilevel fast Physical Optics

    Accelerate far-field scattering calculations using Multilevel Fast Physical Optics (MFPO).

    The multi-layer fast multipole algorithm (MLFMA) proposed by Chew [2001] is used to accelerate the solution of electromagnetic field scattering problems. This algorithm first constructs an octree structure for the surface of hair, for example, where each leaf node represents a sampling point. After the Octree is constructed, the surface current and magnetic current of each leaf node are calculated. Then, starting from the leaf node, the data are accumulated layer by layer. The original $O(MN)$ is reduced to $O(M + \log(M)N)$.

    The author then introduces the three key parts of the octree acceleration algorithm. The first is the far-field scattering kernel.
    $$
    e^{jk_0 r{\prime} \cdot \hat{r}} = e^{jk_0 (r{\prime} – c_L) \cdot \hat{r}} \prod_{i=1}^{L} e^{jk_0 (c_i – c_{i-1}) \cdot \hat{r}}
    \tag{13}
    $$
    The ultimate goal is to calculate the scattering contribution of the electromagnetic wave at each point on the surface of the object to a far-field observation area (distance $R$, direction $\hat{r}$). Specifically, the scattered electric field $E_s(r)$ and magnetic field $H_s(r)$ are calculated, that is, the contribution of the electromagnetic wave emitted from each point $r{\prime}$ on the hair surface in the far field. The left side of the formula is the phase change from a surface point $r{\prime}$ to the far-field observation direction $\hat{r}$. The final time complexity is $O(MN)$. The author divides the surface into different regions, each of which specifies a reference center point. The $c_0, c_1, …, c_L$ in the formula are the node centers of different levels in the octree.

    This alone cannot reduce the amount of calculation. Therefore, it is necessary to merge the contributions of surface points that are closer to each other through the high-level nodes of the octree because their phase changes are very small.

    Let's use Professor Yan's picture to explain. For a certain area $\delta \mathbf{r} $ in the far field, the contribution of all sampling points to the reference point $\mathbf{r}$ will be accurately calculated first. However, other points in the area are approximated using the parent node of the octree. In other words, the nearby $\mathbf{r}_1$ and $\mathbf{r}_2$ no longer need to consider so many sampling points, but directly use the total contribution obtained by the octree.

    The author defines a direction set, and each parent node of the octree stores the cumulative contribution data about different direction sets. Therefore, the parent node contains not only spatial information, but also cumulative information in multiple scattering directions. Finally, a complete 360-degree scattering field distribution is obtained at the root node.

    Next, starting from the second level of the tree, merge upwards in sequence. The specific method of upsampling is to perform forward FFT on the scattering contribution of the child node in the direction, then expand the frequency domain data with zero-padding, and finally convert it to the spatial domain through inverse FFT. Ultimately, the parent node can obtain more accurate scattering information in different directions.

    • Performance

    Octrees have a good acceleration effect. Triple trees have the best effect. The more complex the fiber, the better the optimization effect.

    • Fiber microgeometry and scattering patterns

    The cross section of a hair fiber is generally not a perfect circle, but an ellipse. The geometric parameters of the fiber are defined by the major radius $r_1$ and the minor radius $r_2$ of the ellipse. In order to simulate the microscopic roughness of the fiber surface, the author superimposed aGaussian random height field, simulating the real fiber surface. Furthermore,Cuticle tilt, the simulated flakes are arranged obliquely on the fiber surface.

    By comparing with traditional light-based hair models, wave optics simulations found that in addition to the significant forward-scattering phenomenon predicted by XIA[2020], complex wavelength-dependent granular patterns were observed, which generated rich color effects when converted to RGB colors.

    The study pointed out some regularities observed from the simulations:

    • Regardless of their actual position, fibers with the same geometric parameters (such as radius, roughness, cuticle tilt, etc.) will generateSimilar grain patterns.
    • If the fibers have different geometrical parameters, they will generateParticle patterns with different statistical properties, that is, the scattering patterns are obviously different.
    • The position of the scattered spots depends on the incident angle of the light, and the direction of the deviation followshalf vectordirection.
    • The size of the speckle pattern increases with the wavelength of the light, which is consistent with the results of Goodman [2007].

    5. A PRACTICAL FIBER SCATTERING MODEL

    A PRACTICAL FIBER SCATTERING MODELDesigned to account for microscopic geometric variations and complex scattering behavior of fibers.

    Previous studies generally used Lut to store the scattering distribution function. This method consumes a lot of space and requires simultaneous recording of the longitudinal and azimuthal scattering distributions.

    Now, the authors propose aWavelet-based noise representationThe compact fiber scattering model can represent more geometric complexity and achieve better scattering effect.

    In short, the authors want to compact the statistical speckle phenomenon so that it can beMean,variance,Autocorrelation function(ACF) and other statistics. Therefore, the author usedSpeckle TheoryTo describe the patterns produced by the random interference of light.

    5.1 Speckle statistics

    Here the author mentionedFully Developed SpeckleThis concept. The following is my personal understanding. When light hits a rough surface (such as a fiber/hair surface), each small area on the surface will scatter the light. Due to the tiny irregularities of the surface, the scattered light will interfere with each other, producing a complex light intensity distribution. This distribution is manifested as a series ofBright spots and dark spots, which we callSpeckleThe tiny features of the surface (such as roughness) become sufficiently irregular across the illuminated area (relative to the wavelength of the light) that the phase and intensity of the scattered light at each point is random, which is calledFully developed speckle.

    At this time, the fully developed speckle can be described by the complex Gaussian distribution of Goodman [2007]. That is, the real part $\mathcal{R}$ and the imaginary part $\mathcal{I}$ of the electromagnetic field obey the complex Gaussian distribution in space.
    $$
    p_{\mathcal{R},\mathcal{I}}(\mathcal{R}, \mathcal{I}) = \frac{1}{2 \pi \sigma^2} \exp\left( – \frac {\mathcal{R}^2 + \mathcal{I}^2}{2\sigma^2} \right)
    \tag{14}
    $$
    The real and imaginary parts of the field are independent and normally distributed, with zero mean and the same variance.

    The electromagnetic field intensity $I$ and the probability density function of the intensity distribution obeying the exponential distribution:
    $$
    I = \mathcal{R}^2 + \mathcal{I}^2 \ p_I(I) = \frac{1}{2\sigma^2} \exp\left( -\frac{I}{2\sigma ^2} \right)
    \tag{15}
    $$
    There is nothing much to say about these formulas. In short, the speckle field is very random!

    The author makes the light intensity follow an exponential distribution to ensure the statistical characteristics in a single direction. Secondly, by studying the statistical relationship of the light intensity between two points, the ensemble average of the two points is measured. Here, the autocorrelation function (ACF) is used.
    $$
    C(I_{p_1}, I_{p_2}) = \frac{\overline{(I_{p_1} – \overline{I_{p_1}})(I_{p_2} – \overline{I_{p_2}})} }{\sigma(I_{p_1}) \sigma(I_{p_2})}
    \tag{16}
    $$
    The value of the autocorrelation function is between -1 and 1.When the value is close to 1, indicating that the scattering behaviors of the two light intensities are very similar. This is the key to efficiently reproducing the particle structure in the fiber scattering field.

    5.2 Wavelet noise representation of the speckles

    The author introduces wavelet noise to represent the noise component of speckle $f(\theta_h, \phi_h, \lambda)$. The specific formula is as follows:
    $$
    f(\mathbf{x}) = \sum_{b=0}^{n-1} w_b(\mathbf{x}) I\left(2^b g_{\lambda}(\mathbf{x})\ right)
    \tag{17}
    $$
    The core idea of the formula is to decompose the light intensity of the speckle into noises of different frequency levels and perform weighted combination.

    By adjusting the weights of different frequency bands, the generated autocorrelation function $ C_f(\mathbf{x}_1, \mathbf{x}_2)$ is close to the final noise with the target autocorrelation function.

    According to the Wiener-Khinchin theorem, the autocorrelation function can be expressed byFourier TransformTo calculate. The specific formula is as follows:

    $$
    C_f(\mathbf{x}_1, \mathbf{x}2) = \mathcal{F} \left( \mathcal{F}^{-1} \left( \sum{b=0}^{n-1 } w_b I_b \right)^2 \right)
    \tag{18}
    $$
    This means that we can obtain the autocorrelation function by calculating the Fourier transform of the wavelet noise. The autocorrelation function and the power spectral density function are a pair of Fourier transforms, which is amazing.

    Here the original paper gives a proof of approximating the ACF by weighted summation of frequency bands.

    Traffic saving: By weighting each frequency band of the noise, the autocorrelation function of the entire noise can be approximated without dealing with the interactions between different frequency bands.

    The non-negative weights $v_b$ are found by the least squares method so that the weighted sum of the autocorrelation functions $C_b(\mathbf{x}1, \mathbf{x}_2)$ of each frequency band can approach the target autocorrelation function $C_t(\mathbf{x}_1, \mathbf{x}_2)$ .

    $$ C_t(\mathbf{x}_1, \mathbf{x}2) \approx \sum{b=0}^{n-1} v_b C_b(\mathbf{x}_1, \mathbf{x}_2) \tag{19} $$ After calculating the weights, we need to ensure energy conservation. That is, we adjust the expected value of the noise function $f(\mathbf{x})$ to $\mathbb{E}[f(\mathbf{x})] = 1$ . $$ \begin{aligned} & \mathrm{E}\left[S{\text {avg }}\left(\theta_i, \theta_r, \phi_r, \phi_r, \lambda\right) f\left(\theta_h, \phi_h, \lambda\right)\right] \\\
    & =S_{\text {avg }}\left(\theta_i, \theta_r, \phi_r, \phi_r, \lambda\right) \mathrm{E}[f(\mathbf{x})] \
    & \approx S_{\text {avg }}\left(\theta_i, \theta_r, \phi_r, \phi_r, \lambda\right)
    \end{aligned}
    \tag{20}
    $$


    Although the effect is good, it has limitations. For example, at grazing incidence angles (when the light is almost parallel to the surface), the fitting accuracy decreases, resulting in a decrease in scattering accuracy. Degeneration in the Forward Direction problem, when the light is in the forward direction (the light direction is consistent with the surface normal), the half-vector direction will degenerate.

    6. Validation

    6.1 Wave simulation validation

    The scattered intensity in the xy plane was calculated and calculated through 3600 azimuth angles $\phi_r$, and finally these angles were averaged into 360 directions.

    First, let's compare it to Mie scattering. It is not as good as Mie scattering at grazing angles. The PO approximation is more accurate when the radius of the object is large relative to the wavelength. It is inaccurate when the curvature of the object is small.

    Then compare BEM. To simulate the wave scattering of a small ellipsoid, BEM took three hours, while PO took two seconds.

    Finally, let’s compare 2D BEM. 1D Gaussian height fields are wrapped around circular and elliptical cross sections. PO wins hands down.

    6.2 Measurement

    A HeNe laser with a wavelength of 633 nm was used, and the beam spot size was 0.7mm (along the length of the hair) × 3mm (perpendicular to the hair direction)The laser is shone through a tiny hole onto a small area of the human hair sample.

    In short, the effect is good!

    6.3 Noise representation validation

    In one word, good!

    7. Rendering

    Integrated into PBRT-v3. The original ray-based model is denoted as $S_{\text{ray}}$ , while the diffraction model is denoted as $S_{\text{diffract}}$ .

    For diffraction, it is approximated here as single-slit diffraction, where the width of the slit is equal to the diameter of the cylinder (fiber).
    $$
    f_{\text{diffract}}(\theta_i, \phi_d, a) = a \cos \theta_i \cdot \text{sinc}^2(a \cos \theta_i \sin \phi_d)
    \tag{22}
    $$
    This diffraction model is used to combine with the longitudinal function to obtain a completeBidirectional scattering distribution function (BCSDF)Precompute the diffraction factors in the form of a table of size $50 \times 50 \times 200$ and use a table of the same size for importance sampling.

    The extinction cross section can be simply understood as the "effective area" over which light interacts with an object. The extinction cross section of a fiber will be larger than its actual geometric cross section. The extinction cross section will be close to twice the geometric cross section. Light is both reflected and diffracted in a fiber, so we need to divide the total energy between these two phenomena. As a rule of thumb, half of the energy can be used for diffraction and the other half for reflection.
    $$
    S_{\text{diffract}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) = \frac{1}{2} \left[S_{\text{ray}}(\theta_i, \phi_i , \theta_r, \phi_r, \lambda) + f_{\text{diffract}}(\theta_i, \phi_d, D/\lambda)\right]
    \tag{23}
    $$
    Through importance sampling, reflection and diffraction of light are fairly considered.

    Next, the final rendering formula describing the light scattering phenomenon is in a nice form:
    $$
    S_{0,\text{prac}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) = S_{0,\text{avg}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) f(\theta_h, \phi_h, \lambda)\ \\
    = \frac{1}{2} \left[ S_{\text{ray}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) + f_{\text{diffract}}(\theta_i, \ phi_d, D/\lambda) \right] f(\theta_h, \phi_h, \lambda)\
    \tag{24}
    $$
    Reflection + Diffraction + NoiseThe noise function $f(\theta_h, \phi_h, \lambda)$ is at the end, so that the scattered light is no longer concentrated in a few directions.

    Focus on this part:
    $$
    S_{0,\text{prac}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) = S_{0,\text{avg}}(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) f(\theta_h, \phi_h, \lambda)
    $$
    $S_{0,\text{avg}}$ represents the average reflection/refraction and diffraction behavior of the hair surface, which is calculated by preprocessing (BSDF table).

    Next, let’s talk about how to extract BCSDF from PO.

    The scattered electric and magnetic fields ($E_s^{\text{far}}$ and $H_s^{\text{far}}$) are obtained through Maxwell's equations.

    Then the Poynting vector is used to calculate the energy flow, which is equivalent to calculating the intensity of light.


    $$
    \langle S \rangle = \frac{1}{2} \text{Re}(E \times H^) \tag{25}
    $$

    In order to use the simulation results for rendering, the scattered intensity is related to the incident power. The scattered intensity is calculated by the formula, where $R^2$ is the far field spherical area:

    $$
    I_s(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) = \langle S(\mathbf{r}) \rangle \cdot \hat{n} R^2 \tag{26}
    $$

    The scattered power $P_s$ and the absorbed power $P_a$ are calculated by integrating the intensities of the scattered light and the absorbed light, respectively.

    $$
    P_s = \int_{\Omega} I_s(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) \, d\omega \tag{27}
    $$

    The absorbed power $P_a$ is calculated by integrating the Poynting vector over the surface.

    $$
    \begin{aligned}
    P_a & =\int_{\Gamma} \frac{1}{2} \operatorname{Re}\left(\mathbf{E}1 \times \mathbf{H}_1^{}\right) \cdot \hat{ \mathbf{n}}_1(A) d A \ & = \\
    \int{\Gamma} \frac{1}{2} \operatorname{Re}\left(\mathbf{J}^{} \times \mathbf{M}\right) \cdot \hat{\mathbf{n} }_1(s) ds
    \end{aligned}
    \tag{28}
    $$


    Finally, BCSDF was created.

    $$
    S(\theta_i, \phi_i, \theta_r, \phi_r, \lambda) = \frac{I_s(\theta_i, \phi_i, \theta_r, \phi_r, \lambda)}{|P_a – P_s|}
    \tag{30}
    $$

    Finally, a 5D table was used.

    The 1D dimension describes the wavelength.
    2D describes the direction of incident light.
    2D describes the direction of the outgoing light.

    The storage requirements can be reduced through the memory effect of scattering.

    When rendering, for each incident light direction, query the adjacent table data and apply the corresponding angle offset (angle offset in memory effect).

    The final table size used is $25 \times 32 \times 72 \times 180 \times 360$ , which is very large, requiring about 15GB of memory.

    8. Result

    Compared with XIA[2020], the new model is more vivid. The colorful glints are better. It reflects the glints produced by multiple wavelengths on the hair.

    9. DISCUSSION AND CONCLUSION

    The first practically applicable 3D wave optics fiber scattering model.

    Capable of generating speckle patterns commonly seen in optical scattering.

    Although the 3D simulation is highly accurate, its memory usage and computational cost are very large. A noise utility model is proposed to reduce the computational cost by capturing the statistical properties of the fiber scattering spot (such as the autocorrelation function).

    Currently, this model is mainly used to simulate the first-order reflection mode. In the future, it can be extended to higher-order scattering modes.

    Further development of new techniques may be needed to predict or measure the statistical properties of higher-order modes.

  • 毛发渲染研究:从基于光线到波动光学-学习笔记-1

    Hair Rendering Research: From Light-Based to Wave Optics - Study Notes - 1

    original:https://zhuanlan.zhihu.com/p/776529221


    The historical evolution of hair rendering research

    In 1989, Kajiya and Kay extended the Phong model to hair drawing and proposed an empirical model for hair drawing, the Kajiya-Kay model. This model simplifies hair into a series of elongated cylinders and assumes that light is simply reflected on the surface of hair. Specular reflection: highlights. Diffuse reflection: simulates the scattering of light inside the hair and the overall brightness of the hair.

    In 2003, Marschner et al. published the paper "Light Scattering from Human Hair Fibers", proposing a physics-based hair reflection model, known as the Marschner model.

    Traditional hair rendering models, such as the Marschner model and the Kajiya-Kay model, usually simplify hair into a single-layer cylinder, ignoring the influence of the medulla. In their 2015 paper "Physically-Accurate Fur Reflectance: Modeling, Measurement and Rendering", Yan et al. proposed a more accurate and efficient hair rendering model to address this problem, modeling each hair as two concentric cylinders. The modeling combines the complete hair bidirectional scattering distribution function (BSDF) to accurately describe the multipath propagation and scattering behavior of light in hair. In addition, to ensure physical authenticity, a large amount of physical measurement work was carried out, including nine different hair samples, and finally some reflection parameters of the database were opened for artists to adjust. In order to improve rendering efficiency, Yan [2015] combined the consideration of near-field scattering (R, TT, TRT) of [Zinke and Weber 2007], pre-calculated common scattering paths, stored in Lut, and realized efficient light scattering calculation of single hair fibers. For rendering pipeline optimization, Yan [2015] mainly focused on the dual-cylinder model.

    From Marschner[2003] to the extended models of d'Eon[2011] and Chiang[2016], although the continuous increase of hair parameters (such as azimuthal roughness numerical integration of d'Eon[2011] and near-field azimuthal scattering of Chiang[2016]) has increased rendering accuracy, its complex scattering path and large amount of pre-calculation limit its practicality and real-time performance. The double-cylinder hair reflection model proposed by Yan[2015] also has the problems of high computational cost and low practicality. Therefore, Yan[2017] proposed a simplified version, which achieves fast integration through analytical methods and greatly reduces the number of Lobes.

    The figure above clearly shows that Marschner [2003] used a longitudinal-azimuthal decomposition representation to simplify the complex three-dimensional light scattering process into two relatively independent dimensions. The longitudinal scattering function describes the propagation and scattering of light along the axis of the hair fiber. The azimuthal scattering function describes the scattering of light in the cross section of the hair fiber (the plane perpendicular to the fiber axis). This model considers T, TT and TRT. The energy conservation problem was corrected in d'Eon [2011]. Yan [2015]'s double cylinder model (hair cuticle and hair medulla) complicated the light interaction and considered R, TrT, TtT and TrRrT. Yan [2017] introduced a unified refractive index (IORs) to simplify the light path propagation and no longer distinguish the refractive indices of different materials, namely R, TT, TRT, TT^s and TRT^s (^s represents the simplified path). Yan[2017] pointed out that unifying IORs does not actually significantly affect the rendering results, and it can still maintain a high degree of realism becauseThe refractive index of the hair cortex and medulla is very close..

    In order to solve the problem of high computational complexity in previous hair rendering, Yan[2017] proposedDivision of near field and far field, and introducedLayered Rendering Strategy. The near field is mainly the fine scattering and reflection of light on a single hair. This area requires a high-precision physical model to render hair, such as wave optical phenomena such as interference and diffraction. The far field describes the overall scattering effect of light under the collective action of a large number of hair fibers. In this area, the microscopic structure of a single hair can be averaged, which is more suitable for calculation using statistical/approximate methods to optimize and improve rendering efficiency.Classification criteria: Based on the distance between the light and the hair fiber and the size of the hair fiberThat is, set aThresholdWhen the distance between the light and the hair is less than the threshold, it is classified as near field; otherwise, it is classified as far field.Layered rendering processFirst, ray tracing is used to determine whether it is near field or far field. In the near field, the Mie scattering theory and Fresnel equation are used to calculate reflection and transmission in combination with the pre-calculated scattering table. In the far field, the statistical scattering function + pre-calculated scattering table + MC integral are used to reduce the complexity. Finally, the two are superimposed and normalized. Here is a detailed explanation based on the three points of the paper:

    • Simple Reflectance ModelAlthough Yan's model [2015] introduced the medulla and considered more physical details, it still has high computational complexity, especially in the conversion between near field and far field. Yan [2017] proposedSimplified hierarchical reflection model, retaining the key physical phenomena of reflection and reducing unnecessary complex light scattering paths. They describe reflection as three main light scattering paths (R: specular reflection of light on the surface of the hair cuticle, TT: light passes through the hair cuticle and is transmitted from the other side after internal scattering, TRT: light enters the hair, reflects once inside, and finally transmits). Finally, combined withSimplified Bidirectional Scattering Distribution Function (BSDF)The reflection of the capture path reduces the number of lobes required in the calculation (usually used to describe the distribution curve of different scattering directions). Compared with the previous model, the number of lobes in the calculation process is reduced from 9 to 5.
    • Improved Accuracy and PracticalityHigh-precision models (such as the Marschner model, Yan [2015], etc.) require complex numerical calculations and a large amount of pre-calculated data, and are therefore difficult to implement in real-time applications. Low-precision empirical models lack sufficient physical reality. Therefore, many improvements were made in Yan [2017]. Although the model simplifies the light scattering path,Combining physical phenomenaThe model is more accurate than the traditional empirical model by reasonably simplifying the longitudinal and azimuthal scattering of light.Transition processing between near field and far fieldTraditional models often fail to smoothly handle the optical transition between the near field and the far field. Yan et al. introducedA near-field-far-field analysisThe solution accurately simulates reflections when the light is close to the hair fiber, while quickly approximating the overall reflection behavior of the light in the far field. This makes the rendering efficient enough for real-time rendering.
    • Analytic Near/Far Field SolutionThere is a huge difference in the treatment of the near field (the short-range interaction between light and a single hair fiber, i.e., scattering behavior) and the far field (the long-range collective effect between light and a large number of hair fibers). In order to achieve a seamless transition between the near field and the far field, the authors used an analytical integration method instead of cumbersome numerical integration. The analytical integration can directly calculate the reflection function without the need for complex numerical solutions or pre-calculations, which greatly reduces the calculation time.
    • Significant Speed Up
      • Reduce the number of scatter paths used to describeNumber of lobes;
      • Combination of analytical integration and pre-calculation;
      • A simplified BSDF and analytical reflection calculation formula are used to combine ray tracing and reflection calculation inParallelization on GPU, the rendering speed of the model is increased by 6-8 times compared with previous methods.

    To summarize briefly, the reflection model proposed by Yan [2017] has good effects and performance. By unifying the IOR of the cortex and medulla, the model only needs 5 lobes to represent the complex scattering of fur, and the tensor approximation is used to minimize the storage overhead. Based on this model, the analytical integration of the far and near fields is proposed to extend the model to multi-scale rendering. It is very simple to implement the BCSDF model in real-time rendering. There are already many implementation methods, and it has been applied to the film and television industry. [The Lion King (HD). 2017 movie] (2019 Oscar Nominee for Best Visual Effects)

    XIA[2023] proposed a hair reflection model based on wave optics. Traditional hair rendering models are mostly based on geometric optics approximation. These models work well when processing larger hair fibers, but have poor performance on subtle optical phenomena (such as colored spots on hair, i.e.glints). These scattering effects, including reflection, transmission, and multiple scattering, are difficult to accurately describe with simple geometric optics models. As the diameter of the hair fiber approaches or becomes smaller than the wavelength of light (visible light), wave optics effects become increasingly important, and geometric optics models are unable to capture these effects.

    Wave optical effects of hair, such asInterference and diffraction of lightThe computational complexity is very high. Wave optics simulation requires calculating the propagation of electromagnetic fields, not just the path of light. Hair and fur have highly irregular microstructures that further affect the scattering of light. Methods based on geometric optics cannot handle these wave phenomena, and full-wave simulations require high computing resources.

    As early as XIA[2020], it was proposed to use wave optics to accurately describe the interaction between light and fibers, and to use the boundary element method (BEM) to simulate the fiber scattering of light at any cross section. In addition, XIA[2020] pointed out that due to the diffraction effect, the fiber exhibits an extremely strong forward scattering effect. Therefore, the wave optics effect should focus the light in the direction of forward scattering. It was also pointed out that the small fiber scattering effect depends significantly on the wavelength of light, resulting in strong wavelength scattering. In addition, the singular softening phenomenon brought by the wave field is also the key to determining the real caustic effect. In order to control the amount of calculation of the BEM simulation, the shape of the fiber is ideally a regular cross-sectional shape. However, Marschner[2003] pointed out that the irregularity of the hair surface has an important influence on the appearance of the hair. Whether such an effect is significant in wave optics is still a problem that needs to be explored and solved.

    Traditional geometric optics methods are based on ray tracing, which predicts the propagation path of light by simulating the reflection and refraction of light on hair fibers. However, this method is insufficient when dealing with light waves with wavelengths comparable to the fiber size, and cannot capture the effects of diffraction.Complex optical effects are produced. In actual measurements, fiber scattering shows some sharp optical features, which are caused by the diffraction effect of light. Including the slight color shift in black dog hair, which is also caused by the interference and diffraction of light.

    In order to deal with these phenomena that cannot be explained by geometric optics, XIA[2023] developed a 3D wave optics simulator based on physical optics approximation (PO) and used GPU to accelerate computational efficiency. The space is processed through an octree structure. The simulator has a certain degree of versatility and can handle arbitrary 3D geometric shapes, that is, it can handle the microstructure of the fiber surface.

    However, XIA[2023] points out that it is unrealistic to directly apply this simulator to the current mainstream rendering framework due to the high computational complexity. Therefore, it is necessary to first migrate the model to the existing hair scattering model and then add aDiffraction lobe of elementary diffraction theoryFinally, aRandom ProcessThe modulation method is used to capture the optical speckle effect. Although it is procedural noise, it is still consistent with the physical simulation result, and the visual effect is close to reality.

    XIA[2023] divides the current hair/fiber rendering into two types: traditionalRay-based Fiber Models, the other isWave-based Fiber Models.

    Linder[2014] proposed an analytical solution to deal with the scattering behavior of cylindrical fibers, but it is only applicable to perfect circular cross-sections and cannot handle complex hair surface structures. XIA[2020] studied the scattering behavior of fibers with arbitrary cross-sectional shapes through two-dimensional wave optics rendering, showing the manifestation of diffraction effects, but the paper assumes a perfect extrusion structure, that is, the fiber surface is regular. Bennamira&Pattanaik[2021] proposed a hybrid model that uses wave optics to solve the problem of only forward diffraction and traditional geometric optics in other scattering modes. However, XIA[2023] further considered theDependence of longitudinal angle of incidence.

    At the end of the paper, the procedural noise is fitted to the speckle pattern in wave optics, and a very realistic effect is produced through statistical property fitting.

    XIA[2023] also mentionedComputational Electromagnetics ToolsIt plays an important role in dealing with complex interactions of rays and fibers, especially when using numerical methods such as BEM.Computational ElectromagneticsIt is a computational method used to study electromagnetic phenomena. Since light is an electromagnetic wave, many phenomena in optics can be analyzed using electromagnetic tools. CEM is often used in optics to calculate the interaction between light and the surface of an object (such as hair fibers). Common CEM algorithms include:

    • Finite-Difference Time-Domain (FDTD): A numerical method for solving Maxwell's equations by discretizing them in space and time, first proposed by Kane Yee in 1966. It is a direct time-domain method Kane Yee[1966], Taflove[2005].
    • Finite Element Method (FEM):It is used to solve the electromagnetic field distribution in complex geometric structures by dividing the solution area into a finite number of elements Jin[2015].
    • Boundary Element Method (BEM): Also known as the Method of Moments (MoM), this is a numerical method that reduces the amount of computation by only dealing with the electromagnetic fields on the surface of an object Gibson[2021], Huddleston[1986], Wu[1977].

    Although CEM has many acceleration algorithms, such as Song[1997]'s Multilevel Fast Multipole Algorithm (MLFMA), the improvement is still minimal in hair and fur simulation.

    Since full-wave simulations are computationally expensive, XIA [2023] proposed the physical optics approximation (PO) to simplify the reflection and diffraction processes on the surface of, for example, hair fibers.

    Physical Optics Plane ModelyesAn application of physical optics (PO) that is specifically designed to simulate the behavior of light on flat or nearly flat surfaces.The scattering and diffraction effects on rough surfaces are effectively calculated by Beckmann-Kirchhoff[1987] and Harvey-Shack[1979]. Gaussian random surfaces by He[1991], Kajiya[1985], periodic static surfaces by Stam[1999], and scratched surfaces by Werner[2017] all use physical optics approximations to deal with surface reflection and diffraction.

    For more complex diffraction, Krywonos[2006], Krywonos[2011] proposed improved methods for processing diffraction on rough surfaces. Holzschuch, Pacanowski[2017] proposed a dual-scale microsurface model that combines reflection and diffraction to simulate rough surfaces. Recently, Falster[2020] combined Kirchhoff scalar diffraction theory and path tracing to handle secondary reflection and scattering. Yan[2018] used physical optics to render the mirror microgeometry of rough surfaces.

    Unlike a flat surface, the fiber surface is a closed curved surface, and the geometric shape of the hair fiber makes the interaction with light more complex. In addition to reflection and scattering, forward diffraction scattering and large-scale shadow effects need to be dealt with.

    XIA[2023] also discussedSpeckle effectandProcedural noiseApplication in hair rendering.

    Speckle is a grainy image or diffraction pattern produced when light interacts with a rough surface. Its statistical properties have been extensively studied. When coherent light (such as a laser) is irradiated onto a rough surface or passes through a scattering medium, a random pattern of light and dark spots is produced. In layman's terms, it's like when you shine a laser pointer on a rough wall, you see a granular, flickering pattern instead of a smooth spot of light. Because light is scattered at tiny surface irregularities, light waves from different paths interfere with each other, some are strengthened (forming bright spots) and some cancel each other out (forming dark spots), resulting in this speckled pattern.

    Previous studies have explored howMonte Carlo methodto simulate the speckle effect in volume scattering Bar[2019, 2020]. However, these models are mainly applicable toHomogeneous media, which is not applicable to heterogeneous structures such as hair fibers. Steinberg, Yan [2022] studied speckle rendering of planar rough surfaces. However, the authors pointed out thatThe speckle effect on fiber surfaces is different from that on flat surfaces, showing different statistical characteristics.

    Therefore, XIA[2023] proposedAccurately capture the statistical characteristics of fiber speckle patternsBy studying the special geometric structure and speckle distribution of the fiber surface, the scattering effect of hair fibers is simulated to provide better speckle effects.

    It should be noted that although both thin-film interference and speckle effect are caused by the interference of light, they have significant differences in physical mechanism, visual performance and rendering methods in computer graphics. Monte Carlo methods of thin-film interference, such as random film thickness sampling, can be used to generate random spots of speckle effect to improve the realism of rendering. Approximate algorithms such as hierarchical thickness sampling and pre-calculated interference patterns can also learn from each other. Thin-film interference often involves interference of light waves at different scales, and speckle effect also involves multi-scale scattering of microscopic surface structures.

    Between the two, thin film interferometry has a relatively low rendering complexity, and pre-computation can be fully utilized to avoid the burden of real-time calculation. However, the speckle effect has highly random and statistical characteristics, and a large number of random interference paths need to be processed, especially for simulating heterogeneous structures such as hair. Current research such as XIA[2023] is working to improve its efficiency, but there is still a large gap compared to thin film interferometry.

    XIA[2023] uses the Wavelet band-limited noise of Cook, DeRose[2005] to control the microscopic geometric changes of hair fibers. This noise is different from conventional procedural noise, such as Perlin[1985], Olano[2002], Perlin, Neyret[2001], etc. A significant advantage of Wavelet noise is that itStatistical distributions can be calculated and controlled.

    The advantage of the practical wave optics fiber scattering model of XIA[2023] is its realisticColored highlights (glints)Previous geometric optics models usually assume that the fiber surface is a smooth dielectric cylinder, without considering the complex interaction of light waves on the surface irregular structure. In actual tests, the XIA[2023] model performs well in rendering time, can be used in production environments, and generates more delicate and realistic optical effects than traditional models.

    XIA[2023] is an important breakthrough that buildsFirst 3D wave optics fiber scattering simulatorPrevious fiber models (including early wave optics models such as Xia et al. 2020) mostly assumed thatLongitudinal and azimuthal directionsThe scattering onSeparable, which greatly simplifies the calculation. However, the authors' simulation results show that the highlights areInseparable, which is a phenomenon that previous models could not accurately handle. The simulator also predictedSpeckle patternsThis is a phenomenon that has not been captured by all previous fiber scattering models based on geometric optics and wave optics.5-dimensional scattering distributionThe method is to use tabulation, which is very memory intensive. Therefore, procedural noise is used to directly replace a five-dimensional table.

    XIA[2023] has only been simulated once so farSpeckle Effect in Reflection Mode, higher order reflection modes are still being studied. And light-colored hair may require higher computational requirements to simulate perfectly. The wave optics fiber scattering model used in this study canEasily combined with previous fiber models.

    References

    Zotero one-click generated, needs to be corrected.

    [1] JT Kajiya and TL Kay, “RENDERING FUR WITIt THREE DIMENSIONAL TEXTURES,” 1989.

    [2] SR Marschner, HW Jensen, and M. Cammarano, “Light Scattering from Human Hair Fibers,” 2003.

    [3] A. Zinke and A. Weber, “Light Scattering from Filaments,” IEEE Trans. Visual. Comput. Graphics, vol. 13, no. 2, pp. 342–356, Mar. 2007.

    [4] L.-Q. Yan, C.-W. Tseng, HW Jensen, and R. Ramamoorthi, “Physically-accurate fur reflectance: modeling, measurement and rendering,” ACM Trans. Graph., vol. 34, no. 6, pp. 1–13, Nov. 2015.

    [5] L.-Q. Yan, HW Jensen, and R. Ramamoorthi, “An efficient and practical near and far field fur reflectance model,” ACM Trans. Graph., vol. 36, no. 4, pp. 1–13, Aug. 2017.

    [6] M. Xia, B. Walter, C. Hery, O. Maury, E. Michielssen, and S. Marschner, “A Practical Wave Optics Reflection Model for Hair and Fur,” ACM Trans. Graph., vol. 42, no. 4, pp. 1–15, Aug. 2023.

    Glints Effect Study

    Traditional rendering methods based on geometric optics, such as Yan [2014, 2016], useBidirectional Reflectance Distribution Function (BRDF)To simulate the mirror reflection surface, there are certain limitations.

    Yan [2014, 2016] pointed out that traditional BRDF models usually use a smooth normal distribution function (NDF), assuming that the microfacets are infinitely small. But in reality, real surfaces often have obvious geometric features, such as micron-level bumps and flakes in metallic paint, which can cause significant glints under strong directional light sources (such as sunlight). Yan et al. simulated these small-scale surface geometric features more accurately through high-resolution normal maps, and proposed a new method to effectively render these complex specular highlights.

    Traditional uniform pixel sampling techniques have too large variance when capturing highlights in these small ranges, resulting in low rendering efficiency and inability to effectively handle the uneven distribution of highlights caused by the complexity of the light path. Therefore, Yan [2014, 2016] introduced a search based on normal distribution and targeted sampling.

    In hair renderings, you can observe that the hair and fur will show a shimmering effect of changing color when illuminated by strong directional light sources.

    XIA[2023] uses optical speckle theory to simulate highlight noise, and adds the diffraction lobe of basic diffraction theory to process the diffraction effect of light on the surface of fiber structures such as hair, thereby rendering colored highlight effects.

    XIA[2023], Chapter 8, states that glints can be easily observed in sunlight. Although subtle when viewed from a distance, these color effects can significantly enhance the appearance of the hair when viewed up close, sometimes causing a slight change in the hue of the fiber.

    In Figure 9, the model of XIA[2023] also produces colored shimmer effects on light-colored hair. The shimmer is more subtle on light-colored hair than on dark-colored fibers because multiple scattering averages out the colors, resulting in reduced color contrast. Compared to XIA[2020], XIA[2023] not only handles wavelength-dependent reflections better, but also improves its ability to handle the angle of the hair cuticle, capturing the shift in highlights caused by the tilt of the hair cuticle.

    References

    [1] L.-Q. Yan, M. Hašan, W. Jakob, J. Lawrence, S. Marschner, and R. Ramamoorthi, “Rendering glints on high-resolution normal-mapped specular surfaces,” ACM Trans. Graph., vol. 33, no. 4, pp. 1–9, Jul. 2014.

    [2] L.-Q. Yan, M. Hašan, S. Marschner, and R. Ramamoorthi, “Position-normal distributions for efficient rendering of specular microstructure,” ACM Trans. Graph., vol. 35, no. 4, pp. 1–9, Jul. 2016.

    Full Wave Reference Simulator

    https://dl.acm.org/doi/10.1145/3592414

    1. Introduction

    This paper discusses the theoretical basis of the physical wave simulation three-dimensional wave optical fiber scattering simulator used to generate high-precision light scattering simulation data in the rendering black dog hair paper "A Practical Wave Optics Reflection Model for Hair and Fur".

    Calculating light reflection from rough surfaces is an important topic. Small-scale geometric structures, such as the tiny features of hair fibers, have a significant impact on the reflection behavior of light. The BRDF describes how a surface reflects light given an incident and outgoing direction. The limitations of geometric optics have been repeated many times. This model that treats light as a straight line propagation fails to capture the wave nature of light when the microstructure is close to the wavelength of light.

    Theoretical models that use wave optics to approximate diffraction include the Beckmann-Kirchhoff theory of [Beckmann and Spizzichino 1987] and the Harvey-Shack model of [Krywonos 2006]. The former describes the light reflection behavior of rough surfaces, while the latter is a series of models based on wave optics that more accurately describe the scattering behavior of light on complex surfaces.

    Existing models are all aimed at the average reflection behavior of large-area surfaces, ignoring local detail changes. Yan [2016, 2018] is able to capture the changes in light reflection from microstructures in different regions of space. Even models based on electromagnetic wave propagation still require certain approximate processing due to computational complexity. These methods are not actually ground truth.

    In order to accurately capture the interference effects, XIA[2023] aims to develop a reference simulation tool that simulates the propagation of light faithfully according to Maxwell's equations. The only approximation is the numerical discretization, which ultimately generates the traditional bidirectional reflectance distribution function (BRDF) as output.This simulator truly achieves ground truth.

    That is to say, this simulator can accurately simulate the wave characteristics of light, including interference, diffraction, multiple scattering, etc. The approximations used in the simulator are only meshing and numerical integration errors.

    Through high-precision full-wave simulation, it is possible to generateHigh angular and spatial resolution BRDF data.

    At the same time, the simulator is able to handle large surface areas (such as 60 × 60 × 10 wavelengths). For example, using visible light with a wavelength of about 500 nanometers, 60 wavelengths is equivalent to 30 microns. In other words, the simulator's calculations are based on the scale of light wavelengths.Real physical sizeIn this case, light of different wavelengths will correspond to different numbers of discretized units.The larger the wavelength(For example, the wavelength of red light is longer than that of blue light). For the same physical size, the required discretization units (such as mesh division) will beRelatively less, so the amount of computation required will beRelatively smaller, processing speed may alsoFaster.

    Specifically, the surface is represented as a height field, each grid point corresponds to a height value, and quadrilaterals are used as primitives.

    For the scattered field, the boundary integral formulation is used to transform the scattering problem of electromagnetic waves into an integral equation that is solved only on the surface boundary. The key implementation method is the boundary element method (BEM). The adaptive integral method (AIM) based on the three-dimensional fast Fourier transform (3D FFT) is then used to accelerate the calculation process of the boundary integral.

    And use GPU to accelerate the parallel processing of large-scale surface scattering problems.

    And the paper uses a combined small-scale simulation results to characterize the surface bidirectional scattering behavior.

    Related Work

    Reflection model based on wave optics

    The old-fashioned geometric optics vs. wave optics. This article mainly compares surface scattering models. Classical models of geometric optics include: Cook-Torrance model [Cook and Torrance 1982], Oren-Nayar model [Michael 1994]. In wave optics, physical optics approximations are mainly used to simplify the full wave equation. That is, the first-order approximation (single scattering) in the black dog is used to estimate surface reflection. Classical models includeBeckmann-Kirchoff theoryandHarvey-Shack Model, which use approximate equations in scalar form to model wave optics effects. They are widely used to estimate reflectance on various surface types, such asGaussian random surface,Periodic surfaceEtc. However, the calculation results of these methods are often spatial average results, and it is impossible to perform high-resolution detail reflection.

    • Gaussian random surface models of He et al. (1991) and Lanari et al. (2017).
    • Periodic surface models by Dhillon et al. (2014), Stam (1999), and Toisoul and Ghosh (2017).
    • Multilayer planar surface model of Levin et al. (2013).
    • Surface data table model of Dong et al. (2016).
    • Study of scratched surfaces by Werner et al. (2017).

    In addition, physical optics approximation is also used to estimate theSpace changing appearance,For example:

    • Surface data table from Yan et al. (2018)
    • Random surface models of Steinberg and Yan (2022).

    Some hybrid surface models apply physical optics models to some surface components (such as roughness at small scales), while using geometric optics models for larger scales. Applications of these hybrid models include:

    • Surface roughness models by Falster et al. (2020) and Holzschuch and Pacanowski (2017).
    • Thin-film interference model of Belcour and Barla (2017).
    • Suspended particle model by Guillén et al. (2020).

    In addition, physical optics models are used to handle inter-surface effects at longer distances. For example:

    • Studies by Cuypers et al. (2012) and Steinberg et al. (2022) explored these long-range effects.

    Scattering methods based on wave optics, such asLorenz-Mie theoryandT-Matrix Method, which is also usedVolumetric ScatteringCalculations, for example:

    • Theories of Bohren and Huffman (2008) and Mishchenko et al. (2002).
    • Application of volume scattering by Frisvad et al. (2007) and Guo et al. (2021).

    In addition, complex-valued ray tracing techniques proposed by Sadeghi et al. (2012) and Shimada and Kawaguchi (2005) have been applied to rendering natural phenomena and structural color effects.Long-range effects between surfacesandVolumetric ScatteringSuch issues are currently beyond the scope of this study.

    Numerical Methods in Computational Electromagnetism (CEM)

    There are many methods for numerical calculations:

    • Oskooi et al. (2010) proposed a numerical method based on difference solution of Maxwell's equationsFinite Difference Time Domain (FDTD)FDTD has been used to predict the appearance of wavelength-scale structures (e.g. Auzinger et al. (2018), Musbach et al. (2013)). However, the overhead is considerable as the simulation area increases!
    • Finite Element Method (FEM)It is a widely used numerical method for solving partial differential equations, and can also be used in electromagnetic problems. It solves the problem by discretizing the simulation domain in three dimensions. Similar to FDTD, the amount of calculation is too large, not to mention for real-time rendering.
    • Gibson (2021) provides a detailedBoundary Element Method (BEM)The main advantage of BEM is that it reduces the dimensionality of discretization by converting the scattering problem into an integral equation on the surface of the object. In FDTD and FEM, the entire three-dimensional space needs to be discretized, while BEM only needs to discretize the surface of the object, which significantly reduces the dimensionality and complexity of the calculation.

    The paper chose BEMThe main reason is its scalability, which is conducive to the processing of complex surface structures.

    There are many ways to speed up BEM:

    • Liu and Nishimura (2006), White and Head-Gordon (1994)Fast Multipole Method (FMM).
    • Bleszynski et al. (1996) proposed a three-dimensional fast Fourier transform (3D FFT)Adaptive Integration Method (AIM).
    • Liao et al. (2016), Pak et al. (1997)Sparse Matrix Canonical Grid Method (SMCG).

    Thesis selected AIMThe reason is that AIM is suitable for processing an area with a relatively small axial size.

    result

    BRDF value isHemisphereThe standardSpectral data to XYZ to RGB conversion, a colored BRDF map is generated.

    That is, as the height field resolution increases, the BRDF output gradually stabilizes.8 samples per wavelengthThe resolution is sufficient to produce accurate results.

    By comparing with existing wave optics models,The simulator in this articleIt has the highest accuracy and can handle complex optical phenomena and geometric structures. It is suitable for scenes with high precision requirements and is suitable for multiple reflections, interference, and complex surfaces. However, the computational cost is high, which is a compromise between efficiency and accuracy.

    • OHS and GHS ModelsThe calculation is simple and suitable for smooth surfaces and medium roughness surfaces, but the error is large at large incident angles and on complex surfaces. GHS has improved accuracy at large angles compared to OHS.
    • Kirchhoff modelThe accuracy is relatively high, but it can only be maintained within the first order range.
    • Cutting Plane MethodComputationally efficient, suitable for relatively simple surface geometries. Not so for complex ones.
    • The accuracy of this article is the best and is suitable for high-precision scenarios.

    Comparison of coherent regions. As the illumination coherence increases, the resolution and detail of the BRDF becomes richer. When the coherence is high, the BRDF contains more high-resolution details.

    In addition, the paper also shows the method used to accelerate BRDF calculation.Beam steeringAs shown in Figure 14, the surface isSpecular Reflection, while in the other directionRetroreflective effectThe paper calculates the BRDF values for a series of gradually changing incident angles, as shown in the figure below.

    The BRDF image in each incident direction is reduced to a thin line segment. The comparison in Figure 15 shows that Tangent Plane cannot accurately model the surfaceSecond-order reflection(That is, light that is emitted after multiple reflections.) However, if the surface is smooth, using Tangent Plane is still very fast and accurate.

    Furthermore, the paper compares the simulator results with BRDF measurements of real surfaces, especially inMultiple reflection effectAs shown in Figure 17, the top is the actual measurement, the middle is the theoretical model of the paper, and the bottom is the Tangent Plane.

    You may ask why there is such a big difference? The paper uses an idealized geometric model, while the surface in the experiment may have some slight geometric deviations, which may affect the accuracy of the reflection. In a nutshell, the simulator in the paper can describe higher-order reflections!

    Future Work

    When dealing with more complexStructured surfaceWhen the light is scattered on the surface, the simulator can more accurately simulate the propagation and scattering behavior of light on the surface.

    The future direction of work is of course to reduce computational overhead while ensuring accuracy.

    Then the processing area of this BRDF approximation model is expanded.

    At the same time, the simulator in this paper can be used as a benchmark reference.

    References

    A Full-Wave Reference Simulator for Computing Surface Reflectance

    Petr Beckmann and Andre Spizzichino. 1987. The scattering of electromagnetic waves from rough surfaces. Artech House.

    Andrey Krywonos. 2006. Predicting Surface Scatter using a Linear Systems Formulation of Non-Paraxial Scalar Diffraction. Ph. D. Dissertation. University of Central Florida.

    Ling-Qi Yan, Miloš Hašan, Bruce Walter, Steve Marschner, and Ravi Ramamoorthi. 2018. Rendering Specular Microgeometry with Wave Optics. ACM Trans. Graph. 37, 4 (2018).

    Ling-Qi Yan, Miloš Hašan, Steve Marschner, and Ravi Ramamoorthi. 2016. Positionnormal distributions for efficient rendering of specular microstructure. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1–9.

    RL Cook and KE Torrance. 1982. A Reflectance Model for Computer Graphics. ACM Trans. Graph. 1, 1 (jan 1982). https://doi.org/10.1145/357290.357293

    Michael Oren and Shree K. Nayar. 1994. Generalization of Lambert's Reflectance Model (SIGGRAPH '94). https://doi.org/10.1145/192161.192213

  • Games202 作业三 SSR实现

    Games202 Assignment 3 SSR Implementation

    Assignment source code:

    https://github.com/Remyuu/GAMES202-Homeworkgithub.com/Remyuu/GAMES202-Homework

    TODO List

    • Implements shading of the scene's direct lighting (taking shadows into account).
    • Implements screen space ray intersection (SSR).
    • Implements shading of indirect lighting of the scene.
    • Implement RayMarch with dynamic step size.
    • (Not written yet) Bonus 1: Screen Space Ray Tracing with Mipmap Optimization.
    img

    Number of samples: 32

    Written in front

    The basic part of this assignment is the easiest among all the assignments in 202. There is nothing particularly complicated. But I don't know how to start with the bonus part. Can someone please help me?

    Depth buffer problem of framework

    This time, the operation encountered a more serious problem on macOS. The part of the cube close to the ground showed abnormal cutting jagged problems as the distance of the camera changed. This phenomenon did not occur on Windows, which was quite strange.

    img

    I personally feel that this is related to the accuracy of the depth buffer, and may be caused by z-fighting, in which two or more overlapping surfaces compete for the same pixel. There are generally several solutions to this problem:

    • Adjust the near and far planes: don't make the near plane too close to the camera, and don't make the far plane too far away.
    • Improve the precision of the depth buffer: use 32-bit or higher precision.
    • Multi-Pass Rendering: Use different rendering schemes for objects in different distance ranges.

    The simplest solution is to modify the size of the near plane, located in line 25 of the framework's engine.js.

    // engine.js // const camera = new THREE.PerspectiveCamera(75, gl.canvas.clientWidth / gl.canvas.clientHeight, 0.0001, 1e5); const camera = new THREE.PerspectiveCamera(75, gl.canvas.clientWidth / gl.canvas.clientHeight, 5e-2, 1e2);

    This will give you a pretty sharp border.

    img

    Added "Pause Rendering" function

    This section is optional. To reduce the strain on your computer, simply write a button to pause the rendering.

    // engine.js let settings = { 'Render Switch': true }; function createGUI() { ... // Add the boolean switch here gui.add(settings, 'Render Switch'); ... } function mainLoop (now) { if(settings['Render Switch']){ cameraControls.update(); renderer.render(); } requestAnimationFrame(mainLoop); } requestAnimationFrame(mainLoop);
    img

    image-20231117191114477

    1. Implementing direct lighting

    Implement EvalDiffuse(vec3 wi, vec3 wo, vec2 uv) and EvalDirectionalLight(vec2 uv) in shaders/ssrShader/ssrFragment.glsl.

    // ssrFragment.glsl vec3 EvalDiffuse(vec3 wi, vec3 wo, vec2 screenUV) { vec3 reflectivity = GetGBufferDiffuse(screenUV); vec3 normal = GetGBufferNormalWorld(screenUV); float cosi = max(0., dot(normal, wi)); vec3 f_r = reflectivity * cosi; return f_r; } vec3 EvalDirectionalLight(vec2 screenUV) { vec3 Li = uLightRadiance * GetGBufferuShadow(screenUV); return Li; }

    The first code snippet actually implements the Lambertian reflection model, which corresponds to $f_r \cdot \text{cos}(\theta_i)$ in the rendering equation.

    Here I divide $\pi$, but according to the results given in the assignment framework, there should be no division, so just take it as it is here.

    The second part is responsible for direct lighting (including shadow occlusion), relative to the $L_i \cdot V$ of the rendering equation.

    Lo(p,ωo)=Le(p,ωo)+∫ΩLi(p,ωi)⋅fr(p,ωi,ωo)⋅V(p,ωi)⋅cos⁡(θi)dωi

    Let's review the Lambertian reflection model here. We noticed that EvalDiffuse passed in two directions, wi and wo, but we only used the direction of the incident light, wi. This is because the Lambertian model has nothing to do with the direction of observation, but only with the surface normal and the cosine value of the incident light.

    Finally, set the result in main().

    // ssrFragment.glsl void main() { float s = InitRand(gl_FragCoord.xy); vec3 L = vec3(0.0); vec3 wi = normalize(uLightDir); vec3 wo = normalize(uCameraPos - vPosWorld.xyz); vec2 worldPos = GetScreenCoordinate(vPosWorld.xyz); L = EvalDiffuse(wi, wo, worldPos) * EvalDirectionalLight(worldPos); vec3 color = pow(clamp(L, vec3(0.0), vec3(1.0)), vec3(1.0 / 2.2)) ; gl_FragColor = vec4(vec3(color.rgb), 1.0); }
    img

    2. Specular SSR – Implementing RayMarch

    Implement the RayMarch(ori, dir, out hitPos) function to find the intersection point between the ray and the object and return whether the ray intersects the object. The parameters ori and dir are values in the world coordinate system, representing the starting point and direction of the ray respectively, where the direction vector is a unit vector. For more information, please refer to EA's SIG15Course Report.

    The "cube1" of the work frame itself includes the ground, so the final SSR effect of this thing is not very beautiful. The "beautiful" here refers to the clarity of the result map in the paper or the exquisiteness of the water reflection effect in the game.

    To be precise, what we implement in this article is the most basic "mirror SSR", namely Basic mirror-only SSR.

    img

    The easiest way to implement "mirror SSR" is to use Linear Raymarch, which gradually determines the occlusion relationship between the current position and the depth position of gBuffer through small steps.

    img
    // ssrFragment.glsl bool RayMarch(vec3 ori, vec3 dir, out vec3 hitPos) { const int totalStepTimes = 60; const float threshold = 0.0001; float step = 0.05; vec3 stepDir = normalize(dir) * step; vec3 curPos = ori ; for(int i = 0; i < totalStepTimes; i++) { vec2 screenUV = GetScreenCoordinate(curPos); float rayDepth = GetDepth(curPos); float gBufferDepth = GetGBufferDepth(screenUV); // Check if the ray has hit an object if(rayDepth > gBufferDepth + threshold){ hitPos = curPos; return true; } curPos += stepDir; } return false; }

    Finally, fine-tune the step size. I ended up with 0.05. If the step size is too large, the reflection will be "broken". If the step size is too small and the number of steps is not enough, the calculation may be terminated because the step distance is not enough where the reflection should be. The maximum number of steps in the figure below is 150.

    img
    // ssrFragment.glsl vec3 EvalSSR(vec3 wi, vec3 wo, vec2 screenUV) { vec3 worldNormal = GetGBufferNormalWorld(screenUV); vec3 relfectDir = normalize(reflect(-wo, worldNormal)); vec3 hitPos; if(RayMarch(vPosWorld.xyz ,relfectDir, hitPos)){ vec2 INV_screenUV = GetScreenCoordinate(hitPos); return GetGBufferDiffuse(INV_screenUV); } else{ return vec3(0.); } }

    Write a function that calls RayMarch and wraps it up so it can be used in main().

    // ssrFragment.glsl void main() { float s = InitRand(gl_FragCoord.xy); vec3 L = vec3(0.0); vec3 wi = normalize(uLightDir); vec3 wo = normalize(uCameraPos - vPosWorld.xyz); vec2 screenUV = GetScreenCoordinate(vPosWorld.xyz); // Basic mirror-only SSR float reflectivity = 0.2; L = EvalDiffuse(wi, wo, screenUV) * EvalDirectionalLight(screenUV); L+= EvalSSR(wi, wo, screenUV) * reflectivity; vec3 color = pow(clamp(L, vec3(0.0), vec3(1.0)), vec3(1.0 / 2.2)); gl_FragColor = vec4(vec3(color.rgb), 1.0); }

    If you just want to test the effect of SSR, please adjust it yourself in main().

    img
    img

    Before the release of "Killzone Shadow Fall" in 2013, SSR technology was still subject to great restrictions, because in actual development, we usually need to simulate glossy objects. Due to the performance limitations at the time, SSR technology was not widely adopted. With the release of "Killzone Shadow Fall", it marks a significant progress in real-time reflection technology. Thanks to the special hardware of PS4, it is possible to render high-quality glossy and semi-reflective objects in real time.

    img

    In the following years, SSR technology developed rapidly, especially in combination with technologies such as PBR.

    Starting with Nvidia's RTX graphics cards, the rise of real-time ray tracing has gradually replaced SSR in some scenarios. However, in most development scenarios, traditional SSR still plays a considerable role.

    The future development trend will still be a mixture of traditional SSR technology and ray tracing technology.

    3. Indirect lighting

    Write it according to the pseudocode. That is, use the Monte Carlo method to solve the rendering equation. Unlike before, the samples this time are all in screen space. In the sampling process, you can use the SampleHemisphereUniform(inout s, ou pdf) and SampleHemisphereCos(inout s, out pdf) provided by the framework. These two functions return local coordinates, and the input parameters are the random number s and the sampling probability pdf.

    For this part, you need to understand the pseudo code in the figure below, and then complete EvalIndirectionLight() accordingly.

    img

    First of all, we need to know that our sampling is still based on screen space. Therefore, we treat the content that is not on the screen (gBuffer) as non-existent. It is understood that there is only one layer of shell facing the camera.

    Indirect lighting involves random sampling of the upper hemisphere direction and the calculation of the corresponding PDF. Use InitRand(screenUV) to get the random number, then choose one of the two, SampleHemisphereUniform(inout float s, out float pdf) or SampleHemisphereCos(inout float s, out float pdf), update the random number and get the corresponding PDF and the position dir of the local coordinate system on the unit hemisphere.

    Pass the normal coordinates of the current Shading Point into the function LocalBasis(n, out b1, out b2), and then return b1, b2, where the three unit vectors n, b1, b2 are orthogonal to each other. Through the local coordinate system formed by these three vectors, dir is converted to world coordinates. I will write about the principle of LocalBasis() at the end.

    By the way, the matrix constructed with the vectors n (normal), b1, and b2 is commonly referred to as the TBN matrix in computer graphics.

    // ssrFragment.glsl #define SAMPLE_NUM 5 vec3 EvalIndirectionLight(vec3 wi, vec3 wo, vec2 screenUV){ vec3 L_ind = vec3(0.0); float s = InitRand(screenUV); vec3 normal = GetGBufferNormalWorld(screenUV); vec3 b1, b2; LocalBasis(normal, b1, b2); for(int i = 0; i < SAMPLE_NUM; i++){ float pdf; vec3 direction = SampleHemisphereUniform(s, pdf); vec3 worldDir = normalize(mat3(b1, b2, normal) * direction); vec3 position_1; if(RayMarch(vPosWorld.xyz, worldDir, position_1)){ // The sampling ray hits position_1 vec2 hitScreenUV = GetScreenCoordinate(position_1); vec3 bsdf_d = EvalDiffuse(worldDir, wo, screenUV); // Direct lighting vec3 bsdf_i = EvalDiffuse(wi, worldDir, hitScreenUV); // Indirect lighting L_ind += bsdf_d / pdf * bsdf_i * EvalDirectionalLight(hitScreenUV); } } L_ind /= float(SAMPLE_NUM); return L_ind; } // ssrFragment.glsl // Main entry point for the shader void main() { vec3 wi = normalize(uLightDir); vec3 wo = normalize( uCameraPos - vPosWorld.xyz); vec2 screenUV = GetScreenCoordinate(vPosWorld.xyz); // Basic mirror-only SSR coefficient float ssrCoeff = 0.0; // Indirection Light coefficient float indCoeff = 0.3; // Direction Light vec3 L_d = EvalDiffuse(wi, wo, screenUV) * EvalDirectionalLight(screenUV); // SSR Light vec3 L_ssr = EvalSSR(wi, wo, screenUV) * ssrCoeff; // Indirection Light vec3 L_i = EvalIndirectionLight(wi, wo, screenUV) * IndCorff; vec3 result = L_d + L_ssr + L_i; vec3 color = pow(clamp(result, vec3(0.0), vec3(1.0)), vec3(1.0 / 2.2)); gl_FragColor = vec4(vec3(color.rgb), 1.0); }

    Show only indirect lighting. Samples = 5.

    img

    Direct lighting + indirect lighting. Number of samples = 5.

    img

    It was such a headache to write this part. Even with SAMPLE_NUM set to 1, my computer was sweating profusely. Once the Live Server was turned on, there was a delay when typing directly. I couldn't stand it. Is this the performance of the M1pro? And what I can't stand the most is that the Safari browser is stuck, why is the whole system stuck? Is this your User First strategy of macOS? I don't understand. I had no choice but to take out my gaming computer to pass the LAN test project (sad). I just didn't expect that the RTX3070 would also sweat profusely when running.It seems that the algorithm I wrote is a pile of shit, and my life is also a pile of shit..

    4. RayMarch Improvements

    The current RayMarch() is actually problematic and will cause light leakage.

    img

    When the sampling number is 5, it is only about 46.2 frames. My device is M1pro 16GB.

    img

    Here we will focus on why light leakage occurs. See the figure below. Our gBuffer only has the depth information of the blue part. Even if our algorithm above has determined that the current curPos is deeper than the depth of gBuffer, it cannot ensure that this curPos is the collision point. Therefore, the algorithm above does not consider the situation in the figure, which leads to light leakage.

    img

    forSolve the light leakage problemWe introduce a threshold to solve this problem (yes, it is an approximation). If the difference between curPos and the depth recorded by the current gBuffer is greater than a certain threshold, the situation shown in the figure below will occur. At this time, the information in the screen space cannot correctly provide the reflection information, so the SSR result of this Shading Point is vec3(0). It is so simple and crude!

    img

    The idea of the code is similar to the previous one. At each step, the relationship between the depth of the next step position and the depth of gBuffer is determined. If the next step position is in front of gBuffer (nextDepth

    bool RayMarch(vec3 ori, vec3 dir, out vec3 hitPos) { const float EPS = 1e-2; const int totalStepTimes = 60; const float threshold = 0.1; float step = 0.05; vec3 stepDir = normalize(dir) * step; vec3 curPos = ori + stepDir; vec3 nextPos = curPos + stepDir; for(int i = 0; i < totalStepTimes; i++) { if(GetDepth(nextPos) < GetGBufferDepth(GetScreenCoordinate(nextPos))){ curPos = nextPos; nextPos += stepDir; }else if(GetGBufferDepth(GetScreenCoordinate(curPos )) - GetDepth(curPos) + EPS > threshold){ return false; }else{ curPos += stepDir; vec2 screenUV = GetScreenCoordinate(curPos); float rayDepth = GetDepth(curPos); float gBufferDepth = GetGBufferDepth(screenUV); if(rayDepth > gBufferDepth + threshold){ hitPos = curPos; return true; } } } return false; }

    The frame rate dropped to around 42.6, but the picture was significantly improved! At least there was no noticeable light leakage.

    img

    However, there are still some flaws in the picture, that is, there will be hairy reflection patterns at the edges, which means that the light leakage problem is still not solved, as shown in the following figure:

    img

    The above methodThere is indeed a problemWhen comparing with the threshold, we mistakenly used curPos for comparison (i.e., Step n in the figure below), which caused the code to enter the third branch and return the hitPos of the wrong curPos.

    img

    Taking a step back, we have no way to guarantee that the final calculated curPos falls exactly on the line between the edge of the object and the origin of the camera. To put it bluntly, the blue line in the figure below is quite discrete. We want to get the curPos that is "just" at the boundary, and then deal with the defects in the distance from "Step n" to "the "just" curPos" (that is, the burr error above), but obviously due to various precision reasons, we can't get it. In the figure below, the green line represents a step.

    img

    Even if we adjust the ratio of threshold/step to make it close to 1, we can hardly eliminate the problem and can only alleviate it, as shown in the figure below.

    img

    Therefore, we need to improve the "anti-light leakage" method again.

    In other words, the idea of improvement is very simple. Since I can't get the "exact" curPos point, I will guess it. Specifically, I will do a linear interpolation directly. Before interpolation, I will make an approximation, that is, I will regard the sight lines as parallel to each other, and then make a similar triangle as shown in the figure below, guess the curPos we want, and then use it as hitPos.

    img

    hitPos=curPos+s1s1+s2

    bool RayMarch(vec3 ori, vec3 dir, out vec3 hitPos) { bool result = false; const float EPS = 1e-3; const int totalStepTimes = 60; const float threshold = 0.1; float step = 0.05; vec3 stepDir = normalize(dir ) * step; vec3 curPos = ori + stepDir; vec3 nextPos = curPos + stepDir; for(int i = 0; i < totalStepTimes; i++) { if(GetDepth(nextPos) < GetGBufferDepth(GetScreenCoordinate(nextPos))){ curPos = nextPos; nextPos += stepDir; continue; } float s1 = GetGBufferDepth(GetScreenCoordinate(curPos)) - GetDepth(curPos) + EPS; float s2 = GetDepth(nextPos) - GetGBufferDepth(GetScreenCoordinate(nextPos)) + EPS; if(s1 < threshold && s2 < threshold){ hitPos = curPos + stepDir * s1 / (s1 + s2); result = true; } break; } return result ; }

    The effect is quite good, with no ghosting or border artifacts. And the frame rate is similar to the original algorithm, averaging around 49.2.

    img

    Next, we will focus on optimizing performance, specifically:

    • Add adaptive step
    • Off-screen ignored judgment

    Off-screen ignored judgment Very simple. If the uvScreen of curPos is not between 0 and 1, then the current step is abandoned.

    Let's talk about the adaptive step in detail. That is, add two lines at the beginning of for. The actual frame rate will increase slightly by about 2-3 frames.

    vec2 uvScreen = GetScreenCoordinate(curPos); if(any(bvec4(lessThan(uvScreen, vec2(0.0)), greaterThan(uvScreen, vec2(1.0))))) break;

    Adaptive step It is not difficult. First, set a larger value for the initial step. IfAfter steppingcurPos Not on screen or The depth value is deeper than gBuffer or "s1 < threshold && s2 < threshold" is not satisfied , then let the step be halved to ensure accuracy.

    bool RayMarch(vec3 ori, vec3 dir, out vec3 hitPos) { const float EPS = 1e-2; const int totalStepTimes = 20; const float threshold = 0.1; bool result = false, firstIn = false; float step = 0.8; vec3 curPos = ori; vec3 nextPos; for(int i = 0; i < totalStepTimes; i++) { nextPos = curPos+dir*step; vec2 uvScreen = GetScreenCoordinate(curPos); if(any(bvec4(lessThan(uvScreen, vec2(0.0))), greaterThan(uvScreen, vec2(1.0))))) break; if (GetDepth(nextPos) < GetGBufferDepth(GetScreenCoordinate(nextPos))){ curPos += dir * step; if(firstIn) step *= 0.5; continue; } firstIn = true; if(step < EPS){ float s1 = GetGBufferDepth(GetScreenCoordinate(curPos)) - GetDepth(curPos) + EPS; float s2 = GetDepth(nextPos) - GetGBufferDepth(GetScreenCoordinate(nextPos)) + EPS; if(s1 < threshold && s2 < threshold){ hitPos = curPos + 2.0 * dir * step * s1 / (s1 + s2); result = true; } break; } if(firstIn) step *= 0.5; } return result; }

    After the improvement, the frame rate suddenly reached 100 frames, almost doubling.

    img

    Finally, tidy up the code.

    #define EPS 5e-2 #define TOTAL_STEP_TIMES 20 #define THRESHOLD 0.1 #define INIT_STEP 0.8 bool outScreen(vec3 curPos){ vec2 uvScreen = GetScreenCoordinate(curPos); return any(bvec4(lessThan(uvScreen, vec2(0.0)), greaterThan(uvScreen, vec2(1.0)))); } bool testDepth(vec3 nextPos){ return GetDepth(nextPos) < GetGBufferDepth(GetScreenCoordinate(nextPos)); } bool RayMarch(vec3 ori, vec3 dir, out vec3 hitPos) { float step = INIT_STEP; bool result = false, firstIn = false; vec3 nextPos, curPos = ori; for(int i = 0; i < TOTAL_STEP_TIMES; i++) { nextPos = curPos + dir * step; if(outScreen(curPos)) break; if(testDepth(nextPos)){ // You can improve curPos += dir * step; continue; }else{ // Too advanced firstIn = true; if(step < EPS){ float s1 = GetGBufferDepth(GetScreenCoordinate(curPos)) - GetDepth(curPos) + EPS; float s2 = GetDepth(nextPos) - GetGBufferDepth(GetScreenCoordinate(nextPos)) + EPS; if(s1 < THRESHOLD && s2 < THRESHOLD){ hitPos = curPos + 2.0 * dir * step * s1 / (s1 + s2); result = true; } break; } if(firstIn) step *= 0.5; } } return result; }

    Switching to the cave scene, the sampling rate is set to 32, and the frame rate is only a pitiful 4 frames.

    img

    And the quality of the secondary light source is very good.

    img

    However, this algorithm will cause new problems when applied to reflections, especially the following picture, which has serious distortion.

    img
    img

    5. Mipmap Implementation

    Hierarchical-Z map based occlusion culling

    6. LocalBasis builds TBN principle

    Generally speaking, constructing the normal tangent vector (normal, tangent, and bitangent vector) is achieved through the cross product. The implementation method is very simple. First, select an auxiliary vector that is not parallel to the normal vector, and do a cross product between the two to get the first tangent vector. Then, do a cross product between the tangent vector and the normal vector to get the bitangent vector. The specific code is written as follows:

    void CalculateTBN(const vec3 &normal, vec3 &tangent, vec3 &bitangent) { vec3 helperVec; if (abs(normal.x) < abs(normal.y)) helperVec = vec3(1.0, 0.0, 0.0); else helperVec = vec3(0.0 , 1.0, 0.0); tangent = normalize(cross(helperVec, normal)); bitangent = normalize(cross(normal, tangent)); }

    But the code in the job framework avoids usingCross Product, which is very clever. Simply put, it is to ensure that the vectorDot ProductAll are 0.

    • $b1⋅n=0$
    • $b2⋅n=0$
    • $b1⋅b2=0$
    void LocalBasis(vec3 n, out vec3 b1, out vec3 b2) { float sign_ = sign(nz); if (nz == 0.0) { sign_ = 1.0; } float a = -1.0 / (sign_ + nz); float b = nx * ny * a; b1 = vec3(1.0 + sign_ * nx * nx * a, sign_ * b, -sign_ * nx); b2 = vec3(b, sign_ + ny * ny * a, -ny); }

    This algorithm is a heuristic one, which introduces a symbolic function, which is quite impressive. It also considers the case of division by 0, and the pattern is also full. However, the following four lines should be the author's random disassembly when he wrote the formula one day. Here I will restore the author's disassembly steps at that time. That is, the process of reverse deduction.

    img

    By the way, the sign function in the code can be multiplied in the last step.

    In fact, I can create a hundred such formulas, and I don’t know the difference between them. If you know, please tell me QAQ. If you insist, then it can be explained like this:

    Traditional cross-product-based methods may be numerically unstable because the cross-product result is close to the zero vector in this case. The method adopted in this paper is a heuristic method that constructs an orthogonal basis through a series of carefully designed steps. This method pays special attention to numerical stability, making it effective and stable when dealing with normal vectors close to extreme directions.

    grateful @I am a dragon set little fruit As pointed out by , the above method is very particular. The algorithm provided in the homework framework was obtained by Tom Duff et al. in 2017 by improving Frisvad's method. For details, please refer to the following two papers.

    https://graphics.pixar.com/library/OrthonormalB/paper.pdfgraphics.pixar.com/library/OrthonormalB/paper.pdf

    https://backend.orbit.dtu.dk/ws/portalfiles/portal/126824972/onb_frisvad_jgt2012_v2.pdfbackend.orbit.dtu.dk/ws/portalfiles/portal/126824972/onb_frisvad_jgt2012_v2.pdf

    References

    1. Games 202
    2. LearnOpenGL – Normal Mapping
  • Games202 作业二 PRT实现

    Games202 Assignment 2 PRT Implementation

    img
    img
    img

    Because I am also a newbie, I can't ensure that everything is correct. I hope the experts can correct me.

    Zhihu's formula is a bit ugly, you can go to:GitHub

    Project source code:

    https://github.com/Remyuu/GAMES202-Homeworkgithub.com/Remyuu/GAMES202-Homework

    Precomputed spherical harmonic coefficients

    The spherical harmonics coefficients are pre-computed using the framework nori.

    Ambient lighting: Calculate the spherical harmonic coefficients for each pixel of the cubemap

    ProjEnv::PrecomputeCubemapSH(images, width, height, channel); Use the Riemann integral method to calculate the coefficients of the ambient light spherical harmonics.

    Complete code

    // TODO: here you need to compute light sh of each face of cubemap of each pixel
    // TODO: Here you need to calculate the spherical harmonic coefficients of a certain face of the cubemap for each pixel
    Eigen::Vector3f dir = cubemapDirs[i * width * height + y * width + x];
    int index = (y * width + x) * channel;
    Eigen::Array3f Le(images[i][index + 0], images[i][index + 1],
                      images[i][index + 2]);
    // Describe the current angle in spherical coordinates
    double theta = acos(dir.z());
    double phi = atan2(dir.y(), dir.x());
    // Traverse each basis function of spherical harmonics
    for (int l = 0; l <= SHOrder; l++){
        for (int m = -l; m <= l; m++){
            float sh = sh::EvalSH(l, m, phi, theta);
            float delta = CalcArea((float)x, (float)y, width, height);
            SHCoeffiecents[l*(l+1)+m] += Le * sh * delta;
        }
    }
    C#

    analyze

    Spherical harmonic coefficientsIt is the projection of the spherical harmonic function on a sphere, which can be used to represent the distribution of the function on the sphere. Since we have three channels of RGB values, the spherical harmonic coefficients we will store as a three-dimensional vector. Parts that need to be improved:

    /// prt.cpp - PrecomputeCubemapSH()
    // TODO: here you need to compute light sh of each face of cubemap of each pixel
    // TODO: Here you need to calculate the spherical harmonic coefficients of a certain face of the cubemap for each pixel
    Eigen::Vector3f dir = cubemapDirs[i * width * height + y * width + x];
    int index = (y * width + x) * channel;
    Eigen::Array3f Le(images[i][index + 0], images[i][index + 1],
                      images[i][index + 2]);
    C#

    First, we sample a direction (a 3D vector representing the direction from the center to the pixel) from each pixel of the six cubemaps (the images array) and convert the direction to spherical coordinates (theta and phi).

    Then, each spherical coordinate is passed into sh::EvalSH() to calculate the real value sh of each spherical harmonic function (basis function) and the proportion delta of the spherical area occupied by each pixel in each cubemap is calculated.

    Finally, we accumulate the spherical harmonic coefficients. In the code, we can accumulate all the pixels of the cubemap, which is similar to the original operation of calculating the integral of the spherical harmonic function.

    $$
    Ylm=∫ϕ=02π∫θ=0πf(θ,ϕ)Ylm(θ,ϕ)sin⁡(θ)dθdϕ
    $$

    in:

    • θ is the zenith angle, ranging from 0 to π; ϕ is the azimuth angle, ranging from 0 to 2pi.
    • f(θ,ϕ) is the value of the function at a point on the sphere.
    • Ylm is a spherical harmonic function, which consists of the corresponding Legendre polynomials Plm and some trigonometric functions.
    • l is the order of the spherical harmonics; m is the ordinal number of the spherical harmonics, ranging from −l to l.

    In order to make the readers understand more specifically, here is the estimate of the discrete form of the spherical harmonics in the code, that is, the Riemann integral method for calculation.

    $$
    Ylm=∑i=1Nf(θi,ϕi)Ylm(θi,ϕi)Δωi
    $$

    in:

    • f(θi,ϕi) is the value of the function at a point on the sphere.
    • Ylm(θi,ϕi) is the value of the spherical harmonics at that point.
    • Δωi is the tiny area or weight of the point on the sphere.
    • N is the total number of discrete points.

    Code Details

    • Get RGB lighting information from cubemap
    Eigen::Array3f Le(images[i][index + 0], images[i][index + 1],
                      images[i][index + 2]);
    C#

    The value of channel is 3, corresponding to the three channels of RGB. Therefore, index points to the position of the red channel of a pixel, index + 1 points to the position of the green channel, and index + 2 points to the position of the blue channel.

    • Convert direction vector to spherical coordinates
    double theta = acos(dir.z());
    double phi = atan2(dir.y(), dir.x());
    C#

    theta is the angle from the positive z-axis to the direction of dir, and phi is the angle from the positive x-axis to the projection of dir on the xz plane.

    • Traversing the basis functions of spherical harmonics
    for (int l = 0; l <= SHOrder; l++){
        for (int m = -l; m <= l; m++){
            float sh = sh::EvalSH(l, m, phi, theta);
            float delta = CalcArea((float)x, (float)y, width, height);
            SHCoeffiecents[l*(l+1)+m] += Le * sh * delta;
        }
    }
    C#

    Unshadowed diffuse term

    scene->getIntegrator()->preprocess(scene); calculation Diffuse Unshadowed Simplify the rendering equation and substitute the spherical harmonic function in the previous section to further calculate the coefficients of the spherical harmonic projection of the BRDF. The key function is ProjectFunction. We need to write a lambda expression for this function to calculate the transfer function term.

    analyze

    For the diffuse transmission term, we canThere are three situationsconsider:Shadowed,No shadowandMutually Reflective.

    Let's first consider the simplest case without shadows. We have the rendering equation

    in,

    • is the incident radiance.
    • It is a geometric function, and the microscopic properties of the surface are related to the direction of the incident light.
    • is the incident light direction.

    For a diffuse surface with equal reflection everywhere, we can simplify to Unshadowed Lighting equation

    in:

    • is the diffuse outgoing radiance of the point.
    • is the surface normal.

    The incident radiance and transfer function terms are independent of each other, as the former represents the contribution of the light sources in the scene, and the latter represents how the surface responds to the incident light. Therefore, these two components are treated independently.

    Specifically, when using spherical harmonics approximation, we expand these two items separately. The input of the former is the incident direction of light, and the input of the latter is the reflection (or outgoing direction), and the expansion is two series of arrays, so we use a data structure called Look-Up Table (LUT).

    auto shCoeff = sh::ProjectFunction(SHOrder, shFunc, m_SampleCount);
    C#

    Among them, the most important one is the function ProjectFunction above. We need to write a Lambda expression (shFunc) as a parameter for this function, which is used to calculate the transfer function term.

    ProjectFunction function parameter passing:

    • Spherical harmonic order
    • Functions that need to be projected onto basis functions (that we need to write)
    • Number of samples

    This function will take the result returned by the Lambda function and project it onto the basis function to get the coefficient. Finally, it will add up the coefficients of each sample and multiply them by the weight to get the final coefficient of the vertex.

    Complete code

    Compute the geometric terms, i.e. the transfer function terms.

    //prt.cpp
    ...
    double H = wi.normalized().dot(n.normalized()) / M_PI;
    if (m_Type == Type::Unshadowed){
        // TODO: here you need to calculate unshadowed transport term of a given direction
        // TODO: Here you need to calculate the unshadowed transmission term spherical harmonics value in a given direction
        return (H > 0.0) ? H : 0.0;
    }
    C#

    In short, remember to divide the final integral result by , and then pass it to m_TransportSHCoeffs.

    Shadowed Diffuse Term

    scene->getIntegrator()->preprocess(scene); calculation Diffuse Shadowed This item has an additional visible item.

    analyze

    The Visibility item () is a value that is either 1 or 0. The bool rayIntersect(const Ray3f &ray) function is used to reflect a ray from the vertex position to the sampling direction. If it hits the object, it is considered to be blocked and has a shadow, and 0 is returned; if the ray does not hit the object, it is still returned.

    Complete code

    //prt.cpp
    ...
    double H = wi.normalized().dot(n.normalized()) / M_PI;
    ...
    else{
        // TODO: here you need to calculate shadowed transport term of a given direction
        // TODO: Here you need to calculate the spherical harmonic value of the shadowed transmission term in a given direction
        if (H > 0.0 && !scene->rayIntersect(Ray3f(v, wi.normalized())))
            return H;
        return 0.0;
    }
    C#

    In short, remember to divide the final integral result by , and then pass it to m_TransportSHCoeffs.

    Export calculation results

    The nori framework will generate two pre-calculated result files.

    Add run parameters:

    ./scenes/prt.xml

    In prt.xml, you need to do the followingRevise, you can choose to render the ambient light cubemap. In addition, the model, camera parameters, etc. can also be modified by yourself.

    //prt.xml
    
    <!-- Render the visible surface normals -->
    <integrator type="prt">
        <string name="type" value="unshadowed" />
        <integer name="bounce" value="1" />
        <integer name="PRTSampleCount" value="100" />
    <!--        <string name="cubemap" value="cubemap/GraceCathedral" />-->
    <!--        <string name="cubemap" value="cubemap/Indoor" />-->
    <!--        <string name="cubemap" value="cubemap/Skybox" />-->
        <string name="cubemap" value="cubemap/CornellBox" />
    
    </integrator>
    C#

    Among them, the label optional value:

    • type: unshadowed, shadowed, interreflection
    • bounce: The number of light bounces under the interreflection type (not yet implemented)
    • PRTSampleCount: The number of samples per vertex of the transmission item
    • cubemap: cubemap/GraceCathedral, cubemap/Indoor, cubemap/Skybox, cubemap/CornellBox
    img

    The above pictures are the unshadowed rendering results of GraceCathedral, Indoor, Skybox and CornellBox, with a sampling number of 1.

    Coloring using spherical harmonics

    Manually drag the files generated by nori into the real-time rendering framework and make some changes to the real-time framework.

    After the calculation in the previous chapter is completed, copy the light.txt and transport.txt in the corresponding cubemap path to the cubemap folder of the real-time rendering framework.

    Precomputed data analysis

    Cancel The comments on lines 88-114 in engine.js are used to parse the txt file just added.

    // engine.js
    // file parsing
    ... // Uncomment this code
    C#

    Import model/create and use PRT material shader

    In the materials folderEstablishFile PRTMaterial.js.

    //PRTMaterial.js
    
    class PRTMaterial extends Material {
        constructor(vertexShader, fragmentShader) {
            super({
                'uPrecomputeL[0]': { type: 'precomputeL', value: null},
                'uPrecomputeL[1]': { type: 'precomputeL', value: null},
                'uPrecomputeL[2]': { type: 'precomputeL', value: null},
            }, 
            ['aPrecomputeLT'], 
            vertexShader, fragmentShader, null);
        }
    }
    
    async Function buildPRTMaterial(vertexPath, fragmentPath) {
        let vertexShader = await getShaderString(vertexPath);
        let fragmentShader = await getShaderString(fragmentPath);
    
        return new PRTMaterial(vertexShader, fragmentShader);
    }
    C#

    Then import it in index.html.

    // index.html
    <script src="src/materials/Material.js" defer></script>
    <script src="src/materials/ShadowMaterial.js" defer></script>
    <script src="src/materials/PhongMaterial.js" defer></script>
    <!-- Edit Start --><script src="src/materials/PRTMaterial.js" defer></script><!-- Edit End -->
    <script src="src/materials/SkyBoxMaterial.js" defer></script>
    C#

    在 loadOBJ.js 加载新的材质。

    // loadOBJ.js
    
    switch (objMaterial) {
        case 'PhongMaterial':
            material = buildPhongMaterial(colorMap, mat.specular.toArray(), light, Translation, Scale, "./src/shaders/phongShader/phongVertex.glsl", "./src/shaders/phongShader/phongFragment.glsl");
            shadowMaterial = buildShadowMaterial(light, Translation, Scale, "./src/shaders/shadowShader/shadowVertex.glsl", "./src/shaders/shadowShader/shadowFragment.glsl");
            break;
        // TODO: Add your PRTmaterial here
        //Edit Start
        case 'PRTMaterial':
            material = buildPRTMaterial("./src/shaders/prtShader/prtVertex.glsl", "./src/shaders/prtShader/prtFragment.glsl");
            break;
        //Edit End
        // ...
    }
    C#

    给场景添加mary模型,设置位置与大小,并且使用刚建立的材质。

    //engine.js
    
    // Add shapes
    ...
    // Edit Start
    let maryTransform = setTransform(0, -35, 0, 20, 20, 20);
    // Edit End
    ...
    // TODO: load model - Add your Material here
    ...
    // Edit Start
    loadOBJ(renderer, 'assets/mary/', 'mary', 'PRTMaterial', maryTransform);
    // Edit End
    C#

    计算着色

    将预计算数据载入GPU中。

    在渲染循环的camera pass中给材质设置precomputeL实时的值,也就是传递预先计算的数据给shader。下面代码是每一帧中每一趟camera pass中每一个网格mesh的每一个uniforms的遍历。实时渲染框架已经解析了预计算的数据并且存储到了uniforms中。precomputeL是一个 9×3 的矩阵,代表这里分别有RGB三个通道的前三阶(9个)球谐函数(实际上我们会说这是一个 3×3 的矩阵,但是我们写代码直接写成一个长度为9的数组)。为了方便使用,通过 tool.js 的函数将 precomputeL 转换为 3×9 的矩阵。

    通过 uniformMatrix3fv 函数,我们可以将材质里存储的信息上传到GPU上。这个函数接受三个参数,具体请查阅 WebGL文档 – uniformMatrix 。其中第一个参数的作用是在我们自己创建的 PRTMaterial 中,uniforms 包含了 uPrecomputeL[0] , uPrecomputeL[1] 和 uPrecomputeL[2] 。在GPU内的工作不需要我们关注,我们只需要有在CPU上的 uniform ,就可以通过API自动访问到GPU上对应的内容。换句话说,当获取一个 uniform 或属性的位置,实际上得到的是一个在CPU端的引用,但在底层,这个引用会映射到GPU上的一个具体位置。而链接 uniform 的步骤在 Shader.js 的 this.program = this.addShaderLocations() 中完成(看看代码就能懂了,只是比较绕,在我的HW1文章中也有分析过), shader.program 有三个属性分别是:glShaderProgram, uniforms, 和 attribs。而具体声明的位置则是在 XXXshader.glsl 中,在下一步中我们就会完成它。

    总结一下,下面这段代码主要工作就是为片段着色器提供预先处理过的数据。

    // WebGLRenderer.js
    
    if (k == 'uMoveWithCamera') { // The rotation of the skybox
        gl.uniformMatrix4fv(
            this.meshes[i].shader.program.uniforms[k],
            false,
            cameraModelMatrix);
    }
    
    // Bonus - Fast Spherical Harmonic Rotation
    //let precomputeL_RGBMat3 = getRotationPrecomputeL(precomputeL[guiParams.envmapId], cameraModelMatrix);
    
    // Edit Start
    let Mat3Value = getMat3ValueFromRGB(precomputeL[guiParams.envmapId]);
    
    if (/^uPrecomputeL\[\d\]$$/.test(k)) {
        let index = parseInt(k.split('[')[1].split(']')[0]);
        if (index >= 0 && index < 3) {
            gl.uniformMatrix3fv(
                this.meshes[i].shader.program.uniforms[k],
                false,
                Mat3Value[index]
            );
        }
    }
    // Edit End
    C#

    也可以将 Mat3Value 的计算放在i循环的外面,减少计算次数。

    编写顶点着色器

    明白了上面代码的作用之后,接下来的任务就非常明了了。上一步我们将每一个球谐系数都传到了 GPU 的 uPrecomputeL[] 中,接下来在GPU上编程计算球谐系数和传输矩阵的点乘,也就是下图 light_coefficient * transport_matrix。

    实时渲染框架中已经完成了Light_Transport到对应方向的矩阵的化简,我们只需要分别对三个颜色通道的长度为9的向量做点乘就行了。值得一提的是,PrecomputeL 和 PrecomputeLT 既可以传给顶点着色器也可以传给片段着色器,若传给顶点着色器,就只需要在片段着色器中差值得到颜色,速度更快,但是真实性就稍差一些。怎么计算取决于不同的需求。

    img
    //prtVertex.glsl
    
    attribute vec3 aVertexPosition;
    attribute vec3 aNormalPosition;
    attribute mat3 aPrecomputeLT;  // Precomputed Light Transfer matrix for the vertex
    
    uniform mat4 uModelMatrix;
    uniform mat4 uViewMatrix;
    uniform mat4 uProjectionMatrix;
    uniform mat3 uPrecomputeL[3];  // Precomputed Lighting matrices
    varying highp vec3 vNormal;
    
    varying highp vec3 vColor;     // Outgoing color after the dot product calculations
    
    float L_dot_LT(const mat3 PrecomputeL, const mat3 PrecomputeLT) {
      return dot(PrecomputeL[0], PrecomputeLT[0]) 
            + dot(PrecomputeL[1], PrecomputeLT[1]) 
            + dot(PrecomputeL[2], PrecomputeLT[2]);
    }
    
    void main(void) {
      // 防止因为浏览器优化报错,无实际作用
      aNormalPosition;
    
      for(int i = 0; i < 3; i++) {
          vColor[i] = L_dot_LT(aPrecomputeLT, uPrecomputeL[i]);
      }
    
      gl_Position = uProjectionMatrix * uViewMatrix * uModelMatrix * vec4(aVertexPosition, 1.0);
    }
    C#

    另外值得一说的是,在渲染框架中为一个名为 aNormalPosition 的attribute设置了数值,如果在Shader中没有使用的话就会被WebGL优化掉,导致浏览器不停报错。

    编写片元着色器

    在顶点着色器中完成对当前顶点着色的计算之后,在片元着色器中插值计算颜色。由于在顶点着色器中为每个顶点计算的vColor值会在片元着色器中被自动插值,因此直接使用就可以了。

    // prtFragment.glsl
    
    #ifdef GL_ES
    precision mediump float;
    #endif
    
    varying highp vec3 vColor;
    
    void main(){
      gl_FragColor = vec4(vColor, 1.0);
    }
    C#

    曝光与颜色矫正

    虽然框架作者提及PRT预计算保存的结果是在线性空间中的,不需要再进行 gamma 矫正了,但是显然最终结果是有问题的。如果您没有事先在计算系数的时候除 ,那么以Skybox场景为例子,就会出现过曝的问题。如果事先除了 ,但是没有做色彩矫正,就会在实时渲染框架中出现过暗的问题。

    img

    首先在计算系数的时候除以 ,然后再做一个色彩矫正。怎么做呢?我们可以参照nori框架的导出图片过程中有一个 toSRGB() 的函数:

    // common.cpp
    Color3f Color3f::toSRGB() const {
        Color3f result;
    
        for (int i=0; i<3; ++i) {
            float value = coeff(i);
    
            if (value <= 0.0031308f)
                result[i] = 12.92f * value;
            else
                result[i] = (1.0f + 0.055f)
                    * std::pow(value, 1.0f/2.4f) -  0.055f;
        }
    
        return result;
    }
    C#

    我们可以仿照这个在片元着色其中做色彩矫正。

    //prtFragment.glsl
    
    #ifdef GL_ES
    precision mediump float;
    #endif
    
    varying highp vec3 vColor;
    
    vec3 toneMapping(vec3 color){
        vec3 result;
    
        for (int i=0; i<3; ++i) {
            if (color[i] <= 0.0031308)
                result[i] = 12.92 * color[i];
            else
                result[i] = (1.0 + 0.055) * pow(color[i], 1.0/2.4) - 0.055;
        }
    
        return result;
    }
    
    void main(){
      vec3 color = toneMapping(vColor); 
      gl_FragColor = vec4(color, 1.0);
    }
    C#

    这样就可以保证实时渲染框架渲染的结果与nori框架的截图结果一致了。

    img

    我们也可以做其他的颜色矫正,这里提供几种常见的Tone Mapping方法,用于将HDR范围转换至LDR范围。

    vec3 linearToneMapping(vec3 color) {
        return color / (color + vec3(1.0));
    }
    vec3 reinhardToneMapping(vec3 color) {
        return color / (vec3(1.0) + color);
    }
    vec3 exposureToneMapping(vec3 color, float exposure) {
        return vec3(1.0) - exp(-color * exposure);
    }
    vec3 filmicToneMapping(vec3 color) {
        color = max(vec3(0.0), color - vec3(0.004));
        color = (color * (6.2 * color + 0.5)) / (color * (6.2 * color + 1.7) + 0.06);
        return color;
    }
    C#

    到这里为止,作业的基础部分就完成了。

    添加CornellBox场景

    默认框架代码中没有CornellBox,但是资源文件里面有,这就需要我们自行添加:

    // engine.js
    
    var envmap = [
        'assets/cubemap/GraceCathedral',
        'assets/cubemap/Indoor',
        'assets/cubemap/Skybox',
        // Edit Start
        'assets/cubemap/CornellBox',
        // Edit End
    ];
    // engine.js
    
    function createGUI() {
        const gui = new dat.gui.GUI();
        const panelModel = gui.addFolder('Switch Environemtn Map');
        // Edit Start
        panelModel.add(guiParams, 'envmapId', { 'GraceGathedral': 0, 'Indoor': 1, 'Skybox': 2, 'CornellBox': 3}).name('Envmap Name');
        // Edit End
        panelModel.open();
    }
    C#
    img

    基础部分结果展示

    分别展示shadowed和unshadowed的四个场景。

    img

    考虑传输项光线多次弹射(bonus 1)

    这是提高的第一部分。 计算多次弹射的光线传输与光线追踪有相似之处,在使用球谐函数(Spherical Harmonics,SH)进行光照近似时,您可以结合光线追踪来计算这些多次反射的效果。

    完整代码

    // TODO: leave for bonus
    Eigen::MatrixXf m_IndirectCoeffs = Eigen::MatrixXf::Zero(SHCoeffLength, mesh->getVertexCount());
    int sample_side = static_cast<int>(floor(sqrt(m_SampleCount)));
    
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_real_distribution<> rng(0.0, 1.0);
    
    const double twoPi = 2.0 * M_PI;
    
    for(int bo = 0; bo < m_Bounce; bo++)
    {
        for (int i = 0; i < mesh->getVertexCount(); i++)
        {
            const Point3f &v = mesh->getVertexPositions().col(i);
            const Normal3f &n = mesh->getVertexNormals().col(i);
    
            std::vector<float> coeff(SHCoeffLength, 0.0f);
            for (int t = 0; t < sample_side; t++) {
                for (int p = 0; p < sample_side; p++) {
                    double alpha = (t + rng(gen)) / sample_side;
                    double beta = (p + rng(gen)) / sample_side;
                    double phi = twoPi * beta;
                    double theta = acos(2.0 * alpha - 1.0);
    
                    Eigen::Array3d d = sh::ToVector(phi, theta);
                    const Vector3f wi(d[0], d[1], d[2]);
    
                    double H = wi.dot(n);
                    if(H > 0.0) {
                        const auto ray = Ray3f(v, wi);
                        Intersection intersect;
                        bool is_inter = scene->rayIntersect(ray, intersect);
                        if(is_inter) {
                            for(int j = 0; j < SHCoeffLength; j++) {
                                const Vector3f coef3(
                                    m_TransportSHCoeffs.col((int)intersect.tri_index[0]).coeffRef(j),
                                    m_TransportSHCoeffs.col((int)intersect.tri_index[1]).coeffRef(j),
                                    m_TransportSHCoeffs.col((int)intersect.tri_index[2]).coeffRef(j)
                                );
                                coeff[j] += intersect.bary.dot(coef3) / m_SampleCount;
                            }
                        }
                    }
                }
            }
    
            for (int j = 0; j < SHCoeffLength; j++)
            {
                m_IndirectCoeffs.col(i).coeffRef(j) = coeff[j] - m_IndirectCoeffs.col(i).coeffRef(j);
            }
        }
        m_TransportSHCoeffs += m_IndirectCoeffs;
    }
    C#

    分析

    在计算有遮挡的阴影的基础上(直接光照),加上二次反射光(间接照明)的贡献。而二次反射的光线也可以再进行相同的步骤。对于间接光照的计算,使用球谐函数对这些反射光线的照明进行近似。如果考虑多次弹射,则使用 进行递归计算,终止条件可以是递归深度或光线强度低于某个阈值。下面就是文字的公式描述。

    简略代码与注释如下:

    // TODO: leave for bonus
    // 首先初始化球谐系数
    Eigen::MatrixXf m_IndirectCoeffs = Eigen::MatrixXf::Zero(SHCoeffLength, mesh->getVertexCount());
    // 采样侧边的大小 = 样本数量的平方根 // 这样我们在后面可以进行二维的采样
    int sample_side = static_cast<int>(floor(sqrt(m_SampleCount)));
    
    // 生成随机数,范围是 [0,1]
    ...
    std::uniform_real_distribution<> rng(0.0, 1.0);
    
    // 定义常量 2 \pi
    ...
    
    // 循环计算多次反射 (m_Bounce 次)
    for (int bo = 0; bo < m_Bounce; bo++) {
      // 对每个顶点做处理
      // 对于每个顶点,会做如下操作
      // - 获取该顶点的位置和法线 v n
      // - rng()获得随机的二维方向 alpha beta
      // - 如果wi在顶点法线的同一侧,则继续进行:
      // - 生成一条从顶点出发的射线,并检查这条射线是否与场景中的其他物体相交
      // - 如果有相交的物体,代码会使用相交处的信息和现有的球谐系数来更新该顶点的光线间接反射信息。
      for (int i = 0; i < mesh->getVertexCount(); i++) {
        const Point3f &v = mesh->getVertexPositions().col(i);
        const Normal3f &n = mesh->getVertexNormals().col(i);
        ...
        for (int t = 0; t < sample_side; t++) {
          for (int p = 0; p < sample_side; p++) {
            ...
            double H = wi.dot(n);
            if (H > 0.0) {
              // 这里就是公式中的 $$(1-V(w_i))$$ 如果不满足,这一轮循环就不累加
              bool is_inter = scene->rayIntersect(ray, intersect);
              if (is_inter) {
                for (int j = 0; j < SHCoeffLength; j++) {
                  ...
                  coeff[j] += intersect.bary.dot(coef3) / m_SampleCount;
                }
              }
            }
          }
        }
        // 对于每个顶点,会根据计算的反射信息更新其球谐系数。
        for (int j = 0; j < SHCoeffLength; j++) {
          m_IndirectCoeffs.col(i).coeffRef(j) = coeff[j] - m_IndirectCoeffs.col(i).coeffRef(j);
        }
      }
      m_TransportSHCoeffs += m_IndirectCoeffs;
    }
    C#

    在之前的步骤中,我们只是计算了每一个顶点的球谐函数,并不涉及到三角形中心的插值计算。但是在光线多次弹射的实现中,从顶点向正半球发射的光线会与顶点之外的位置相交,因此我们需要通过重心坐标插值计算获取发射光线与三角形内部的交点的信息,这就是 intersect.bary 的作用。

    结果

    观察一下,整体上没有太大差异,只是阴影的地方更加亮了。

    img

    环境光照球谐函数旋转(bonus 2)

    提高2。 低阶的球鞋光照的旋转可以使用「低阶SH快速旋转方法」。

    代码

    首先让Skybox转起来。 [0, 1, 0] 意味着绕y轴旋转。然后通过 getRotationPrecomputeL 函数计算旋转后的球谐函数。最后应用到 Mat3Value 。

    // WebGLRenderer.js
    let cameraModelMatrix = mat4.create();
    // Edit Start
    mat4.fromRotation(cameraModelMatrix, timer, [0, 1, 0]);
    // Edit End
    if (k == 'uMoveWithCamera') { // The rotation of the skybox
        gl.uniformMatrix4fv(
            this.meshes[i].shader.program.uniforms[k],
            false,
            cameraModelMatrix);
    }
    
    // Bonus - Fast Spherical Harmonic Rotation
    // Edit Start
    let precomputeL_RGBMat3 = getRotationPrecomputeL(precomputeL[guiParams.envmapId], cameraModelMatrix);
    Mat3Value = getMat3ValueFromRGB(precomputeL_RGBMat3);
    // Edit End
    C#

    接下来跳转到 tool.js ,编写 getRotationPrecomputeL 函数。

    // tools.js
    function getRotationPrecomputeL(precompute_L, rotationMatrix){
        let rotationMatrix_inverse = mat4.create()
        mat4.invert(rotationMatrix_inverse, rotationMatrix)
        let r = mat4Matrix2mathMatrix(rotationMatrix_inverse)
    
        let shRotateMatrix3x3 = computeSquareMatrix_3by3(r);
        let shRotateMatrix5x5 = computeSquareMatrix_5by5(r);
    
        let result = [];
        for(let i = 0; i < 9; i++){
            result[i] = [];
        }
        for(let i = 0; i < 3; i++){
            let L_SH_R_3 = math.multiply([precompute_L[1][i], precompute_L[2][i], precompute_L[3][i]], shRotateMatrix3x3);
            let L_SH_R_5 = math.multiply([precompute_L[4][i], precompute_L[5][i], precompute_L[6][i], precompute_L[7][i], precompute_L[8][i]], shRotateMatrix5x5);
    
            result[0][i] = precompute_L[0][i];
            result[1][i] = L_SH_R_3._data[0];
            result[2][i] = L_SH_R_3._data[1];
            result[3][i] = L_SH_R_3._data[2];
            result[4][i] = L_SH_R_5._data[0];
            result[5][i] = L_SH_R_5._data[1];
            result[6][i] = L_SH_R_5._data[2];
            result[7][i] = L_SH_R_5._data[3];
            result[8][i] = L_SH_R_5._data[4];
        }
    
        return result;
    }
    
    function computeSquareMatrix_3by3(rotationMatrix){ // 计算方阵SA(-1) 3*3 
    
        // 1、pick ni - {ni}
        let n1 = [1, 0, 0, 0]; let n2 = [0, 0, 1, 0]; let n3 = [0, 1, 0, 0];
    
        // 2、{P(ni)} - A  A_inverse
        let n1_sh = SHEval(n1[0], n1[1], n1[2], 3)
        let n2_sh = SHEval(n2[0], n2[1], n2[2], 3)
        let n3_sh = SHEval(n3[0], n3[1], n3[2], 3)
    
        let A = math.matrix(
        [
            [n1_sh[1], n2_sh[1], n3_sh[1]], 
            [n1_sh[2], n2_sh[2], n3_sh[2]], 
            [n1_sh[3], n2_sh[3], n3_sh[3]], 
        ]);
    
        let A_inverse = math.inv(A);
    
        // 3、用 R 旋转 ni - {R(ni)}
        let n1_r = math.multiply(rotationMatrix, n1);
        let n2_r = math.multiply(rotationMatrix, n2);
        let n3_r = math.multiply(rotationMatrix, n3);
    
        // 4、R(ni) SH投影 - S
        let n1_r_sh = SHEval(n1_r[0], n1_r[1], n1_r[2], 3)
        let n2_r_sh = SHEval(n2_r[0], n2_r[1], n2_r[2], 3)
        let n3_r_sh = SHEval(n3_r[0], n3_r[1], n3_r[2], 3)
    
        let S = math.matrix(
        [
            [n1_r_sh[1], n2_r_sh[1], n3_r_sh[1]], 
            [n1_r_sh[2], n2_r_sh[2], n3_r_sh[2]], 
            [n1_r_sh[3], n2_r_sh[3], n3_r_sh[3]], 
    
        ]);
    
        // 5、S*A_inverse
        return math.multiply(S, A_inverse)   
    
    }
    
    function computeSquareMatrix_5by5(rotationMatrix){ // 计算方阵SA(-1) 5*5
    
        // 1、pick ni - {ni}
        let k = 1 / math.sqrt(2);
        let n1 = [1, 0, 0, 0]; let n2 = [0, 0, 1, 0]; let n3 = [k, k, 0, 0]; 
        let n4 = [k, 0, k, 0]; let n5 = [0, k, k, 0];
    
        // 2、{P(ni)} - A  A_inverse
        let n1_sh = SHEval(n1[0], n1[1], n1[2], 3)
        let n2_sh = SHEval(n2[0], n2[1], n2[2], 3)
        let n3_sh = SHEval(n3[0], n3[1], n3[2], 3)
        let n4_sh = SHEval(n4[0], n4[1], n4[2], 3)
        let n5_sh = SHEval(n5[0], n5[1], n5[2], 3)
    
        let A = math.matrix(
        [
            [n1_sh[4], n2_sh[4], n3_sh[4], n4_sh[4], n5_sh[4]], 
            [n1_sh[5], n2_sh[5], n3_sh[5], n4_sh[5], n5_sh[5]], 
            [n1_sh[6], n2_sh[6], n3_sh[6], n4_sh[6], n5_sh[6]], 
            [n1_sh[7], n2_sh[7], n3_sh[7], n4_sh[7], n5_sh[7]], 
            [n1_sh[8], n2_sh[8], n3_sh[8], n4_sh[8], n5_sh[8]], 
        ]);
    
        let A_inverse = math.inv(A);
    
        // 3、用 R 旋转 ni - {R(ni)}
        let n1_r = math.multiply(rotationMatrix, n1);
        let n2_r = math.multiply(rotationMatrix, n2);
        let n3_r = math.multiply(rotationMatrix, n3);
        let n4_r = math.multiply(rotationMatrix, n4);
        let n5_r = math.multiply(rotationMatrix, n5);
    
        // 4、R(ni) SH投影 - S
        let n1_r_sh = SHEval(n1_r[0], n1_r[1], n1_r[2], 3)
        let n2_r_sh = SHEval(n2_r[0], n2_r[1], n2_r[2], 3)
        let n3_r_sh = SHEval(n3_r[0], n3_r[1], n3_r[2], 3)
        let n4_r_sh = SHEval(n4_r[0], n4_r[1], n4_r[2], 3)
        let n5_r_sh = SHEval(n5_r[0], n5_r[1], n5_r[2], 3)
    
        let S = math.matrix(
        [    
            [n1_r_sh[4], n2_r_sh[4], n3_r_sh[4], n4_r_sh[4], n5_r_sh[4]], 
            [n1_r_sh[5], n2_r_sh[5], n3_r_sh[5], n4_r_sh[5], n5_r_sh[5]], 
            [n1_r_sh[6], n2_r_sh[6], n3_r_sh[6], n4_r_sh[6], n5_r_sh[6]], 
            [n1_r_sh[7], n2_r_sh[7], n3_r_sh[7], n4_r_sh[7], n5_r_sh[7]], 
            [n1_r_sh[8], n2_r_sh[8], n3_r_sh[8], n4_r_sh[8], n5_r_sh[8]], 
        ]);
    
        // 5、S*A_inverse
        return math.multiply(S, A_inverse)  
    }
    
    function mat4Matrix2mathMatrix(rotationMatrix){
    
        let mathMatrix = [];
        for(let i = 0; i < 4; i++){
            let r = [];
            for(let j = 0; j < 4; j++){
                r.push(rotationMatrix[i*4+j]);
            }
            mathMatrix.push(r);
        }
        // Edit Start
        //return math.matrix(mathMatrix)
        return math.transpose(mathMatrix)
        // Edit End
    }
    function getMat3ValueFromRGB(precomputeL){
    
        let colorMat3 = [];
        for(var i = 0; i<3; i++){
            colorMat3[i] = mat3.fromValues( precomputeL[0][i], precomputeL[1][i], precomputeL[2][i],
                                            precomputeL[3][i], precomputeL[4][i], precomputeL[5][i],
                                            precomputeL[6][i], precomputeL[7][i], precomputeL[8][i] ); 
        }
        return colorMat3;
    }
    C#

    结果

    img

    动画GIF可以在此处 获取。

    原理

    两项关键性质

    首先简单说说原理,这里利用了球谐函数的两个性质。

    1. 旋转不变性

    在三维空间中旋转一个函数的坐标,并将这个旋转后的坐标代入球谐函数,那么你会得到与原始函数相同的结果。

    1. 旋转的线性性

    对于球谐函数的每一“层”或“带”(band)(也就是给定的阶数 l 的所有球谐函数),其SH系数可以被旋转,并且这个旋转是线性的。也就是说,可以通过一个矩阵乘法来旋转一个球谐函数展开的系数。

    Wigner D矩阵旋转方法概述

    球谐函数的旋转是一个深入的话题,这里直接概述,不涉及复杂的数学证明。 作业框架中给的是基于投影的方法,本文先介绍一个更精确的方法,Wigner D矩阵。 更加详细的内容请去看:球谐光照笔记(旋转篇) – 网易游戏雷火事业群的文章 – 知乎 ,反正我是没看懂QAQ。

    由于当前使用的是前三阶的球谐函数,并且 band0 只有一个投影系数,所以我们只需要处理band1, band2 两层上各自 , 的旋转矩阵 。

    球谐函数 的旋转可以表示为:

    其中, 是旋转矩阵元素,它给出了如何将球谐系数从原始方向旋转到新方向。

    假设有一个函数 ,它可以展开为球谐函数的线性组合 :

    如果想要旋转这个函数,我们不直接旋转每一个球谐函数,而是旋转它们的系数。新的展开系数 可以由原始系数 通过旋转矩阵得到 :

    接下来就到关键的一步了,如何计算旋转矩阵

    img

    在作业框架中,我们了解到,band 1需要构建一个 的矩阵,band 2需要构建 的矩阵。也就是说,对于每个阶数为 的 band,它都有 个合法的解,每个解对应当前 band 上的一个基函数,这是勒让德方程的一个特性。

    现在,我们来考虑旋转的影响。

    当我们旋转一个环境光照 ,我们不会去旋转基函数,而是“旋转”所有的系数。旋转一个特定的系数的过程涉及到使用Wigner D矩阵 。首先,当我们谈论旋转,我们通常指的是围绕某个轴的旋转,定义由欧拉角来指定。我们就为每一阶都计算一个边长是 的方阵 。

    一旦得到了每一阶对应的旋转矩阵,我们就可以轻松计算出“旋转”后的新系数:

    然而,计算Wigner D矩阵的元素可能会有些复杂,特别是对于较高的阶数。因此,作业提示中给出的是一种基于投影的方法。接下来我们看看上面两段代码是怎么实现的。

    投影的近似方法

    首先,选择 个 normal vector ,这个量的选取需要确保线性独立性,也就是尽可能均匀的覆盖球面(Fibonacci球面采样也许是个不错的选择),否则在后面会出现计算奇异的矩阵的错误,确保生成的矩阵是满秩的

    对于每一个normal vector ,在球谐函数上投影(SHEval函数),这实际上是在计算球谐函数与该方向上的点乘。从这个投影中,可以得到一个 维向量 ,它的每一个分量都是球谐函数的一个系数。

    使用上面得到的 向量,我们可以构建矩阵 和逆矩阵 。如果我们记 为 normal vector 在球谐函数上的第 个系数, 那么矩阵 可以写为:

    对于每一个normal vector , 应用旋转 , 得到 ,即(前乘):

    然后,对于这些旋转后的normal vectors, 再次进行球谐函数投影, 得到 。

    使用从旋转后的normal vectors得到的 向量, 我们可以构建矩阵S。计算旋转矩阵 : 旋转矩阵 可以告诉我们如何通过简单的矩阵乘法来旋转球谐系数。

    使用矩阵 乘以原始的球谐系数向量,我们可以得到旋转后的球谐系数。对每个 层重复: 为了得到完整的旋转后的球谐系数,我们需要对每个 层重复上述过程。

    Reference

    1. Games 202
    2. https://github.com/DrFlower/GAMES_101_202_Homework/tree/main/Homework_202/Assignment2
  • Games202 作业一 软阴影实现

    Games202 作业一 软阴影实现

    本文内容:JS和WebGL相关知识、2-pass shadow算法、BIAS缓解自遮挡、PCF算法、PCSS、物体移动。

    项目源代码:

    GitHub – Remyuu/GAMES202-Homework: GAMES202-Homework​

    上面这个图画着好玩。


    写在前面

    由于我对JS以及WebGL一窍不通,只能遇事不决 console.log() 。

    除了作业要求的内容,我在coding的时候也有一些疑问,希望大佬解答QAQ。

    1. 如何实现动态的点光源阴影效果?我们需要使用点光源阴影技术才可以实现万向阴影贴图(omnidirectional shadow maps),具体怎么做?
    2. possionDiskSamples函数并不是真正的泊松圆盘分布?

    框架修正

    在作业开始时请先对作业框架做一些修正。框架改动原文:https://games-cn.org/forums/topic/zuoyeziliao-daimakanwu/

    • 框架提供的 unpack 函数算法实现不准确,在不加 bias 时,会导致严重的 banding(地面一半白一半黑而不是典型的 z-fighting 效果),一定程度上影响作业调试。
    // homework1/src/shaders/shadowShader/shadowFragment.glsl
    vec4 pack (float depth) {
        // 使用rgba 4字节共32位来存储z值,1个字节精度为1/255
        const vec4 bitShift = vec4(1.0, 255.0, 255.0 * 255.0, 255.0 * 255.0 * 255.0);
        const vec4 bitMask = vec4(1.0/255.0, 1.0/255.0, 1.0/255.0, 0.0);
        // gl_FragCoord:片元的坐标,fract():返回数值的小数部分
        vec4 rgbaDepth = fract(depth * bitShift); //计算每个点的z值
        rgbaDepth -= rgbaDepth.gbaa * bitMask; // Cut off the value which do not fit in 8 bits
        return rgbaDepth;
    }
    
    // homework1/src/shaders/phongShader/phongFragment.glsl
    float unpack(vec4 rgbaDepth) {
        const vec4 bitShift = vec4(1.0, 1.0/255.0, 1.0/(255.0*255.0), 1.0/(255.0*255.0*255.0));
        return dot(rgbaDepth, bitShift);
    }
    • 清屏还需要添加一个glClear。
    // homework1/src/renderers/WebGLRenderer.js
    gl.clearColor(0.0, 0.0, 0.0,1.0);// Clear to black, fully opaque
    gl.clearDepth(1.0);// Clear everything
    gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);

    JS最基础的知识

    变量

    • 在JavaScript中,我们主要使用 var,let 和 const 这三个关键字来声明变量/常量。
    • var是声明变量的关键字,可以在整个函数范围内使用声明的变量(函数作用域)。
    • let行为与 var 类似,也是声明了一个变量,但是 let 的作用域限制在块中(块作用域),比如 for 循环或 if 语句中定义的块。
    • const:用于声明常量。 const 的作用域也是块级别的。
    • 推荐使用 let 和 const 而不是 var 来声明变量,因为它们遵循块级作用域,更符合大多数编程语言中的作用域规则,更易理解和预测。

    一个基本的JavaScript类的结构如下:

    class MyClass {
      constructor(parameter1, parameter2) {
        this.property1 = parameter1;
        this.property2 = parameter2;
      }
      method1() {
        // method body
      }
      static sayHello() {
        console.log('Hello!');
      }
    }

    创建实例:

    let myInstance = new MyClass('value1', 'value2');
    myInstance.method1(); // 调用类的方法

    也可以直接调用静态类(不用创建实例了):

    MyClass.sayHello();  // "Hello!"

    项目流程简述

    程序入口是engine.js,主函数 GAMES202Main 。首先初始化WebGL相关的内容,包括相机、相机交互、渲染器、光源、物体加载、用户GUI界面以及最重要的主循环main loop部分。

    物体加载过程,会调用loadOBJ.js。首先从文件中加载对应的glsl,构建Phong材质、Phong相关阴影还有阴影的材质。

    // loadOBJ.js
    case 'PhongMaterial':
        material = buildPhongMaterial(colorMap, mat.specular.toArray(), light, Translation, Scale, "./src/shaders/phongShader/phongVertex.glsl", "./src/shaders/phongShader/phongFragment.glsl");
        shadowMaterial = buildShadowMaterial(light, Translation, Scale, "./src/shaders/shadowShader/shadowVertex.glsl", "./src/shaders/shadowShader/shadowFragment.glsl");
        break;
    }

    然后,通过MeshRender直接生成2-pass阴影Shadow Map和常规的Phong材质,具体代码如下:

    // loadOBJ.js
    material.then((data) => {
        // console.log("现在制作表面材质")
        let meshRender = new MeshRender(renderer.gl, mesh, data);
        renderer.addMeshRender(meshRender);
    });
    shadowMaterial.then((data) => {
        // console.log("现在制作阴影材质")
        let shadowMeshRender = new MeshRender(renderer.gl, mesh, data);
        renderer.addShadowMeshRender(shadowMeshRender);
    });

    注意到,MeshRender具备一定的通用性,它接受任何类型的材质作为其参数。具体是怎么区分的呢?通过判断传入的material.frameBuffer是否为空,如果是空,将加载表面材质,否则加载阴影图Shadow Map。在MeshRender.js的draw()函数中,看到如下代码:

    // MeshRender.js
    if (this.material.frameBuffer != null) {
        // Shadow map
        gl.viewport(0.0, 0.0, resolution, resolution);
    } else {
        gl.viewport(0.0, 0.0, window.screen.width, window.screen.height);
    }

    利用MeshRender生成了阴影之后,推入到renderer中,可以在 WebGLRenderer.js 中找到对应实现:

    addShadowMeshRender(mesh) { this.shadowMeshes.push(mesh); }

    最后进入mainLoop()主循环实现一帧帧的更新画面。

    项目流程详细解释

    这一章节将会从一个小问题出发,探讨片段着色器是如何构造的。这将会串联起几乎整个项目,而这也是我认为比较舒服的阅读项目流程。

    glsl是在哪里工作? — 从片段着色器的流程入手详细讲解代码流程

    在上文中我们并没有详细提及glsl文件是怎么调用的,这里我们详细说说。

    首先在loadOBJ.js中首次用过路径的方式将.glsl文件引入:

    // loadOBJ.js - function loadOBJ()
    material = buildPhongMaterial(colorMap, mat.specular.toArray(), light, Translation, Scale, "./src/shaders/phongShader/phongVertex.glsl", "./src/shaders/phongShader/phongFragment.glsl");
    shadowMaterial = buildShadowMaterial(light, Translation, Scale, "./src/shaders/shadowShader/shadowVertex.glsl", "./src/shaders/shadowShader/shadowFragment.glsl");

    这里以phongFragment.glsl为例子,phongFragment.glsl通过位于 PhongMaterial.js 的buildPhongMaterial函数中的 getShaderString方法将glsl代码从硬盘中加载进来,与此同时将glsl代码通过构造参数的形式传入并用之构造一个PhongMaterial对象。PhongMaterial在构造的过程中会调用super()函数实现父类Material.js的构造函数,即将glsl代码传到Material.js中:

    // PhongMaterial.js
    super({...}, [], ..., fragmentShader);

    在c++中,子类可以选择是否完全继承父类的构造函数的参数。这里父类的构造函数有5个,实际只实现了4个,这也是完全没问题的。

    在Material.js中,子类通过构造函数的第四个参数#fsSrc将glsl代码传到了此处。至此,glsl代码的传送之路就走到了尽头,接下来等待他的将是一个名为compile()的函数。

    // Material.js
    this.#fsSrc = fsSrc;
    ...
    compile(gl) {
        return new Shader(..., ..., this.#fsSrc,{...});
    }

    至于这个compile函数什么时候调用呢?回到loadOBJ.js的流程中,现在我们已经完全执行完毕buildPhongMaterial()代码,接下来就到了上一小节提及到的then()部分。

    注意,loadOBJ()只是一个函数,不是对象!

    // loadOBJ.js
    material.then((data) => {
        let meshRender = new MeshRender(renderer.gl, mesh, data);
        renderer.addMeshRender(meshRender);
        renderer.ObjectID[ObjectID][0].push(renderer.meshes.length - 1);
    });

    在构造MeshRender对象时,就会调用compile():

    // MeshRender.js
    constructor(gl, mesh, material) {
    ...
        this.shader = this.material.compile(gl);
    }
    // Material.js
    compile(gl) {
        return new Shader(..., ..., this.#fsSrc,{...});
    }

    接下来,我们具体看一下shader.js的构造。Material在构造shader对象时实现了所有的四个构造参数。这里还是挑重点fsSrc看,即继续看看glsl代码接下来的命运。

    // shader.js
    constructor(gl, vsSrc, fsSrc, shaderLocations) {
        ...
        const fs = this.compileShader(fsSrc, ...);
        ...
    }

    在构造shader对象实现fs编译着色器时是通过compileShader()函数的。这个compileShader函数会创建一个全局变量shader,代码如下:

    // shader.js
    compileShader(shaderSource, shaderType) {
        const gl = this.gl;
        var shader = gl.createShader(shaderType);
        gl.shaderSource(shader, shaderSource);
        gl.compileShader(shader);
    
        if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
            console.error(shaderSource);
            console.error('shader compiler error:\n' + gl.getShaderInfoLog(shader));
        }
    
        return shader;
    };

    这个gl是什么呢?是在loadOBJ()通过构造MeshRender对象时以参数renderer.gl一路传到shader.js中的。而renderer则是loadOBJ()的第一个参数,在engine.js中传入。

    实际上loadOBJ.js中renderer是一个WebGLRenderer对象。而renderer.gl的gl是在engine.js中创建的:

    // engine.js
    const gl = canvas.getContext('webgl');

    而gl可以理解为从index.html中获取canvas的WebGL对象。实际上gl为开发者提供了一个接口来与WebGL API进行交互。

    <!-- index.html -->
    <canvas id="glcanvas"></canvas>

    WebGL推荐参考资料:

    1. https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API
    2. https://webglfundamentals.org
    3. https://www.w3cschool.cn/webgl/vjxu1jt0.html

    Tips:网站都有对应的中文版本,但是有能力的还是推荐阅读英文版本~ WebGL API:

    1. https://developer.mozilla.org/en-US/docs/Web/API
    2. https://webglfundamentals.org/docs/

    知道了gl是什么之后,自然也就发现了项目框架是在哪里通过什么方式与WebGL联系在一起的了。

    // Shader.js
    compileShader(shaderSource, shaderType) {
        const gl = this.gl;
        var shader = gl.createShader(shaderType);
        gl.shaderSource(shader, shaderSource);
        gl.compileShader(shader);
    
        if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS)) {
            console.error(shaderSource);
            console.error('shader compiler error:\n' + gl.getShaderInfoLog(shader));
        }
    
        return shader;
    };

    也就是说,所有关于gl的方法都是通过WebGL API调用的。gl.createShader就是我们接触到的第一个WebGL API。

    我们只需知道这个createShader()函数会返回 WebGLShader 着色器对象。我们在后文再详细说明,这里先关注shaderSource究竟何去何从。

    • gl.shaderSource:A string containing the GLSL source code to set.

    也就是说,我们一路追踪的GLSL源代码是通过gl.shaderSource函数解析进了WebGLShader中。

    然后通过gl.compileShader()函数编译WebGLShader,使其成为为二进制数据,然后就可以被WebGLProgram对象所使用。

    简单地说,WebGLProgram是一个包含已编译的WebGL着色器的GLSL程序,至少需要包含一个顶点着色器和一个片段着色器。在WebGL中,会创建一个或多个WebGLProgram对象,每个对象包含一组特定的渲染指令。通过使用不同的WebGLProgram,就可以实现各种画面。

    if语句是检查着色器是否成功编译的部分。如果编译失败,则执行括号内的代码。最后,返回编译后(或尝试编译后)的着色器对象shader。

    至此,我们就完成了将GLSL文件从硬盘中取出最后编译进着色器对象的工作。

    但是渲染的流程还没有结束。回到Shadow对象的构造处:

    // Shadow.js
    class Shader {
        constructor(gl, vsSrc, fsSrc, shaderLocations) {
            this.gl = gl;
            const vs = this.compileShader(vsSrc, gl.VERTEX_SHADER);
            const fs = this.compileShader(fsSrc, gl.FRAGMENT_SHADER);
    
            this.program = this.addShaderLocations({
                glShaderProgram: this.linkShader(vs, fs),
            }, shaderLocations);
        }
        ...

    虽然刚才我们只解说了片段着色器的GLSL编译流程,但是顶点着色器也是相当类似的,故此省略。


    这里我们介绍linkShader()链接着色器的流程。代码在文字的下方。

    1. 首先创建一个创建程序命名为WebGLProgram。
    2. 将编译后的顶点着色器和片段着色器vs和fs添加到程序中,这一步叫做附加着色器。具体而言是使用gl.attachShader()将他们附加到WebGLProgram上。
    3. 使用gl.linkProgram()链接WebGLProgram。这会生成一个可执行的程序,该程序结合了前面附加的着色器。这一步叫做链接程序
    4. 最后检查链接状态,返回WebGL对象。
    // Shader.js
    linkShader(vs, fs) {
        const gl = this.gl;
        var prog = gl.createProgram();
        gl.attachShader(prog, vs);
        gl.attachShader(prog, fs);
        gl.linkProgram(prog);
    
        if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) {
            abort('shader linker error:\n' + gl.getProgramInfoLog(prog));
        }
        return prog;
    };

    WebGLProgram可以被视为着色器的容器,它包含了将3D数据转换为屏幕上的2D像素所需的全部信息和指令。


    得到与着色器链接的程序glShaderProgram后,会与shaderLocations对象一同被载入。

    简单地说shaderLocations对象包含了两个属性

    • Attributes是”个体”的数据(比如每个顶点的信息)
    • Uniforms是”整体”的数据(比如一个灯光的信息)

    框架将载入的流程打包进了addShaderLocations()中。简单地说,经过这一步操作之后,当你需要给这些uniform和attribute赋值时,就可以直接通过已经获取到的位置进行操作,而不需要每次都去查询位置。

    addShaderLocations(result, shaderLocations) {
        const gl = this.gl;
        result.uniforms = {};
        result.attribs = {};
    
        if (shaderLocations && shaderLocations.uniforms && shaderLocations.uniforms.length) {
            for (let i = 0; i < shaderLocations.uniforms.length; ++i) {
                result.uniforms = Object.assign(result.uniforms, {
                    [shaderLocations.uniforms[i]]: gl.getUniformLocation(result.glShaderProgram, shaderLocations.uniforms[i]),
                });
            }
        }
        if (shaderLocations && shaderLocations.attribs && shaderLocations.attribs.length) {
            for (let i = 0; i < shaderLocations.attribs.length; ++i) {
                result.attribs = Object.assign(result.attribs, {
                    [shaderLocations.attribs[i]]: gl.getAttribLocation(result.glShaderProgram, shaderLocations.attribs[i]),
                });
            }
        }
    
        return result;
    }

    回顾一下目前已经完成的工作:成功构建好了一个编译后(或尝试编译后)的Shader着色器对象给MeshRender:

    // MeshRender.js - construct()
    this.shader = this.material.compile(gl);

    至此,loadOBJ的任务已经圆满完成。在engine.js中,这样的加载要做三次:

    // loadOBJ(renderer, path, name, objMaterial, transform, meshID);
    loadOBJ(renderer, 'assets/mary/', 'Marry', 'PhongMaterial', obj1Transform);
    loadOBJ(renderer, 'assets/mary/', 'Marry', 'PhongMaterial', obj2Transform);
    loadOBJ(renderer, 'assets/floor/', 'floor', 'PhongMaterial', floorTransform);

    接下来来到程式主循环mainLoop。也即,一个循环表示一帧:

    // engine.js
    loadOBJ(...);
    ...
    function mainLoop() {...}
    ...

    程序主循环 — mainLoop()

    实际上,执行mainLoop,该函数会再次调用自己,形成一个无限循环。这就是所谓的游戏循环或动画循环的基础机制。

    // engine.js
    function mainLoop() {
        cameraControls.update();
        renderer.render();
        requestAnimationFrame(mainLoop);
    };
    requestAnimationFrame(mainLoop);

    cameraControls.update();在更新相机的位置或方向,例如响应用户的输入。

    renderer.render();场景被渲染或绘制到屏幕上。具体的渲染内容和方式取决于renderer对象的实现。

    requestAnimationFrame的好处是它会尽量与屏幕的刷新率同步,这样可以提供更流畅的动画和更高的性能,因为它不会在屏幕刷新之间无谓地执行代码。

    关于requestAnimationFrame()函数的详细信息可以参考以下文章: https://developer.mozilla.org/en-US/docs/Web/API/window/requestAnimationFrame

    接下来重点关心render()函数的运作。

    render()渲染函数

    这是一个典型的光源渲染、阴影渲染和最终摄像机视角渲染的流程。此处就不详细展开了,放到后面的多光源部分。

    // WebGLRenderer.js - render()
    const gl = this.gl;
    
    gl.clearColor(0.0, 0.0, 0.0, 1.0); // shadowmap默认白色(无遮挡),解决地面边缘产生阴影的问题(因为地面外采样不到,默认值为0会认为是被遮挡)
    gl.clearDepth(1.0);// Clear everything
    gl.enable(gl.DEPTH_TEST); // Enable depth testing
    gl.depthFunc(gl.LEQUAL); // Near things obscure far things
    
    console.assert(this.lights.length != 0, "No light");
    console.assert(this.lights.length == 1, "Multiple lights");
    
    for (let l = 0; l < this.lights.length; l++) {
        gl.bindFramebuffer(gl.FRAMEBUFFER, this.lights[l].entity.fbo);
        gl.clear(gl.DEPTH_BUFFER_BIT);
        // Draw light
        // TODO: Support all kinds of transform
        this.lights[l].meshRender.mesh.transform.translate = this.lights[l].entity.lightPos;
        this.lights[l].meshRender.draw(this.camera);
    
        // Shadow pass
        if (this.lights[l].entity.hasShadowMap == true) {
            for (let i = 0; i < this.shadowMeshes.length; i++) {
                this.shadowMeshes[i].draw(this.camera);
            }
        }
    }
    // Camera pass
    for (let i = 0; i < this.meshes.length; i++) {
        this.gl.useProgram(this.meshes[i].shader.program.glShaderProgram);
        this.gl.uniform3fv(this.meshes[i].shader.program.uniforms.uLightPos, this.lights[0].entity.lightPos);
        this.meshes[i].draw(this.camera);
    }

    GLSL快速入门 — 分析片段着色器FragmentShader.glsl

    上文我们讨论了如何载入GLSL,这一章节介绍GLSL的概念与实际用法。

    在WebGL中进行渲染时,我们需要至少一个 顶点着色器(Vertex Shader) 和一个 片段着色器(Fragment Shader) 才能绘制出一幅画面。上一节我们以片段着色器为例,介绍了框架是怎么将GLSL文件从硬盘读取进renderer的。接下来我们也以Flagment Shader片段着色器为例子(即phongFragment.glsl),介绍编写GLSL的流程。

    FragmentShader.glsl有什么用?

    Fragment Shader的作用是在光栅化的时候为当前像素渲染正确的颜色。以下是一个Fragment Shader的最简单形式,其包含一个main()函数,在函数其中指定了当前像素的颜色gl_FragColor。

    void main(void){
        ...
        gl_FragColor = vec4(Color, 1.0);
    }

    Fragment Shader接受什么数据?

    Fragment Shader需要知道数据,数据是由以下三种主要方式提供的,具体的用法可以参考 附录1.6

    1. Uniforms (全局变量): 这些是在单个绘制调用中对所有顶点和片段都保持不变的值。常见的例子包括变换矩阵(平移旋转等操作)、光源参数和材质属性。由于它们在绘制调用中是恒定的,所以称为“uniform”。
    2. Textures (纹理): 纹理是图像数据数组,它们可以被片段着色器采样来为每个片段获得颜色、法线或其他类型的信息。
    3. Varyings (可变量): 这些是顶点着色器输出的值,它们在图形基元(如三角形)的顶点之间插值,并传递给片段着色器。这允许我们在顶点着色器中计算值(如变换后的位置或顶点颜色),并在片段之间进行插值,以便在片段着色器中使用。

    项目中用了Uniforms和Varyings两种。

    GLSL基本语法

    这里不会把基本的用法过一篇,因为那样太无聊了。我们直接看项目:

    // phongFragment.glsl - PCF pass
    void main(void) {
        // 声明变量
        float visibility;     // 可见性(用于阴影)
        vec3 shadingPoint;     // 从光源处的视点坐标
        vec3 phongColor;      // 计算出的Phong光照颜色
    
        // 将vPositionFromLight的坐标值归一化到[0,1]范围内
        shadingPoint = vPositionFromLight.xyz / vPositionFromLight.w;
        shadingPoint = shadingPoint * 0.5 + 0.5; // 进行坐标转换,使其在[0,1]范围内
    
        // 计算可见性(阴影)。
        visibility = PCF(uShadowMap, vec4(shadingPoint, 1.0)); // 使用PCF(Percentage Closer Filtering)技术
    
        // 使用blinnPhong()函数计算Phong光照颜色
        phongColor = blinnPhong();
    
        // 计算最终的片段颜色,将Phong光照颜色与可见性相乘,得到考虑阴影的片段颜色
        gl_FragColor = vec4(phongColor * visibility, 1.0);
    }

    和c语言一样,glsl是强类型语言,你不能这样赋值:float visibility = 1;,因为1是int类型。

    矢量或矩阵

    另外,glsl还内置了很多特别的类型,比如浮点类型向量vec2, vec3和 vec4,矩阵类型mat2, mat3 和 mat4。

    上面这些数据的访问方式也比较有意思,

    • .xyzw:通常用于表示三维或四维空间中的点或向量。
    • .rgba:当向量表示颜色时使用,其中r代表红色,g代表绿色,b代表蓝色,a代表透明度。
    • .stpq:当向量用作纹理坐标时使用。

    因此,

    • v.x 与 v[0] 与 v.r 与 v.s 都表示该向量的第一个分量。
    • v.y 与 v[1] 与 v.g 与 v.t 都表示该向量的第二个分量。
    • 对于vec3和vec4,v.z 与 v[2] 与 v.b 与 v.p 都表示该向量的第三个分量。
    • 对于vec4,v.w 与 v[3] 与 v.a 与 v.q 都表示该向量的第四个分量。

    你甚至可以使用一种叫“分量重组”或“分量选择”的方式访问这些类型的数据:

    1. 重复某个分量:
    2. v.yyyy 会得到一个新的vec4,其中每个分量都是原始v的y分量。这与vec4(v.y, v.y, v.y, v.y)的效果相同。
    3. 交换分量:
    4. v.bgra 会得到一个新的vec4,其中的分量按照b, g, r, a的顺序从v中选取。这与vec4(v.b, v.g, v.r, v.a)的效果相同。

    当构造一个矢量或矩阵时可以一次提供多个分量,例如:

    • vec4(v.rgb, 1)与vec4(v.r, v.g, v.b, 1)是等价的
    • vec4(1) 与vec(1, 1, 1, 1)也是等价的

    参考资料:GLSL语言规范 https://www.khronos.org/files/opengles_shading_language.pdf

    矩阵存储方式

    这些提示都可以在glmatrix的Doc中找到:https://glmatrix.net/docs/mat4.js.html。另外,如果看得仔细我们会发现这个组件也都是用列优先存储矩阵的,WebGL和GLSL中也是列有限存储。如下所示:

    将一个物体移动到一个新的位置,可以用mat4.translate()函数,并且这个函数接受三个参数分别是:一个4×4的输出out,传入的4×4矩阵a,一个1×3的位移矩阵v。

    最简单的矩阵乘法可以使用mat4.multiply,缩放矩阵使用mat4.scale(),调整“看向”的方向使用mat4.lookAt(),正交投影矩阵mat4.ortho()。

    实现光源相机的矩阵变换

    如果我们用透视投影操作,则是这里需要将下面Frustum放缩到一个正交视角的空间,如下图所示:

    但是如果我们使用正交投影,那么就可以保持深度值的线性,使得 Shadow Map 的精度尽可能大。

    // DirectionalLight.js - CalcLightMVP()
    let lightMVP = mat4.create();
    let modelMatrix = mat4.create();
    let viewMatrix = mat4.create();
    let projectionMatrix = mat4.create();
    
    // Model transform
    mat4.translate(modelMatrix, modelMatrix, translate);
    mat4.scale(modelMatrix, modelMatrix, scale);
    
    // View transform
    mat4.lookAt(viewMatrix, this.lightPos, this.focalPoint, this.lightUp);
    
    // Projection transform
    let left = -100.0, right = -left, bottom = -100.0, top = -bottom, 
        near = 0.1, far = 1024.0;  
        // Set these values as per your requirement
    mat4.ortho(projectionMatrix, left, right, bottom, top, near, far);
    
    
    mat4.multiply(lightMVP, projectionMatrix, viewMatrix);
    mat4.multiply(lightMVP, lightMVP, modelMatrix);
    
    return lightMVP;

    2-Pass Shadow 算法

    在实现两趟算法之前,先看看main()函数是怎么调用的。

    // phongFragment.glsl
    void main(void){  
      vec3 shadingPoint = vPositionFromLight.xyz / vPositionFromLight.w;
      shadingPoint = shadingPoint*0.5+0.5;// 归一化至 [0,1]
    
      float visibility = 1.0;
      visibility = useShadowMap(uShadowMap, vec4(shadingPoint, 1.0));
    
      vec3 phongColor = blinnPhong();
    
      gl_FragColor=vec4(phongColor * visibility,1.0);
    }

    那么问题来了,vPositionFromLight是怎么来的?是在顶点着色器中算出来的。

    统一空间坐标

    说人话就是,将场景的顶点的世界坐标转换为光相机的NDC空间对应的新坐标。目的为了渲染主相机的某个Shading Point的阴影时,可以在光源相机的空间中取出所需的深度值。

    vPositionFromLight表示从光源的视角看到的一个点的齐次坐标。这个坐标在光源的正交空间中,其范围是[-w, w]。他是由phongVertex.glsl计算出来的。phongVertex.glsl的作用是处理输入的顶点数据,通过上一章计算的MVP矩阵将一系列顶点转化为裁剪空间坐标。将vPositionFromLight转换到NDC标准空间得到shadingPoint,就可以将shadingPoint里面这些需要做阴影判断的Shading Point传入useShadowMap函数中。附上顶点转换的相关代码:

    // phongVertex.glsl - main()
    vFragPos = (uModelMatrix * vec4(aVertexPosition, 1.0)).xyz;
    vNormal = (uModelMatrix * vec4(aNormalPosition, 0.0)).xyz;
    
    gl_Position = uProjectionMatrix * uViewMatrix * uModelMatrix *
                vec4(aVertexPosition, 1.0);
    
    vTextureCoord = aTextureCoord;
    vPositionFromLight = uLightMVP * vec4(aVertexPosition, 1.0);

    phongVertex.glsl是和phongFragment.glsl一同在loadOBJ.js中被加载的。

    比对深度值

    接下来实现useShadowMap()函数。这个函数的目的是为了确定片段(像素)是否在阴影中。

    texture2D() 是一个GLSL的内置函数,用于对2D纹理进行采样。

    代码框架中的unpack()和pack()函数是为了增加数值精度而设置的。原因如下:

    • 深度信息是一个连续的浮点数,它的范围和精度可能超出了一个8位通道所能提供的。直接将这样的深度值存储在一个8位通道中会导致大量的精度丢失,从而导致阴影效果不正确。因此我们可以充分利用其他的三个通道,也就是将深度值编码到多个通道中。通过分配深度值的不同部分到R, G, B, A四个通道,我们可以用更高的精度来存储深度值。当我们需要使用深度值时,就可以从这四个通道解码出来。

    closestDepthVec是blocker的深度信息,

    最后,closestDepth与currentDepth进行比对,如果blocker(closestDepth)比主相机要渲染的片元的深度值(shadingPoint.z)大,说明当前的Shading Point没有被遮挡,visibility返回1.0。另外为了解决一些阴影痤疮和自遮挡问题,可以将blocker的位置调大一些,即加上EPS。

    // phongFragment.glsl
    float useShadowMap(sampler2D shadowMap, vec4 shadingPoint){
      // Retrieve the closest depth value from the light's perspective using the fragment's position in light space.
      float closestDepth = unpack(texture2D(shadowMap, shadingPoint.xy));
      // Compare the fragment's depth with the closest depth to determine if it's in shadow.
      return (closestDepth + EPS + getBias(.4)> shadingPoint.z) ? 1.0 : 0.0;
    }

    其实目前还是有点问题。我们目前的光源相机并不是万向的,也就是说其照射范围只有一小部分。如果模型在lightCam的范围内,那么画面是完全正确的。

    但是当模型在lightCam的范围外,就不应该参与useShadowMap的计算。但是目前我们并没有完成相关的逻辑。也就是说,如果在lightCam的MVP变换矩阵范围之外的位置在经过计算之后可能会出现意想不到的错误。再看一下灵魂示意图:

    上一节我们在定向光源脚本中定义了zFar、zNear等信息。如下代码所示:

    // DirectionalLight.js - CalcLightMVP()
    let left = -100.0, right = -left, bottom = -100.0, top = -bottom, near = 0.1, far = 1024.0;

    因此,为了解决模型在lightCam范围之外的问题,我们在useShadowMap或在useShadowMap之前的代码中,加入以下逻辑以剔除不在lightCam范围的采样点:

    // phongFragment.glsl - main()
    ...
    if(shadingPoint.x<0.||shadingPoint.x>1.||
       shadingPoint.y<0.||shadingPoint.y>1.){
      visibility=1.;// 光源看不见的地方,因此不会被阴影所覆盖
    }else{
      visibility=useShadowMap(uShadowMap,vec4(shadingPoint,1.));
    }
    ...

    效果如下图所示,左边是做了剔除逻辑的,右边是没有做剔除逻辑的。当202酱移动到lightCam的视锥体边界时,她就直接被截肢了,非常吓人:

    当然了,不完成这一步也没问题。实际上,在开发中我们会使用万向光源,即lightCam是360度全方位的,我们只需要剔除那些在zFar平面之外的点就可以了。

    添加bias改善自遮挡问题

    当我们从光源的视角渲染深度图时,由于浮点数精度的限制,可能会出现误差。因此,当我们在主渲染过程中使用深度图时,可能会看到物体自己的阴影,这称为自遮挡或阴影失真。

    在完成了2-pass渲染之后,我们会在202酱的头发等多处位置发现了这样的阴影痤疮,十分不美观。如下图所示:

    我们理论上可以通过添加bias缓解自遮挡问题。这里我提供一种动态调整bias的方法:

    // phongFragment.glsl
    // 使用bias偏移值优化自遮挡
    float getBias(float ctrl) {
      vec3 lightDir = normalize(uLightPos);
      vec3 normal = normalize(vNormal);
      float m = 200.0 / 2048.0 / 2.0; // 正交矩阵宽高/shadowmap分辨率/2
      float bias = max(m, m * (1.0 - dot(normal, lightDir))) * ctrl;
      return bias;
    }

    首先当光线和法线几乎垂直的时候,极有可能发生自遮挡现象,比如我们的202酱的后脑勺处。因此我们需要获取光线的方向与法线的方向。其中,m表示光源视图下每个像素代表的场景空间的大小。

    最后将 phongFragment.glsl 的 useShadowMap() 改为下文:

    // phongFragment.glsl
    float useShadowMap(sampler2D shadowMap, vec4 shadingPoint){
      ...
      return (closestDepth + EPS + getBias(.3)> shadingPoint.z) ? 1.0 : 0.0;
    }

    效果如下:

    需要注意,较大的bias值可能导致过度矫正带来的阴影缺失结果,较小的值又可能起不到改善痤疮的效果,因此需要多次尝试。

    PCF

    但是ShadowMap的分辨率是有限的。实际游戏中,ShadowMap的分辨率是远远小于分辨率的(原因是性能消耗太大),因此我们需要一种柔化锯齿的方法。PCF方法就是在ShadowMap上为每个像素取其周边的多个像素做平均计算出Shading Point的。

    最初人们想用这个方法软化阴影,但是做到后面发现这个方法可以做到软阴影的效果。

    在使用PCF算法估计阴影比例之前,我们需要准备一组采样点。对于PCF阴影,在移动设备我们只会采用4-8个采样点,而高质量的画面则来到16-32个。在这一节我们使用8个采样点,在这个基础上通过调整生成的样本的参数从而改进画面,减少噪点等等。

    但是,以上不同采样方式对于最终的画面影响其实不算特别大,最影响画面的其实是做PCF时候的阴影贴图大小,也就是shadow map的大小。具体来说,是代码中的textureSize,但是一般而言这一项在项目中都是固定一个值。

    所以我们接下来的思路是先实现PCF,最后再微调采样方式。

    毕竟,premature optimization是大忌。

    实现PCF

    在main()中,修改使用的阴影算法。

    // phongFragment.glsl
    void main(void){  
        ...
        visibility = PCF(uShadowMap, vec4(shadingPoint, 1.0));
        ...
    }

    shadowMap.xy 是用于在阴影贴图上采样的纹理坐标,shadowMap.z 是该像素的深度值。

    采样函数要求我们传入一个Vec2变量作为随机种子,接着会在一个半径为1的圆域内返回随机的点。

    接着将$[0, 1]^2$的uv坐标中分成textureSize份,设置好滤波窗口之后,就在当前的shadingPoint位置附近采样多次,最后统计:

    // phongFragment.glsl
    float PCF(sampler2D shadowMap,vec4 shadingPoint){
      // 采样 采样结果会返回到全局变量 - poissonDisk[]
      poissonDiskSamples(shadingPoint.xy);
    
      float textureSize=256.; // shadow map 的大小, 越大滤波的范围越小
      float filterStride=1.; // 滤波的步长
      float filterRange=1./textureSize*filterStride; // 滤波窗口的范围
      int noShadowCount=0; // 有多少点不在阴影里
      for(int i=0;i<NUM_SAMPLES;i++){
        vec2 sampleCoord=poissonDisk[i]*filterRange+shadingPoint.xy;
        vec4 closestDepthVec=texture2D(shadowMap,sampleCoord);
        float closestDepth=unpack(closestDepthVec);
        float currentDepth=shadingPoint.z;
        if(currentDepth<closestDepth+EPS){
          noShadowCount+=1;
        }
      }
      return float(noShadowCount)/float(NUM_SAMPLES);
    }

    效果如下:

    image-20230805213129275

    poissonDisk采样参数设置

    在作业框架中,我发现这个possionDiskSamples函数并不是真正的泊松圆盘分布?有点奇怪。个人感觉更像是均匀分布在螺旋线上的点。希望读者朋友可以指导一下。我首先先按照框架中的代码分析。


    框架中poissonDiskSamples的相关数学公式

    // phongFragment.glsl
    float ANGLE_STEP = PI2 * float( NUM_RINGS ) / float( NUM_SAMPLES );
    float INV_NUM_SAMPLES = 1.0 / float( NUM_SAMPLES );
    float angle = rand_2to1( randomSeed ) * PI2;
    float radius = INV_NUM_SAMPLES;
    float radiusStep = radius;

    转换极坐标为笛卡尔坐标: 更新规则: 半径变化:

    具体代码如下:

    // phongFragment.glsl
    vec2 poissonDisk[NUM_SAMPLES];
    
    void poissonDiskSamples( const in vec2 randomSeed ) {
      float ANGLE_STEP = PI2 * float( NUM_RINGS ) / float( NUM_SAMPLES );
      float INV_NUM_SAMPLES = 1.0 / float( NUM_SAMPLES );// 把样本放在了一个半径为1的圆域内
    
      float angle = rand_2to1( randomSeed ) * PI2;
      float radius = INV_NUM_SAMPLES;
      float radiusStep = radius;
    
      for( int i = 0; i < NUM_SAMPLES; i ++ ) {
        poissonDisk[i] = vec2( cos( angle ), sin( angle ) ) * pow( radius, 0.75 );
        radius += radiusStep;
        angle += ANGLE_STEP;
      }
    }

    也就是说,以下参数我们可以调整:

    • 半径变化指数的选取

    关于作业框架中为什么要用0.75这个数字,我做了一个比较形象的动画,展示了在泊松采样时每个结果坐标与圆心的距离(半径)的指数在0.2到1.1之间的变化,也就是说,当数值取到0.75以上时,基本可以认为数据重心会更偏向于取圆心的位置。下面动画的代码我放在了 附录1.2 中,读者可以自行编译调试。

    上面是一则视频,若您是PDF版本则需要前往网站查看。

    • 绕圈数NUM_RINGS

    NUM_RINGS与NUM_SAMPLES一起用来计算每个采样点之间的角度差ANGLE_STEP。

    此时可以有如下分析:

    如果NUM_RINGS等于NUM_SAMPLES,那么ANGLE_STEP将等于$2π$,这意味着每次迭代中的角度增量都是一个完整的圆,这显然没有意义。如果NUM_RINGS小于NUM_SAMPLES,那么ANGLE_STEP将小于$2π$,这意味着每次迭代中的角度增量都是一个圆的部分。如果NUM_RINGS大于NUM_SAMPLES,那么ANGLE_STEP将大于$2π$,这意味着每次迭代中的角度增量都超过了一个圆,这可能会导致覆盖和重叠。

    所以在这个代码框架中,当我们的采样数固定时(我这里是8),我们就可以采取决策让采样点更加均匀的分布。

    因此理论上,这里NUM_RINGS直接设置为1就可以了。

    上面是一则视频,若您是PDF版本则需要前往网站查看。

    当采样点分布均匀的情况下,效果还不错:

    如果采样非常不均匀,比如NUM_RINGS等于NUM_SAMPLES的情况,就会出现比较脏的画面:

    得到这些采样点之后,我们还可以对采样点进行权重分配处理。比如在202的课程上闫老师提到可以根据原始像素的距离设置不同的权重,更远的采样点可能会被赋予较低的权重,但项目中不涉及这部分的代码。

    PCSS

    首先找到Shadow Map中任意一处uv坐标的AVG Blocker Depth。

    float findBlocker(sampler2D shadowMap,vec2 uv,float z_shadingPoint){
      float count=0., depth_sum=0., depthOnShadowMap, is_block;
      vec2 nCoords;
      for(int i=0;i<BLOCKER_SEARCH_NUM_SAMPLES;i++){
        nCoords=uv+BLOKER_SIZE*poissonDisk[i];
    
        depthOnShadowMap=unpack(texture2D(shadowMap,nCoords));
        if(abs(depthOnShadowMap) < EPS)depthOnShadowMap=1.;
        // step函数用于比较两个值。
        is_block=step(depthOnShadowMap,z_shadingPoint-EPS);
        count+=is_block;
        depth_sum+=is_block*depthOnShadowMap;
      }
      if(count<EPS)
        return z_shadingPoint;
      return depth_sum/count;
    }

    三步走,这里不再赘述,跟着理论公式走都不太难。

    image-20230731142003749
    float PCSS(sampler2D shadowMap,vec4 shadingPoint){
      poissonDiskSamples(shadingPoint.xy);
      float z_shadingPoint=shadingPoint.z;
      // STEP 1: avgblocker depth
      float avgblockerdep=findBlocker(shadowMap,shadingPoint.xy,z_shadingPoint);
      if(abs(avgblockerdep - z_shadingPoint) <= EPS) // No Blocker
        return 1.;
    
      // STEP 2: penumbra size
      float dBlocker=avgblockerdep,dReceiver=z_shadingPoint-avgblockerdep;
      float wPenumbra=min(LWIDTH*dReceiver/dBlocker,MAX_PENUMBRA);
    
      // STEP 3: filtering
      float _sum=0.,depthOnShadowMap,vis;
      vec2 nCoords;
      for(int i=0;i<NUM_SAMPLES;i++){
        nCoords=shadingPoint.xy+wPenumbra*poissonDisk[i];
    
        depthOnShadowMap=unpack(texture2D(shadowMap,nCoords));
        if(abs(depthOnShadowMap)<1e-5)depthOnShadowMap=1.;
    
        vis=step(z_shadingPoint-EPS,depthOnShadowMap);
        _sum+=vis;
      }
    
      return _sum/float(NUM_SAMPLES);
    }

    框架部分解析

    这一部分属于是在我随便翻阅代码的时候写下的注释,在这里稍微整理了一下。

    loadShader.js

    虽然这个文件中两个函数都是加载glsl文件,但是后者的getShaderString(filename)函数更加简洁高级。这主要体现在前者返回的是Promise对象,后者直接返回文件内容。关于Promise的内容可以看本文的 附录1.3 – JS的Promise简单用法 ,关于async await的内容可以看本文 附录1.4 – async await介绍,关于.then()的用法可以查看 附录1.5 – 关于.then

    专业一点的说法就是,这两个函数提供了不同级别的抽象。前者提供了直接加载文件的原子级别能力,拥有更细粒度的控制,而后者更加简洁与方便。

    添加物体平移效果

    控制器添加到GUI上

    每一帧都要计算阴影的消耗是很大的,这里我手动创建光源控制器,手动调节是否需要每一帧都计算一次阴影。此外,当Light Moveable取消勾选的时候禁止用户改变光源位置:

    勾选上Light Moveable后,出现lightPos选项框:

    具体代码实现:

    // engine.js
    // Add lights
    // light - is open shadow map == true
    let lightPos = [0, 80, 80];
    let focalPoint = [0, 0, 0]; // 定向光聚焦方向(起点是lightPos)
    let lightUp = [0, 1, 0]
    const lightGUI = {// 光源移动控制器,如果不勾选,则不会重新计算阴影。
        LightMoveable: false,
        lightPos: lightPos
    };
    ...
    function createGUI() {
        const gui = new dat.gui.GUI();
        const panelModel = gui.addFolder('Light properties');
        const panelCamera = gui.addFolder("OBJ properties");
        const lightMoveableController = panelModel.add(lightGUI, 'LightMoveable').name("Light Moveable");
        const arrayFolder = panelModel.addFolder('lightPos');
        arrayFolder.add(lightGUI.lightPos, '0').min(-10).max( 10).step(1).name("light Pos X");
        arrayFolder.add(lightGUI.lightPos, '1').min( 70).max( 90).step(1).name("light Pos Y");
        arrayFolder.add(lightGUI.lightPos, '2').min( 70).max( 90).step(1).name("light Pos Z");
        arrayFolder.domElement.style.display = lightGUI.LightMoveable ? '' : 'none';
        lightMoveableController.onChange(function(value) {
            arrayFolder.domElement.style.display = value ? '' : 'none';
        });
    }

    附录1.1

    import numpy as np
    import matplotlib.pyplot as plt
    
    def simulate_poisson_disk_samples(random_seed, num_samples=100, num_rings=2):
        PI2 = 2 * np.pi
        ANGLE_STEP = PI2 * num_rings / num_samples
        INV_NUM_SAMPLES = 1.0 / num_samples
    
        # Initial angle and radius
        angle = random_seed * PI2
        radius = INV_NUM_SAMPLES
        radius_step = radius
    
        x_vals = []
        y_vals = []
    
        for _ in range(num_samples):
            x = np.cos(angle) * pow(radius, 0.1)
            y = np.sin(angle) * pow(radius, 0.1)
    
            x_vals.append(x)
            y_vals.append(y)
    
            radius += radius_step
            angle += ANGLE_STEP
    
        return x_vals, y_vals
    
    plt.figure(figsize=(8, 8))
    
    # Generate and plot the spiral 5 times with different random seeds
    for _ in range(50):
        random_seed = np.random.rand()
        x_vals, y_vals = simulate_poisson_disk_samples(random_seed)
        plt.plot(x_vals, y_vals, '-o', markersize=5, linewidth=2)
    
    plt.title("Poisson Disk Samples")
    plt.axis('on')
    plt.gca().set_aspect('equal', adjustable='box')
    plt.show()

    附录1.2 – 泊松采样点后处理动画代码

    说明:附录1.2 的代码直接基于 附录1.1 修改而成。

    import numpy as np
    import matplotlib.pyplot as plt
    from matplotlib.animation import FuncAnimation
    
    def simulate_poisson_disk_samples_with_exponent(random_seed, exponent, num_samples=100, num_rings=2):
        PI2 = 2 * np.pi
        ANGLE_STEP = PI2 * num_rings / num_samples
        INV_NUM_SAMPLES = 1.0 / num_samples
    
        angle = random_seed * PI2
        radius = INV_NUM_SAMPLES
        radius_step = radius
    
        x_vals = []
        y_vals = []
    
        for _ in range(num_samples):
            x = np.cos(angle) * pow(radius, exponent)
            y = np.sin(angle) * pow(radius, exponent)
            x_vals.append(x)
            y_vals.append(y)
            radius += radius_step
            angle += ANGLE_STEP
    
        return x_vals, y_vals
    
    fig, ax = plt.subplots(figsize=(8, 8))
    ax.axis('on')
    ax.set_xlim(-1, 1)
    ax.set_ylim(-1, 1)
    ax.set_aspect('equal', adjustable='box')
    
    lines = [ax.plot([], [], '-o', markersize=5, linewidth=2)[0] for _ in range(50)]
    exponent = 0.2
    
    def init():
        for line in lines:
            line.set_data([], [])
        return lines
    
    def update(frame):
        global exponent
        exponent += 0.005  # Increment to adjust the exponent
        for line in lines:
            random_seed = np.random.rand()
            x_vals, y_vals = simulate_poisson_disk_samples_with_exponent(random_seed, exponent)
            # plt.title(exponent +"Poisson Disk Samples")
            line.set_data(x_vals, y_vals)
        plt.title(f"{exponent:.3f} Poisson Disk Samples")
        return lines
    
    ani = FuncAnimation(fig, update, frames=180, init_func=init, blit=False)
    
    ani.save('animation.mp4', writer='ffmpeg', fps=12)
    
    # plt.show()

    附录1.3 – JS的Promise简单用法

    关于Promise的用法这里给出一个例子:

    function delay(milliseconds) {
        return new Promise(function(resolve, reject) {
            if (milliseconds < 0) {
                reject('Delay time cannot be negative!');
            } else {
                setTimeout(function() {
                    resolve('Waited for ' + milliseconds + ' milliseconds!');
                }, milliseconds);
            }
        });
    }
    
    // 使用示例
    delay(2000).then(function(message) {
        console.log(message);  // 两秒后输出:"Waited for 2000 milliseconds!"
    }).catch(function(error) {
        console.log('Error: ' + error);
    });
    
    // 错误示例
    delay(-1000).then(function(message) {
        console.log(message);
    }).catch(function(error) {
        console.log('Error: ' + error);  // 立即输出:"Error: Delay time cannot be negative!"
    });

    使用Promise的固定操作是写一个Promise构造函数,这个函数有两个参数(参数也是一个函数):resolve和reject。这样可以构建错误处理的分支,比如在这个案例中,输入的内容不满足需求,则可以调用reject进入拒绝Promise分支。

    比方说现在进入reject分支,reject(XXX)中的XXX就传到了下面then(function(XXX))的XXX中。

    总结一下,Promise是JS中的一个对象,核心价值在于它提供了一种非常优雅统一的方式处理异步操作与链式操作,另外还提供了错误处理的功能。

    1. 通过Promise的.then()方法,你可以确保一个异步操作完成后再执行另一个异步操作。
    2. 通过 .catch() 方法可以处理错误,不需要为每个异步回调设置错误处理。

    附录1.4 – async/await

    async/await是ES8引入的feature,旨在化简使用Promise的步骤。

    直接看例子:

    async function asyncFunction() {
        return "Hello from async function!";
    }
    
    asyncFunction().then(result => console.log(result));  // 输出:Hello from async function!

    函数加上了async之后,会隐式的返回一个Promise对象。

    await 关键字只能在 async 函数内部使用。它会“暂停”函数的执行,直到 Promise 完成(解决或拒绝)。另外你也可以用try/catch捕获reject。

    async function handleAsyncOperation() {
        try {
            const result = await maybeFails();// 
            console.log(result);// 如果 Promise 被解决,这里将输出 "Success!"
        } catch (error) {
            console.error('An error occurred:', error);// 如果 Promise 被拒绝,这里将输出 "An error occurred: Failure!"
        }
    }

    这里的”暂停”指的是暂停了该特定的异步函数,而不是整个应用或JavaScript的事件循环。

    以下是关于await如何工作的简化说明:

    1. 当执行到await关键字时,该异步函数的执行暂停。
    2. 控制权返回给事件循环,允许其他代码(如其他的函数、事件回调等)在当前异步函数之后立即运行。
    3. 一旦await后面的Promise解决(fulfilled)或拒绝(rejected),原先暂停的异步函数继续执行,从暂停的位置恢复,并处理Promise的结果。

    也就是说,虽然你的特定的async函数在逻辑上”暂停”了,JavaScript的主线程并没有被阻塞。其他的事件和函数仍然可以在后台执行。

    举一个例子:

    console.log('Start');
    
    async function demo() {
        console.log('Before await');
        await new Promise(resolve => setTimeout(resolve, 2000));
        console.log('After await');
    }
    
    demo();
    
    console.log('End');

    输出将是:

    Start Before await End (wait for 2 seconds) After await

    希望以上解释可以帮助你理解JS的异步机制。欢迎在评论区讨论,我会尽可能立即回复您。

    附录1.5 关于.then()

    .then() 是在 Promise 对象上定义的,用于处理 Promise 的结果。当你调用 .then(),它不会立即执行,而是在 Promise 解决 (fulfilled) 或拒绝 (rejected) 后执行。

    .then() 的关键点:

    1. 非阻塞:当你调用 .then() 时,代码不会暂停等待 Promise 完成。相反,它会立即返回,并在 Promise 完成时执行 then 里的回调。
    2. 返回新的 Promise:.then() 总是返回一个新的 Promise。这允许你进行链式调用,即一系列的 .then() 调用,每个调用处理前一个 Promise 的结果。
    3. 异步回调:当原始 Promise 解决或拒绝时,.then() 里的回调函数是异步执行的。这意味着它们在事件循环的微任务队列中排队,而不是立即执行。

    举个例子:

    console.log('Start');
    
    const promise = new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve('Promise resolved');
        }, 2000);
    });
    
    promise.then(result => {
        console.log(result);
    });
    
    console.log('End');

    输出会是:

    Start End (wait for 2 seconds) Promise resolved

    附录1.6 – 片段着色器:Uniforms/Textures

    https://webglfundamentals.org/webgl/lessons/zh_cn/webgl-fundamentals.html

    Uniforms 全局变量

    全局变量在一次绘制过程中传递给着色器的值都一样,在下面的一个简单的例子中, 用全局变量给顶点着色器添加了一个偏移量:

    attribute vec4 a_position;uniform vec4 u_offset; void main() {   gl_Position = a_position + u_offset;}

    现在可以把所有顶点偏移一个固定值,首先在初始化时找到全局变量的地址

    var offsetLoc = gl.getUniformLocation(someProgram, "u_offset");

    然后在绘制前设置全局变量

    gl.uniform4fv(offsetLoc, [1, 0, 0, 0]);  // 向右偏移一半屏幕宽度

    要注意的是全局变量属于单个着色程序,如果多个着色程序有同名全局变量,需要找到每个全局变量并设置自己的值。

    Textures 纹理

    在着色器中获取纹理信息,可以先创建一个sampler2D类型全局变量,然后用GLSL方法texture2D 从纹理中提取信息。

    precision mediump float; 
    uniform sampler2D u_texture; 
    void main() {   
        vec2 texcoord = vec2(0.5, 0.5);  // 获取纹理中心的值   
        gl_FragColor = texture2D(u_texture, texcoord);
    }

    从纹理中获取的数据取决于很多设置。 至少要创建并给纹理填充数据,例如

    var tex = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, tex);
    var level = 0;
    var width = 2;
    var height = 1;
    var data = new Uint8Array([
       255, 0, 0, 255,   // 一个红色的像素
       0, 255, 0, 255,   // 一个绿色的像素
    ]);
    gl.texImage2D(gl.TEXTURE_2D, level, gl.RGBA, width, height, 0, gl.RGBA, gl.UNSIGNED_BYTE, data);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);

    在初始化时找到全局变量的地址

    var someSamplerLoc = gl.getUniformLocation(someProgram, "u_texture");

    在渲染的时候WebGL要求纹理必须绑定到一个纹理单元上

    var unit = 5;  // 挑选一个纹理单元
    gl.activeTexture(gl.TEXTURE0 + unit);
    gl.bindTexture(gl.TEXTURE_2D, tex);

    然后告诉着色器你要使用的纹理在那个纹理单元

    gl.uniform1i(someSamplerLoc, unit);

    References

    1. GAMES202
    2. Real-Time Rendering 4th Edition
    3. https://webglfundamentals.org/webgl/lessons/webgl-shaders-and-glsl.html
en_USEN