The estimation of point normals is vital for accurate 3D meshing of point clouds, as normals give additional information on curvature and enable smoother non-linear mesh interpolation. Thus, a critical step in the computer vision pipeline of the MOCI satellite is to estimate point normals. Though point cloud meshing is not currently implemented int the MOCI computer vision software, it should be considered reasonable future work.
Aside from the point cloud coordinates, the only information needed in the point normal calculation is the camera position (Cx,Cy,Cz) which generated each point. The final result will be the same list of input point coordinates along with the computed normal vector of each point. The problem of determining the normal to a point on the surface is approximated by estimating the tangent plane of the point, and then taking the normal vector to the plane. However, there are two valid normals for estimated tangent plane but only one is suitable for reconstruction. The correct orientation of the normal vector cannot be directly inferred, so an additional subroutine is needed to choose the correct normal vector.
Let a given point cloud be referenced as PC=p1,p2,p3,...,pn where a given point is pi=(xi,yi,zi) and for each point pi∈PC we seek to find the correct normal vector ni=(nx,ny,nz). Also note that each point has an associated camera of the form Ci=(Cx,Cy,Cz).
First, the k nearest neighbors of point pi must be retrieved, let these points be defined as Qi,k=q1,q2,q3,...qk where any neighbor qi∈PC. Then a centroid of the subset Qi,k is calculated with the following equation:
m=1k∑q∈QqNext we seek to produce an approximation of a plane by calculating two vectors v1 and v2 from the given subset of k points. First, let A be a k x 3 matrix built from the centroid being subtracted from each point in the nearest neighbor subset. To find the desired vectors we must perform a singular value decomposition (SVD), seen in the equation below, and notice that the covariance matrix ATA can be diagonalized so that the eigenvectors of the covariance matrix are the columns of vector V (or the rows of vector VT).
A=UΣVTATA=(UΣTUT)(UΣVT)=V(ΣTΣ)VTIn general, the best r-rank approximation of an (n x n) matrix, r<n, is found by diagonalizing the matrix as above, only keeping the first r columns of V (similarly only the first r rows of VT), and only the first r diagonal elements of ΣTΣ (or only first r rows and columns), assuming that the values on the diagonal of Σ were in descending order. More precisely, for randomly ordered diagonal elements (σi)2∈ΣTΣ we keep only the maximum r many of them, along with their corresponding eigenvectors in matrix V. The reason for choosing the maximum valued eigenvalues is that it minimizes the amount of information lost in moving to a lower rank approximation matrix. Therefore, to produce the best approximation of a plane in R3 we would take the two eigenvectors, v1 and v2, of the covariance matrix (which are exactly the columns of V), with the highest corresponding eigenvalues. Those two eigenvectors span the plane we are looking for. Thus, the normal vector ni is simply the cross product of these eigenvectors: ni=v1×v2.
The reason for introducing the SVD is because in computing the covariance matrix ATA we may lose some level of precision in the calculation. By simply factoring matrix A into its singular value decomposition and taking the cross product of the first two rows of VT, we can avoid this problem.
As previously mentioned, there are two viable normals that could be computed with this method, but only one normal is the desired normal. To solve this issue we could simply compute the vector from the camera position C to point pi such that (C−pi)⋅ni<0 holds. If this does not hold then the vector can be flipped by changing the signs of its components. However, because there are likely to be many camera locations, say C=Ci,1,Ci,2,Ci,3,...,Ci,N for all N cameras of a given point pi, a point’s normal can be considered ambiguous if the following is true:
• There exists a ¯C1∈C such that (¯C1−pi)⋅ni<0
• There exists a ¯C1∈C such that (¯C1−pi)⋅ni>0
Such points cannot easily be oriented and thus additional computation is needed; fortunately, in most cases there are very few such normals. When these normals are discovered they are added to a queue of unfinished normals while the rest are placed in a list of correct normals. The algorithm iterates through the queue of ambiguous normals and tries to determine the orientation by looking at the neighboring points of pi. If the neighboring points of pi have already finished normals, then ni is oriented such that it is consistent with the neighboring normals mi by setting ni⋅mi>0 . If the neighboring points do not have already finished normals, then we move pi to the back of the queue, and continue until all normals are finalized.
This is copied from a section of my thesis. If you found this useful to your research please consider using the following bibtex:
@mastersthesis{CalebAdamsMSThesis,
author={Caleb Ashmore Adams},
title={High Performance Computation with Small Satellites and Small Satellite Swarms for 3D Reconstruction},
school={The University of Georgia},
url={http://piepieninja.github.io/research-papers/thesis.pdf},
year=2020,
month=may
}