## Fast wire-frame rendering with OpenCV

Lets say you have mesh data in the typical format, triangulated, vertex buffer and index buffer. E. g. something like

>>> vertices

[[[ 46.27500153  19.2329998   48.5       ]]

[[  7.12050009  15.28199959  59.59049988]]

[[ 32.70849991  29.56100082  45.72949982]]

...,

>>> indices

[[1068 1646 1577]
[1057  908  938]
[ 420 1175  237]
..., 

Typically you would need to feed it into OpenGL to get an image out of it. However, there are occasions when setting up OpenGL would be too much hassle or when you deliberately want to render on the CPU.

In this case we can use the OpenCV to do the rendering in two function calls as:

img = np.full((720, 1280, 3), 64, dtype=np.uint8)

pts2d = cv2.projectPoints(vertices, rot, trans, K, None)[0].astype(int)
cv2.polylines(img, pts2d[indices], True, (255, 255, 255), 1, cv2.LINE_AA)

See the documentation of cv2.projectPoints for the meaning of the parameters.

Note how we only project each vertex once and only apply the mesh topology afterwards. Here, we just use the numpy advanced indexing as pts2d[indices] to perform the expansion.

This is pretty fast as well. The code above only takes about 9ms on my machine.

In case you want filled polygons, this is pretty easy as well

for face in indices:
cv2.fillConvexPoly(img, pts2d[face], (64, 64, 192))

However, as we need to a python loop in this case and also have quite some overdraw, it is considerable slower at 20ms.

Of course you can also combine both to get an image like in the post title.

From here on you can continue to go crazy and compute face normals to do culling and shading.

## OGRECave 1.10 release

The 1.10.0 release of the OGRECave fork was just created. This means that the code is considered stable enough for general usage and the current interfaces will be supported in subsequent patch releases (i.e. 1.10.1, 1.10.2 …).

This release represents more than 3 years of work from various contributors when compared to the previous 1.9 release. At the time of writing it contains all commits from the bitbucket version as well as many fork specific features and fixes.

If you are reading about the fork for the first time and wonder why it was created, see this blog post. For a comparison between the github and bitbucket version see this log.

For a general overview of the 1.10 features when compared to 1.9, see the OGRECave 1.10 release notes.

The highlights probably are:

• upstream Python bindings as an component
• improved GL3+/ GLES2 renderers
• A new HLMS Component implementing physically based shading
• SDL2 based input handling
• Bites Component for rapid prototyping of applications
• Emscripten platform support

For further information see the github page of the fork.

## Introducing the OGRE fork on GitHub

in this post I want to introduce the OGRE fork on github. The goal of the fork is to provide a stable and reliable OGRE 1.x series while at the same time modernizing parts under the hood updates.

The idea behind this is that there are many existing 1.x codebases, actually a whole 1.x ecosystem, that can be modernized that way.
The last release of the 1.x series was over 2 years ago, so using the current 1.10 branch already gives a lot of improvements.

## Learning Modern 3D Graphics Programming

one of the best resources to learn modern OpenGL and the one which helped me quite a lot is the Book at www.arcsynthesis.org/gltut/ – or lets better say was. Unfortunately the domain expired so the content is no longer reachable.

Luckily the Book was designed as an open source project and the code to generate the website is still available at Bitbucket. Unfortunately this repository does not seem to be actively maintained any more.

Therefore I set out to make the Book to be available again using Github Pages. You can find the results here:

https://paroj.github.io/gltut/

However I did not simply mirror the pages, but also improved it at several places. So what has changed so far?

## OpenGL Matrices – the missing bits

While generally the available documentation on how the OpenGL matrices work is quite good, there are some missing bits. Although not necessary for your everyday rendering, they give one some insight on how rasterization in general and OpenGL in special works.

# W coordinate after perspective divide

After conversion to normalized device coordinates(ndc) one might think each vertex looks like

$$\vec{v}_{ndc} = \frac{1}{w} \begin{pmatrix} x \\ y \\ z \\ w \end{pmatrix} = \begin{pmatrix} \tfrac{x}{w} \\ \tfrac{y}{w} \\ \tfrac{z}{w} \\ 1 \end{pmatrix}$$

however it actually looks like

$$\vec{v}_{ndc} = \begin{pmatrix} \tfrac{x}{w} \\ \tfrac{y}{w} \\ \tfrac{z}{w} \\ \tfrac{1}{w} \end{pmatrix}$$

the $w$ coordinate is not divided by itself, but is inverted instead. This is done because the interpolation between vertices still needs to take place and for perspective correct interpolation one needs the camera space depth $z = -w_{cam}$.

\begin{aligned} \vec{v}_{\alpha} &= \frac{(1-\alpha) \tfrac{\vec{v}_0}{-z_0} + \alpha \tfrac{\vec{v}_1}{-z_1}}{(1 - \alpha)\tfrac{1}{-z_0} + \alpha \tfrac{1}{-z_1}} \\[1.5em] &= \frac{(1-\alpha)\vec{v}_0 w_{0_{ndc}} + \alpha\vec{v}_1 w_{1_{ndc}}}{(1-\alpha) w_{0_{ndc}} + \alpha w_{1_{ndc}}} \end{aligned}

instead of dividing by $-z$ we can multiply with $w_{ndc}$ as multiplication is faster than division.

Note that for brevity the given formula assumes a scanline based rasterizer as it interpolates only between two vertices. The general approach is to use barycentric coordinates to interpolate between all three vertices simultaneously.

# Row major or column major

Even though even Wikipedia says OpenGL is column major, it is actually storage agnostic. However by default it interprets your 16 element array as:

$$\begin{bmatrix}m_0 & m_4 & m_8 & m_{12}\\ m_1 & m_5& m_9 & m_{13}\\ m_2 & m_6 & m_{10} & m_{14}\\ m_3 & m_7 & m_{11} & m_{15}\end{bmatrix}$$

Yet most OpenGL functions dealing with matrices offer a transpose parameter which you can use to specify the used order. For a comparison of storage orders see the Eigen documentation.

Notably however, GLSL matrices do neither follow C nor mathematical notation; the mat2x4 M type has 2 columns and 4 rows and thus $M \in \mathbb{R}^{4 \times 2}$ mathematically.
Consequently though – albeit breaking with the C notation – M[0] will return the first column (vec4).

Now, if you use the transpose parameter mentioned before, prepare to think hard about the data you are actually getting in the Shader.

## How to draw a line interpolating 2 colors with opencv

The built-in opencv line() drawing function allows to draw a variety of lines. Unfortunately it does not allow drawing a gradient line interpolating the colors at its start and end.

However implementing this on our own is quite easy:

using namespace cv;

void line2(Mat& img, const Point& start, const Point& end,
const Scalar& c1,   const Scalar& c2) {
LineIterator iter(img, start, end, LINE_8);

for (int i = 0; i < iter.count; i++, iter++) {
double alpha = double(i) / iter.count;
// note: using img.at<T>(iter.pos()) is faster, but
// then you have to deal with mat type and channel number yourself
img(Rect(iter.pos(), Size(1, 1))) = c1 * (1.0 - alpha) + c2 * alpha;
}
}