I made this video about possible issues with concave polygons in 3D modeling.

Concave Polygons in 3D Modeling

I had this issue with a knife model that I made before for games. One of the quads was kind of shaped like triangle and I didn’t notice. The model itself was pretty well optimised, there was no ngons whatsoever, and all was well. But this one anarchist poly just kept flipping about in 3Ds Max.

Convex and concave (non-convex) polygons and a quad shaped like tri.

Now I know there are those who know much more about geometry in mathematics than me and I’m quite sure I will get some serious feedback. However, what my question really is is this; if you imagine a quad that is shaped like triangle, that one of the sides is actually two sides that share the same angle, is this convex or concave? Especially I’m interested about this from the perspective of a 3D modelling application. Max seems to think about this differently than other apps.

Now, it’s possible to use concave polygons even in final render. I have sometimes used it and even the letters in the flat 3D plane text that’s visible in the video is basically just that. (and the question mark has this actual issue!) But having extreme concave polygons can be a huge pain to deal with in render time.

None of this really matters if you are dealing with subD cage mesh, those meshes can and will get pretty wild and that’s just, well normal, anyway the mesh is subdivided and it’s nothing but a cage mesh after all. And as William Vaughan described in his wonderful tutorial series, four sided triangle can be a great way to optimise sub-D meshes.

Quad that is shaped like a tri. Is this convex or non-convex polygon?

However it’s best to avoid non-convex polygons in traditional meshes.

Please see also my guide to non-manifold geometry and a ways to fix it.