Giving a mesh meaningful part labels

Triangles need meaning before tools can use them

A reconstructed mesh contains triangles. The next tool needs to know which triangles belong to an ear, an eye or the body. A mistaken boundary can send an ear bone into the head, permit fur over an eye, or put a mouth search in the wrong place.

The SEGS path combines image evidence with a 3D segmentation. SAM 3 provides concept masks in rendered pictures. SegViGen lifts a conditioned part map onto the surface. Our surrounding work keeps the cameras registered, assigns declared names, resolves visible votes and carries the result to the tools that consume it.

Those model capabilities are established inputs to the method. Our contribution is the evidence flow and the constraints around them, rather than inventing segmentation itself.

Keep the view and the surface registered

The mesh is rendered from four views, with a face identifier and visibility information behind each picture. Declared part terms are requested for its body plan. Face terms are restricted to the front conditioning view: in the recorded experiments, rear views could put a “nose” mask on an ear.

The front masks form the conditioning map for the 3D lift. The resulting segments are then cut where masks in other views separate them. Names are assigned using evidence from the views that can actually see those surfaces, with area weighting rather than treating a grazing glimpse as equally persuasive.

Small features need special handling. Some visible triangles are too small to own a pixel in a raster image, so the method also samples ray-visible face centres inside a mask. Without that step, a tiny but useful part can lose its vote simply because of image resolution.

A part name is not an exact anatomical fit

Coarse identity and fine geometry answer different questions. A head label tells us where to look for facial features. It does not specify the exact eye opening, ear attachment or centre of a joint inside the body.

The finer region work uses geometry within that semantic permission. Creases and connected surfaces can help establish a socket or attachment. The permission must stay soft enough to reveal disagreement; a coarse mistake should not silently force the wrong fine answer.

Texture seams add another problem. Two vertices can occupy the same point while carrying different texture coordinates. Neighbourhood operations need to recognize that shared surface, or one anatomical part can appear to be many disconnected pieces. Label transfer and welded adjacency address different representations of the same geometry.

Make disagreement inspectable

Connected-island cleanup can remove stray patches, but a cleaner picture is not proof of correct anatomy. The body may legitimately contain more than one piece of a declared part. Conflicting views, a missing mask and an unconstrained region all deserve their own recorded outcome.

The paused hybrid-label experiment belongs to that distinction. Its boundary and consistency measures explored another approach; they did not establish ground-truth accuracy or make it the production path. Agreement with another prediction is useful evidence, not a substitute for knowing the right answer.

A practical check follows one label into a consumer. Does the ear frame attach to the real root? Does the eye package fit the painted opening? Does the mouth candidate remain below the eyes and nose? That is where a plausible segmentation becomes a useful tool—or exposes a mistake.

Declare the anatomy before finding it

The requested parts come from the body plan. This is how a tool can distinguish an absent feature from a feature it failed to find. A body without ears should not receive guessed ear coordinates; a missing ear label on an ear-bearing body is an unresolved input.

The useful human decision is the intended anatomy and, when needed, a correction to the evidence. Rendering, voting and geometric fitting should then carry that choice forward. A generated jelly is a separate case: its construction zones are known at creation, so this recovery step is unnecessary.