Triplanar texture projection

[nblauch]

Description

Add triplanar texture projection for meshes without UV coordinates, equivalent to OmniPBR's project_uvw mode. This enables texture rendering on meshes that rely on runtime UV projection rather than baked
UV maps.

When a mesh has a texture but no UVs, sample_texture() now calls sample_texture_triplanar() which computes the face normal from triangle vertices, samples the texture from 3 axis-aligned planes (YZ, XZ,
XY), and blends using abs(face_normal) as weights. This produces seamless results on arbitrarily oriented surfaces.

Also includes a prerequisite change to preserve mesh.texture during USD import when UVs are missing (previously the texture was stripped), and sets mesh.color to white for textured meshes so the texture
renders at full brightness.

Checklist

• New or existing tests cover these changes
• The documentation is up to date with these changes
• CHANGELOG.md has been updated (if user-facing change)

Test plan

Verified end-to-end with a dextrah Kuka-Allegro manipulation scene (64 envs, Newton Warp renderer) containing heterogeneous grasp objects — a mix of meshes with baked UVs and meshes without UVs that rely on
project_uvw=True:

• Objects with UVs: continue to use existing sample_texture_mesh path (no change)
• Objects without UVs: now render with triplanar-projected textures instead of flat white/palette color
• No visible seams at face orientation transitions on 3D objects (cubes, cylinders, irregular shapes)

New feature / API change

import newton
import numpy as np
from newton import ModelBuilder, Mesh

# Create a mesh with a texture but no UVs
vertices = np.array([[0, 0, 0], [1, 0, 0], [0, 1, 0], [0, 0, 1]], dtype=np.float32)
indices = np.array([0, 1, 2, 0, 2, 3, 0, 3, 1, 1, 3, 2], dtype=np.int32)
mesh = Mesh(vertices, indices, compute_inertia=False)
mesh.texture = "path/to/texture.png"  # texture without UVs — previously stripped

builder = ModelBuilder()
builder.add_shape_mesh(body=-1, mesh=mesh)
model = builder.finalize(device="cuda:0")

sensor = newton.sensors.SensorTiledCamera(model)
sensor.render_config.enable_textures = True
# Rendering now uses triplanar projection instead of returning white

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added triplanar texture projection for meshes without UV coordinates.

* **Bug Fixes**
* Meshes with textures but missing UV data now display correctly instead of being ignored; textures now render using projected coordinates.
* Improved material and texture handling during USD import with refined fallback logic.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

[linux-foundation-easycla]

• ❌ - login: @nblauch / name: nblauch . The commit (476cbe3, 8871a9f) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.
• ❌ - login: @nblauch / name: Nicholas Blauch . The commit (5f43ef1) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

[coderabbitai]

📝 Walkthrough

Walkthrough

The changes introduce triplanar texture sampling for meshes lacking UV coordinates and update the material import logic to reflect this capability. The texturing system now computes texture coordinates from object-space position and surface normals when UV data is unavailable.

Changes

Cohort / File(s)	Summary
Triplanar Texture Sampling newton/_src/sensors/warp_raytrace/textures.py	Added new sample_texture_triplanar() function that derives blending weights from surface normal components and samples texture from three axis-aligned planar projections. Updated sample_texture() control flow to invoke triplanar sampling when mesh UVs are missing, replacing the default return behavior.
Material Import Logic newton/_src/utils/import_usd.py	Updated fallback warning message to indicate projected UVs will be used for textures without UV coordinates instead of being ignored. Removed texture-clearing behavior and changed color assignment precedence: texture presence forces white color; otherwise, authored material color is applied.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Triplanar texture projection' directly summarizes the main change: adding triplanar texture projection for meshes without UV coordinates.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

newton/_src/sensors/warp_raytrace/textures.py (1)

113-126: ⚠️ Potential issue | 🔴 Critical

Out-of-bounds array access in @wp.func kernel—requires guard restructuring.

The condition on line 119 is unsafe:

if mesh_data_index < 0 or mesh_data[mesh_data_index].uvs.shape[0] == 0:

NVIDIA Warp does not support short-circuit evaluation for boolean operators in @wp.func kernels (per official documentation and GitHub issue #1329). This means both conditions in the or expression are always evaluated, even when mesh_data_index < 0. Accessing mesh_data[mesh_data_index] with a negative index causes out-of-bounds access.

Restructure the guard to check mesh_data_index >= 0 independently:

if mesh_data_index >= 0 and mesh_data[mesh_data_index].uvs.shape[0] > 0:
    # Use UV-based sampling
else:
    # Use triplanar (covers both negative index and zero-length UVs cases)

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@newton/_src/sensors/warp_raytrace/textures.py` around lines 113 - 126, The
current guard in the `@wp.func` kernel uses "if mesh_data_index < 0 or
mesh_data[mesh_data_index].uvs.shape[0] == 0" which can still index mesh_data
when mesh_data_index is negative (Warp does not short-circuit booleans); change
the logic to first test mesh_data_index >= 0 and only then inspect
mesh_data[mesh_data_index].uvs.shape to decide which sampler to call — e.g., if
mesh_data_index >= 0 and mesh_data[mesh_data_index].uvs.shape[0] > 0 then call
sample_texture_mesh(bary_u, bary_v, face_id, mesh_id,
mesh_data[mesh_data_index], texture_data[texture_index]) else call
sample_texture_triplanar(hit_point, shape_transform, mesh_id, face_id,
texture_data[texture_index]) — so mesh_data is never indexed when
mesh_data_index is negative.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@newton/_src/sensors/warp_raytrace/textures.py`:
- Around line 113-126: The current guard in the `@wp.func` kernel uses "if
mesh_data_index < 0 or mesh_data[mesh_data_index].uvs.shape[0] == 0" which can
still index mesh_data when mesh_data_index is negative (Warp does not
short-circuit booleans); change the logic to first test mesh_data_index >= 0 and
only then inspect mesh_data[mesh_data_index].uvs.shape to decide which sampler
to call — e.g., if mesh_data_index >= 0 and
mesh_data[mesh_data_index].uvs.shape[0] > 0 then call
sample_texture_mesh(bary_u, bary_v, face_id, mesh_id,
mesh_data[mesh_data_index], texture_data[texture_index]) else call
sample_texture_triplanar(hit_point, shape_transform, mesh_id, face_id,
texture_data[texture_index]) — so mesh_data is never indexed when
mesh_data_index is negative.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 2c6d023e-85ce-4e06-b9b9-eff6777ee85b

📥 Commits

Reviewing files that changed from the base of the PR and between 8532e01 and 5f43ef1.

📒 Files selected for processing (2)

• newton/_src/sensors/warp_raytrace/textures.py
• newton/_src/utils/import_usd.py

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ All tests successful. No failed tests found.

📢 Thoughts on this report? Let us know!