Exploring AI-native interfaces for immersive environments. Generative 3D, spatial understanding, voice-first interaction, and intelligent virtual worlds.
We're building AI that works in 3D space — generating environments, understanding physical context, and responding to gestures and voice. Our primary test platform is Meta Quest, with experiments spanning standalone VR, passthrough AR, and tethered displays.
The interesting problems: How do you ground an LLM in 3D coordinates? Can you generate usable 3D assets fast enough to feel interactive? What happens when you combine Quest's hand tracking and passthrough with on-device inference?
Applications we're exploring: generative design (describe a room, see it built around you), adaptive training simulations, spatial data visualization, and multi-user spaces where AI agents have presence alongside humans.
What we're building, testing, and learning.
Natural language descriptions → complete 3D scenes. Testing various approaches: diffusion-based, retrieval-augmented, and hybrid methods.
'Move that over there' — voice commands with spatial deixis on Meta Quest 3. Combining speech recognition with Quest's hand tracking and eye gaze to understand references.
Using Quest's passthrough cameras to let AI understand your real environment — identify objects, suggest placements, answer questions about the space.
Training environments on Quest that adjust difficulty and scenarios based on user performance. Testing industrial safety scenarios that run entirely on-device.
Things we're still figuring out.
Can generative 3D ever be fast enough for real-time creation on standalone headsets?
How do you ground AI understanding in 3D space — not just 2D projections?
What interaction patterns work best when you have Quest's hand tracking, voice, and eye gaze?
Is immersive AI interaction genuinely better, or just more engaging?
Interested in this research? Have a related problem?
Let's talk →Reach out to us at info@deepklarity.com