In a world where artificial intelligence is stepping off our screens and into our physical spaces, a fundamental question arises: how does it remember what it sees? Shawn Shen, co-founder of Memories.ai, argues that visual memory is the missing link for AI in wearables and robotics. His company is now building the infrastructure to make that possible.

Memories.ai has partnered with Nvidia, leveraging the chipmaker's Cosmos-Reason 2 model and Metropolis application framework to advance its core technology. The idea was born from experience. Shen and CTO Ben Zhou previously worked on the AI for Meta's Ray-Ban smart glasses. They realized a glaring gap: users could record video, but the AI couldn't recall or reason about that visual data later.

"AI performs exceptionally in the digital domain," Shen noted. "But for physical applications—devices and robots that see the world—they need visual memories. We're convinced that's the next necessary step."

While memory features for text-based AI, like those in ChatGPT or Google's Gemini, have become common, visual memory presents a distinct engineering challenge. Video is unstructured, vast, and computationally demanding to index and retrieve efficiently.

Since its 2024 launch, Memories.ai has secured $16 million in funding. The team developed a two-part solution: a data infrastructure to embed and index video into a searchable format, and a method to gather training data. They built their own recording hardware, LUCI, to capture the specific video data needed, avoiding consumer-grade recorders ill-suited for machine learning pipelines.

The company released its Large Visual Memory Model (LVMM) last year and has since announced a partnership with Qualcomm to run its models on the chipmaker's processors. Shen reports active collaborations with major wearable makers, though he keeps names confidential. For now, Memories.ai is concentrating on the underlying model and infrastructure, preparing for a market it believes is on the horizon. "The wearables and robotics market will arrive," Shen said. "We're building the layer that will let them see, and remember."

Source: TechCrunch