Smriti-Yi (Memo-ri)

2023

installationLead Artist & Filmmaker (Collaboration with Ying Li, ML Engineer)

Smriti-Yi takes its name from the Sanskrit word for memory. Its central proposition is deceptively simple: what if AI could reconstruct a personal memory as a living visual narrative, in real time, from nothing more than a spoken story? The system listens to a person recount a memory, extracts emotional tone, sentiment, body language, and narrative structure, and generates a dynamic visual scene — including characters, environments, light, and mood — that reflects not the memory as it happened, but as it was felt. The project is grounded in the cognitive science of how memory actually works. Rather than simulating perfect recall, the system is built around the Seven Sins of Memory, the framework established by Harvard psychologist Daniel Schacter, which holds that memory is not a recording but a reconstruction shaped by bias, emotion, time, and desire. Recency bias, rosy retrospection, and the malleability of familiar faces are not problems to be engineered away but conditions to be honored in the design. Visual output shifts in color and sharpness based on sentiment scores, scenes blur where memory detail fades, and characters are placed in plausible environments rather than photographically accurate ones. Faithfulness to the feeling of remembering matters more than faithfulness to the fact of what happened. The production pipeline runs from spoken audio through Whisper transcription, semantic topic modeling via BERTopic, vector storage in ChromaDB, and generative image construction using fine-tuned character and scene models, composited through cinematographic principles of frame, composition, and depth of field, then post-processed for sentiment and memory detail, and finally synchronized with the original voice. The full system was built, tested with community participants, and exhibited at the Harvard Graduate School of Design 2024 exhibition. Three paths for the technology were identified: a collective memory platform, a model trained on film frames for cinematic applications, and a production tool for documentary filmmakers.
Smriti-Yi (Memo-ri) 1
Smriti-Yi (Memo-ri) 2
Smriti-Yi (Memo-ri) 3
Smriti-Yi (Memo-ri) 4
Smriti-Yi (Memo-ri) 5
Smriti-Yi (Memo-ri) 6
Smriti-Yi (Memo-ri) 7
Smriti-Yi (Memo-ri) 8
Smriti-Yi (Memo-ri) 9
Smriti-Yi (Memo-ri) 10
Smriti-Yi (Memo-ri) 11
Smriti-Yi (Memo-ri) 12
Smriti-Yi (Memo-ri) 13
Smriti-Yi (Memo-ri) 14
Smriti-Yi (Memo-ri) 15
Cognitive ScienceSpeculative DesignAIGenerative ArtMemoryHarvard MDE
TECHNOLOGY Whisper, BERTopic, ChromaDB, Stable Diffusion, Stable Video Diffusion, IP-Adapter, Runway, ElevenLabs
RECOGNITION Exhibited at Harvard Graduate School of Design 2024 Exhibition
LINKS

Project Dossier

Open Dossier ↗