Synthetic Memories: Reconstructing the Past with Generative AI

Synthetic memories can emotionally reconnect individuals with their past, enhancing identity and belonging through generative AI.

### Transforming Memories: The Power of Synthetic Visualizations
* 👵 A year ago, I spoke with Carmen, a **90-year-old woman**, about her earliest memory from 1941, when she could only see her father, a **political prisoner**, from a balcony.
* 🖼 Carmen expressed her desire to visualize this memory, leading us to create images using generative AI, which allowed her to **reconnect emotionally with her past**.
* 🎥 The project evolved to generate a video that included her father looking back at her and her mother, evoking **deep emotions and memories long forgotten**.
* 💡 We discovered that "**blurry, undefined images**" work much better than hyper-realistic ones in reconstructing memories, emphasizing **emotional truth** over factual accuracy.
* 🧠 In collaboration with social workers, we explored using synthetic memories in reminiscence therapy for Alzheimer’s patients, showing **promising results in enhancing cognitive engagement**.
* ❤ One patient humorously remarked that the generated image of her husband was inaccurate, highlighting how **love can reshape our perceptions of memory**.
* 🌍 Our initiative aims to **recover lost visual memories** due to aging, displacement, or disasters, and we plan to open a public office in Barcelona for **community memory reconstruction**.
* 📸 Ultimately, as Montserrat Roig said, “If there is an act of love, that is to **remember**,” and our hope is to enable everyone to **cherish and share their memories in meaningful ways**.

A year ago, I was in front of Carmen, a 90-year-old woman from Barcelona. I asked her, Carmen, what is your earliest memory? Now, for someone who has almost seen a century unfold, this can be a difficult question. I was half expecting her to say, Son, my earliest memory is of you asking me this question. (Laughter) But the truth was that it was from 1941. She was six years old, and her mother would pay another family so they could enter their house and go up to their balcony. What was particular about that balcony was that it was facing La Modelo prison. During that time, during the Spanish dictatorship in Spain, it was a political prison, and her father, a doctor for the antifascist front, was a prisoner there. So the only way they could see each other was from that balcony to the window of the prison. And that was her earliest memory of him.

Between bars, through that street. I asked her, Carmen, would you like to have an image of that memory? And she said, Yes, of course. I would love to show to my family what I experienced, the things that I went through so they can remember where we all come from. And this is how we started this experiment, trying to transform a memory into an image, in this case using generative AI. We started by creating a description of the memory, a prompt, to say so. And then Carmen generated tens of images. You could see her going through all of them. But it was not until she saw these two images that something clicked in her. She pointed at the image and said, Yes, this is me and my mother on the balcony. I can see the haircut. It was as if she had found something long lost. From there we did something else. We used that image as a starting point to generate a video,

but this time, not only showing her and her mother, but also showing her father looking back at them from the other side of the street. When she saw it, there was a long silence. And later she told me that it had been a long time since she had seen an image of her father. And as you can see, these are not factual recreations of the past. This feels a bit more like dreams, right? And in fact, they are. They are visualizations based [on] memories with their imperfections and undefinition. That's how we discovered that blurry, undefined images work much better than hyper-realistic ones when we are trying to reconstruct memories. That's why early generative AI models work much better than the ultimate state-of-the-art ones. It's not the factual accuracy that moves us but the emotional truth that we find embedded into some of these systems. And we call this project Synthetic Memories.

It's one of the several projects we are doing at Domestic Data Streamers, a team of researchers, designers, and engineers exploring the impact that generative AI can have in society, going from media literacy to cultural heritage and, of course, memories. Now, memories are the architects of our identity, right? Memories remind us who we are. And visual memories are very important. We all have a picture of someone we love in our wallet, in our phones, in our homes. Visual memories shape our sense of self, and they can shape our sense of belonging to a specific place. They can teach us things from the past. And because of that, they can actually make us understand in a deeper level how we react to things. And like organic memories, which are formed and stored in the human brain, synthetic memories are visual memories from a person's past which have been never documented or lost, and that we can now, using gen-AI,

transform from a text or an oral description into an image. Now, there are many reasons why we can lose visual memories and memories itself, but the most common one is the one that comes with aging and diseases like Alzheimer's or Parkinson's. I myself have lived through that in my family, and I guess many of you have had similar experiences. It's a terrible disease as you not only see the memories fade away, but also the character and the identity of someone that you love. So, two years ago, we started to create these meetups to meet people from very different disciplines, trying to figure out how synthetic memories could be used in different social setups. And then I met David, a social worker in a nursing home. He was taking care of a group of Alzheimer’s patients, and he was using a therapy called reminiscence therapy. Now, reminiscence therapy is a kind of therapy that uses music and old photographs from a person's past

to trigger these very visceral, emotional memories. Think of it as kind of lubricating an old rusty lock, making it much easier to open the doors of certain forgotten rooms that we have in our head. And it has been proven to, for certain people, help very much in the prevention of depression, of cognitive decay, and in some cases, even improve cognitive abilities. So we designed a pilot experiment using synthetic memories within reminiscence therapy. And for over two months, 12 patients did both individual sessions and group therapy. And we could see a direct relation between the level of engagement and their cognitive abilities, pointing out that this could not only be a new way of doing therapy, but also it could be another way to detect early signs of cognitive decay. Now, this image was generated with Nuria, a 96-year-old woman from a nursing home. And she was telling me about one of her favorite memories of her husband was during Sundays,

he used to read the newspaper in the living room. And she will always play this old, jumpy gramophone with some music just to annoy him. When she saw the image, she was so happy her eyes [lit] up, but after a while, she looked at me and she said, But Pau, we have a problem here. This man over here is not my husband. And I said, How come? And she said, half laughing, “Well, my husband was much more ugly.” (Laughter) [Proving] once again that love is not just blind, but also nearsighted. (Laughter) As you can see, this is far from being a cure of Alzheimer’s, but it could be a way of making the journey a bit less daunting. Another patient told me It feels a bit like finding your glasses on your head after searching for them everywhere. It's small but very significant relief. And from a medical perspective,

it could be a new way to temporarily enhance cognitive abilities, making it much easier for family members and caregivers in moments of distress or disconnection. So, the pilot experiment was promising enough to partner up with the AGE-WELL network here in Canada, together with researchers from the University of Toronto and the University of British Columbia. And during this summer, we will be exploring and designing an ethical framework to protect both family members and patients, as well as comparing this therapy to other therapeutic interventions and exploring how we can make it truly accessible for both families and institutions all over the world. But as you can imagine, synthetic memories and this project can go beyond the research on dementia. There are many other reasons why we can lose visual memories. There is the loss that comes from sudden displacement due to war, political persecution, or natural disasters. Situations in which our photo albums, hard drives, phones, diaries can be lost,

damaged or just left behind. Synthetic memories is a way to recover a part of this past heritage that has been lost and that right now is at risk of being forgotten. It's a new way of reconstructing a past that has been hidden to our eyes. Now, soon we will open the first public office for visual memory reconstruction in the city of Barcelona, with the support of the city council. The idea is to make a city-scale experiment, a place where any citizen will be able to generate their own memories, to reconstruct a part of their past together in community, understanding which are the subjectivities of the past from different communities. This image was sent to me not long ago by David, the social worker, and he did it with a dementia patient that for a very long time had not spoken with his daughter. They were in kind of bad terms, and he told David that he was terrified about horses,

but because his daughter loved them, he would bring her to ride every week during her childhood. And he asked David to generate this image of both of them riding in this kind of dreamy landscape. This was not a memory; this never happened. It was a remembrance. It was a way that he wanted to remember his daughter. With his permission, David sent this image to his daughter, who, because of that, after almost ten years without speaking to each other, came back to Barcelona to speak back to him. Afterwards, she told David that when she saw the image, she realized that her father still had good memories of her. An image can work as a witness to the world, we know that, but it can also change the way we see it. We have now tools that can help us understand a time that was not ours. Experiences from others that have never been shown in images and bridge realities that were hidden to our eyes.

Montserrat Roig, a beloved Catalan writer, once wrote (speaking in Spanish), If there is an act of love, that is to remember. And that is truly my hope: that we all get the opportunity to love and be loved in that way. Thank you very much. (Applause)

Can AI Preserve Your Most Precious Memories? | Pau Aleikum Garcia | TED

The TED Talks channel features the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and more. You're welcome to link to or embed these videos, forward them to others and share these ideas with people you know.

TED's videos may be used for non-commercial purposes under a Creative Commons License, Attribution–Non Commercial–No Derivatives (or the CC BY – NC – ND 4.0 International) and in accordance with our TED Talks Usage Policy: https://www.ted.com/about/our-organization/our-policies-terms/ted-talks-usage-policy. For more information on using TED for commercial purposes (e.g. employee learning, in a film or online course), please submit a Media Request at https://media-requests.ted.com

Transforming Memories into Images: Carmen's Story

Carmen, a 90-year-old woman, shared her earliest memory of her father, a political prisoner, from a balcony in 1941.

The experiment to transform her memory into images using generative AI led to a powerful moment of recognition for Carmen.

The project, called Synthetic Memories, discovered that blurry and undefined images are more effective in reconstructing memories than hyper-realistic ones.

Domestic Data Streamers is exploring the role of generative AI in shaping memories and identity.

Visual memories are crucial for our sense of self and belonging, and synthetic memories can now be created from text or oral descriptions using AI.

The project aims to use synthetic memories in various social setups, particularly in supporting Alzheimer's patients through reminiscence therapy.

Using Synthetic Memories in Reminiscence Therapy

Synthetic memories are being used in reminiscence therapy to trigger emotional memories and improve cognitive abilities.

A pilot experiment with 12 patients showed a direct relation between engagement and cognitive improvement.

While not a cure for Alzheimer's, the therapy offers significant relief and a new approach to detecting early signs of cognitive decay.

Enhancing Cognitive Abilities Through Synthetic Memories

The project aims to temporarily enhance cognitive abilities, supporting family members and caregivers in distress.

Partnering with the AGE-WELL network and Canadian universities, the initiative will design an ethical framework and explore accessibility for families and institutions worldwide.

The concept of synthetic memories also addresses the loss of visual memories due to displacement, offering a way to reconstruct and recover a forgotten past, with a public office for visual memory reconstruction opening in Barcelona.

A father, who had a strained relationship with his daughter, used a generated image of them riding together to remember her fondly.

This image prompted the daughter to reconnect with her father after nearly ten years of no contact.

The speaker highlights the transformative power of images in bridging hidden realities and fostering love and connection.