합성 기억: 생성적 AI로 과거를 재구성하는 방법

### 기억을 변화시키다: 합성 시각화의 힘
* 👵 1년 전, 저는 **90세의 카르멘 할머니와 1941년, 정치범이었던 아버지를 발코니에서만 볼 수 있었던 기억**에 대해 이야기를 나눴습니다.
* 🖼 카르멘은 이 기억을 시각화하고 싶다고 말했고, **제너레이티브 AI를 사용하여 이미지를 만들어 과거와 감정적으로 다시 연결**할 수 있게 되었습니다.
* 🎥 이 프로젝트는 **아버지가 자신과 어머니를 돌아보는 모습**을 담은 영상으로 발전하여 오랫동안 잊고 있던 깊은 감정과 추억을 불러일으켰습니다.
* 💡 우리는 기억을 재구성하는 데 있어 **'흐릿하고 정의되지 않은 이미지'**가 극사실적인 이미지보다 훨씬 더 효과적이며, 사실적 정확성보다 **정서적 진실**에 중점을 둔다는 사실을 발견했습니다.
* 🧠 사회복지사들과 협력하여 알츠하이머 환자를 위한 회상 치료에 **합성 기억을 사용하는 방법**을 연구한 결과, **인지적 참여를 향상시키는 데 유망한 결과**를 보여주었습니다.
* ❤ 한 환자는 **생성된 남편의 이미지가 부정확**하다고 유머러스하게 말하며 **사랑이 기억에 대한 우리의 인식을 어떻게 변화**시킬 수 있는지를 강조했습니다.
* 🌍 이 이니셔티브는 노화, 이주, 재난 등으로 인해 잃어버린 시각적 기억을 회복하는 것을 목표로 하며, **바르셀로나에 커뮤니티 기억 재건을 위한 공공 사무실**을 열 계획입니다.
* 📸 궁극적으로 "사랑의 행위가 있다면 그것은 기억하는 것"이라는 몬세라트 로익의 말처럼, 모두가 의미 있는 방식으로 **기억을 소중히 간직하고 공유**할 수 있도록 하는 것이 우리의 희망입니다.

합성 기억은 개인을 과거와 정서적으로 다시 연결하고, 생성 AI를 통해 정체성과 소속감을 강화할 수 있습니다.

A year ago, I was in front of Carmen, a 90-year-old woman from Barcelona. I asked her, Carmen, what is your earliest memory? Now, for someone who has almost seen a century unfold, this can be a difficult question. I was half expecting her to say, Son, my earliest memory is of you asking me this question. (Laughter) But the truth was that it was from 1941. She was six years old, and her mother would pay another family so they could enter their house and go up to their balcony. What was particular about that balcony was that it was facing La Modelo prison. During that time, during the Spanish dictatorship in Spain, it was a political prison, and her father, a doctor for the antifascist front, was a prisoner there. So the only way they could see each other was from that balcony to the window of the prison. And that was her earliest memory of him.

Between bars, through that street. I asked her, Carmen, would you like to have an image of that memory? And she said, Yes, of course. I would love to show to my family what I experienced, the things that I went through so they can remember where we all come from. And this is how we started this experiment, trying to transform a memory into an image, in this case using generative AI. We started by creating a description of the memory, a prompt, to say so. And then Carmen generated tens of images. You could see her going through all of them. But it was not until she saw these two images that something clicked in her. She pointed at the image and said, Yes, this is me and my mother on the balcony. I can see the haircut. It was as if she had found something long lost. From there we did something else. We used that image as a starting point to generate a video,

but this time, not only showing her and her mother, but also showing her father looking back at them from the other side of the street. When she saw it, there was a long silence. And later she told me that it had been a long time since she had seen an image of her father. And as you can see, these are not factual recreations of the past. This feels a bit more like dreams, right? And in fact, they are. They are visualizations based [on] memories with their imperfections and undefinition. That's how we discovered that blurry, undefined images work much better than hyper-realistic ones when we are trying to reconstruct memories. That's why early generative AI models work much better than the ultimate state-of-the-art ones. It's not the factual accuracy that moves us but the emotional truth that we find embedded into some of these systems. And we call this project Synthetic Memories.

It's one of the several projects we are doing at Domestic Data Streamers, a team of researchers, designers, and engineers exploring the impact that generative AI can have in society, going from media literacy to cultural heritage and, of course, memories. Now, memories are the architects of our identity, right? Memories remind us who we are. And visual memories are very important. We all have a picture of someone we love in our wallet, in our phones, in our homes. Visual memories shape our sense of self, and they can shape our sense of belonging to a specific place. They can teach us things from the past. And because of that, they can actually make us understand in a deeper level how we react to things. And like organic memories, which are formed and stored in the human brain, synthetic memories are visual memories from a person's past which have been never documented or lost, and that we can now, using gen-AI,

transform from a text or an oral description into an image. Now, there are many reasons why we can lose visual memories and memories itself, but the most common one is the one that comes with aging and diseases like Alzheimer's or Parkinson's. I myself have lived through that in my family, and I guess many of you have had similar experiences. It's a terrible disease as you not only see the memories fade away, but also the character and the identity of someone that you love. So, two years ago, we started to create these meetups to meet people from very different disciplines, trying to figure out how synthetic memories could be used in different social setups. And then I met David, a social worker in a nursing home. He was taking care of a group of Alzheimer’s patients, and he was using a therapy called reminiscence therapy. Now, reminiscence therapy is a kind of therapy that uses music and old photographs from a person's past

to trigger these very visceral, emotional memories. Think of it as kind of lubricating an old rusty lock, making it much easier to open the doors of certain forgotten rooms that we have in our head. And it has been proven to, for certain people, help very much in the prevention of depression, of cognitive decay, and in some cases, even improve cognitive abilities. So we designed a pilot experiment using synthetic memories within reminiscence therapy. And for over two months, 12 patients did both individual sessions and group therapy. And we could see a direct relation between the level of engagement and their cognitive abilities, pointing out that this could not only be a new way of doing therapy, but also it could be another way to detect early signs of cognitive decay. Now, this image was generated with Nuria, a 96-year-old woman from a nursing home. And she was telling me about one of her favorite memories of her husband was during Sundays,

he used to read the newspaper in the living room. And she will always play this old, jumpy gramophone with some music just to annoy him. When she saw the image, she was so happy her eyes [lit] up, but after a while, she looked at me and she said, But Pau, we have a problem here. This man over here is not my husband. And I said, How come? And she said, half laughing, “Well, my husband was much more ugly.” (Laughter) [Proving] once again that love is not just blind, but also nearsighted. (Laughter) As you can see, this is far from being a cure of Alzheimer’s, but it could be a way of making the journey a bit less daunting. Another patient told me It feels a bit like finding your glasses on your head after searching for them everywhere. It's small but very significant relief. And from a medical perspective,

it could be a new way to temporarily enhance cognitive abilities, making it much easier for family members and caregivers in moments of distress or disconnection. So, the pilot experiment was promising enough to partner up with the AGE-WELL network here in Canada, together with researchers from the University of Toronto and the University of British Columbia. And during this summer, we will be exploring and designing an ethical framework to protect both family members and patients, as well as comparing this therapy to other therapeutic interventions and exploring how we can make it truly accessible for both families and institutions all over the world. But as you can imagine, synthetic memories and this project can go beyond the research on dementia. There are many other reasons why we can lose visual memories. There is the loss that comes from sudden displacement due to war, political persecution, or natural disasters. Situations in which our photo albums, hard drives, phones, diaries can be lost,

damaged or just left behind. Synthetic memories is a way to recover a part of this past heritage that has been lost and that right now is at risk of being forgotten. It's a new way of reconstructing a past that has been hidden to our eyes. Now, soon we will open the first public office for visual memory reconstruction in the city of Barcelona, with the support of the city council. The idea is to make a city-scale experiment, a place where any citizen will be able to generate their own memories, to reconstruct a part of their past together in community, understanding which are the subjectivities of the past from different communities. This image was sent to me not long ago by David, the social worker, and he did it with a dementia patient that for a very long time had not spoken with his daughter. They were in kind of bad terms, and he told David that he was terrified about horses,

but because his daughter loved them, he would bring her to ride every week during her childhood. And he asked David to generate this image of both of them riding in this kind of dreamy landscape. This was not a memory; this never happened. It was a remembrance. It was a way that he wanted to remember his daughter. With his permission, David sent this image to his daughter, who, because of that, after almost ten years without speaking to each other, came back to Barcelona to speak back to him. Afterwards, she told David that when she saw the image, she realized that her father still had good memories of her. An image can work as a witness to the world, we know that, but it can also change the way we see it. We have now tools that can help us understand a time that was not ours. Experiences from others that have never been shown in images and bridge realities that were hidden to our eyes.

Montserrat Roig, a beloved Catalan writer, once wrote (speaking in Spanish), If there is an act of love, that is to remember. And that is truly my hope: that we all get the opportunity to love and be loved in that way. Thank you very much. (Applause)

Can AI Preserve Your Most Precious Memories? | Pau Aleikum Garcia | TED

The TED Talks channel features the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and more. You're welcome to link to or embed these videos, forward them to others and share these ideas with people you know.

TED's videos may be used for non-commercial purposes under a Creative Commons License, Attribution–Non Commercial–No Derivatives (or the CC BY – NC – ND 4.0 International) and in accordance with our TED Talks Usage Policy: https://www.ted.com/about/our-organization/our-policies-terms/ted-talks-usage-policy. For more information on using TED for commercial purposes (e.g. employee learning, in a film or online course), please submit a Media Request at https://media-requests.ted.com

기억을 이미지로 변환하기: 카르멘의 이야기

90세 여성인 카르멘은 1941년 발코니에서 정치범이었던 아버지에 대한 첫 기억을 공유했습니다.

제너레이티브 AI를 사용하여 그녀의 기억을 이미지로 변환하는 실험은 카르멘에게 강력한 인식의 순간을 가져다주었습니다.

합성 기억이라고 불리는 이 프로젝트는 초현실적인 이미지보다 흐릿하고 정의되지 않은 이미지가 기억을 재구성하는 데 더 효과적이라는 사실을 발견했습니다.

기억에 대한 제너레이티브 AI의 영향

국내 데이터 스트리머는 기억과 정체성을 형성하는 데 있어 생성형 AI의 역할을 탐구하고 있습니다.

시각적 기억은 자아감과 소속감을 형성하는 데 매우 중요하며, 이제 AI를 사용하여 텍스트나 구두 설명을 통해 합성 기억을 만들 수 있습니다.

이 프로젝트는 특히 회상 치료를 통해 알츠하이머 환자를 지원하는 등 다양한 사회 환경에서 합성 기억을 활용하는 것을 목표로 합니다.

회상 치료에서 합성 기억 활용하기

합성 기억은 회상 요법에서 감정적 기억을 유발하고 인지 능력을 향상시키는 데 사용되고 있습니다.

12명의 환자를 대상으로 한 파일럿 실험에서 참여와 인지 능력 향상 사이에 직접적인 상관관계가 있는 것으로 나타났습니다.

이 치료법은 알츠하이머를 완치하는 것은 아니지만, 인지 기능 저하의 초기 징후를 감지하는 새로운 접근 방식과 상당한 완화 효과를 제공합니다.

합성 기억을 통한 인지 능력 향상

이 프로젝트는 일시적으로 인지 능력을 향상시켜 어려움을 겪고 있는 가족과 간병인을 지원하는 것을 목표로 합니다.

이 이니셔티브는 AGE-WELL 네트워크 및 캐나다 대학과 협력하여 윤리적 프레임워크를 설계하고 전 세계 가족과 기관을 위한 접근성을 모색할 것입니다.

또한 합성 기억이라는 개념은 실향으로 인한 시각적 기억의 상실을 다루며 잊혀진 과거를 재구성하고 회복할 수 있는 방법을 제공하며, 바르셀로나에 시각적 기억 재구성을 위한 공공 사무소가 문을 열었습니다.

딸과 관계가 소원했던 한 아버지는 딸을 다정하게 기억하기 위해 딸과 함께 승마하는 이미지를 생성했습니다.

이 이미지를 통해 딸은 거의 10년 동안 연락이 끊겼던 아버지와 다시 연결될 수 있었습니다.

연사는 숨겨진 현실을 연결하고 사랑과 연결을 촉진하는 이미지의 변혁적인 힘을 강조합니다.