Georgia Institute of Technology School of Interactive Computing

Ayanna Howard: In the late 1990s, Professor Gregory Abowd of Georgia Tech's School of Interactive Computing developed a tool to allow people to collect and reflect upon memories over a long period of time. Motivated by his father's collection of 30 years' worth of video, Gregory wanted to create something that assisted in annotating and search videos to create short memories. Around 2002, he began using this for his own family memories and made a discovery while watching one of the videos. His oldest son, who was then 5 years old and already diagnosed with autism, demonstrated stark differences in behavior and communication between videos at 18 months and others at 26 months. Amazed by what he saw on the videos, Gregory began to consider other more serious applications of this memory-capturing tool. In the coming years, it would become a key research initiative for Gregory and others at Georgia Tech.

Today, we'll talk to Gregory to learn more about the history and future of this space. What are the roles computer scientists can play in detection and intervention? What technologies are currently being used? As a community, what should we understand about autism?

(Music begins)

I'm Ayanna Howard, chair of Georgia Tech's School of Interactive Computing, and this is the Interaction Hour.

(Music fades out)

Professor Abowd is Regents' Professor at Georgia Tech, where he's been faculty since 1994. He is the associate dean of off campus and special initiatives in the College of Computing, and of course a member of the School of Interactive Computing. He has a long list of research honors, such as 10-year impact awards in both 2017 and 2018 for papers at Ubicomp, he's a member of the prestigious SIGCHI Academy, and many more. He's been advisor to 30 -- that's right, 3-0 -- Ph.D. students, many of whom have gone on to incredibly impressive careers of their own.

Thank you for joining us Gregory.

Gregory Abowd: Thank you, it's a pleasure to be here.

Ayanna: So, we have a lot to get into in this episode, but I want to start back with the story that was briefly talked about in our intro. So, you're a parent of two boys on the spectrum, so you come at this from a parent's perspective as well as from the researcher perspective. Walk us through this journey and how you were motivated to begin research in this space.

Gregory: It is an interesting story to share, and I love to share it. It all kind of starts in the year 1998. In retrospect, that was a very important year in my life. My father died that year, my oldest son Aidan was diagnosed on the autism spectrum, and I launched a research initiative with several colleagues at Georgia Tech called the Aware Home Initiative. Those things, in retrospect, kind of all came together and drove me in this direction of doing work in autism and technology.

After my father died, he left behind a 30-year archive of 8mm family movies. My father was known in my very large extended family as the one who always had the camera in front of his face recording lots of family events. I'm from a very large family -- 12 brothers and sisters -- and we had lots of family events in a variety of places, and dad would always record these. But when he died, so did the projector that we used to enjoy these movies. So, I digitized them and with a couple of students built a tool -- and remember this is before the YouTube era -- so, digitized them and built a tool to allow you to browse and annotate the videos so that you could then search on this large archive and pull up things that you wanted to see. Like, show me Christmas episodes from the 1960s. And I used this tool to provide very valuable memories with my brothers and sisters and aunts and uncles, and as a result I took my own family movies -- this was around 2002. This recording effort was part of one of the initiatives within the Aware Home. I put my own family videos in there -- this was 2002, when at that time I had two sons. Aidan, who was 5, and Blaise, who was 2. Blaise was ultimately diagnosed with autism but hadn't been diagnosed at that point.

I saw these recordings of my oldest son Aidan when he was 18 months in his mother's arms acting like an 18-month-old. Looking, smiling, responding to requests to point to body parts. And then saw another clip eight months later when we were on vacation in Hawaii, and he was very different. He was showing sort of stimming behaviors, flapping his hands, kind of rocking, making verbal stimming noises, not speaking, not making eye contact, kind of classic autistic behaviors. These things transpired over a period of eight months and my wife and I really didn't see it happening at the time. It just made me think, wow, I'm spending a lot of time as a researching trying to use technologies to record histories, but for enjoyment. I said, well there may be other more serious applications of that particular work. So, that launched me in the direction of trying to explore any kind of challenge related to autism and technology.

Ayanna: So, this is the early 2000s, and like you mentioned there was no YouTube or Snapchat or Instagram. I'm trying to put myself back into the world of 8mm film, which I can't even fathom what that looked like. Before the adoption of all this technology, what did this really look like at the time? If you think about the projects that existed that you could really do, what did this world look like?

Gregory: What was interesting in the late 90s and early 2000s was that you were starting to see broadband internet going to domestic spaces, so out of the workplace. You wanted to think, well, what would it mean if you had pipes into and out of the home space. What kinds of things could you collect and could you share? And I was very interested in looking at these rich media types, like video. The problem then was it was reasonably easy to record -- we had camcorders then, so that was not difficulty -- but it was very hard to try and summarize and to search and curate these large repositories. What was really valuable about this project, which was called the family video archive, was that it showed that with a little bit of effort and technology you could amplify an individual's ability to browse a large repository, annotate it and essentially search very easily to find the things that you wanted to find later on. And I think the whole power of technology at that point was trying to suggest that with this burgeoning of information -- high-quality, rich data -- the challenge was still how humans were going to be able to manage and understand them appropriately. And, though it took a lot of effort to build this kind of tool, it gave a little window into what would it would mean if consuming and sharing these video types were made much easier. And you started to see that a few years later with things like YouTube, where it became very easy to share a video. You could annotate it as this whole video is about something, but you couldn't do it at finer level of granularity, which was what we were trying to explore.

So, in the early 2000s, it was all about showing the possibility -- even though the tools were very crude still then, the idea was that we would have a world in which it would be easier to manage this amount of data that we would be able to collect for sharing, enjoyment purposes, but also for more serious kind of health and medical applications.

Ayanna: So, I imagine this must have been risky. You mention an effort to build this tool. You have beautiful Ph.D. students, you're bright, excited, young computer scientists, and you want to build something for people? Was that allowed?

Gregory: Well, I'm in the area of human-computer interaction, and it's in our blood that we aren't just interested in technologies for technologies' sake, but in providing use of these technologies for real use. And I had already built a reputation and a career based on taking existing technologies and integrating them together to provide living experiences to see what it's like when we had these kinds of tools of the future and we used them every day. So, that's really part of my whole goal. The risky thing for me was to try to encourage students to want to go in the direction of a very serious application, autism, when they had no experience in that particular space. That was the bigger risk. It wasn't the risk of let's build tools for real people to use, it's let's explore this domain of autism.

Ayanna: So, how has it changed, then, from then and now, given that technology and mobile technologies have come about? What can we do that it's even more?

Gregory: Clearly, technology has advanced quite a bit, and particularly with how it's impacted people impacted by autism. The advent of cheap, mobile devices that not only are easy to use and engaging to use, but have a platform that make it relatively easy for developers to build tools and get them out to a wide audience. That's what's really different today. I would say it's not so much the smartphone or the tablet that's as important, but it's the combination of that with an economy of being able to develop and deploy relatively inexpensive applications. That's really transformed both in a good and a worrying way this particular space of technologies for individuals with special needs.

Ayanna: Okay, so worrying -- I can imagine, you know, you have this pervasive technology. Any developer that wants to make a dime can send it out. So, if I think of myself as a parent and maybe I'm worried or I have a child with a learning disability or I may suspect, how do I actually figure out what's effective or not?

Gregory: Well, you've hit to the heart of the challenge here. Because it's so easy to develop and to deploy things and, you know, it's not hard to get thousands and tens of thousands and millions of people to give you a few dollars for something, it's an easy way to make some money. There's no requirement in the stores for these devices that there be evidence of efficacy of these technologies. So, anyone can put something out there and it makes it very challenging for parents to know what's the right thing for my children.

Ayanna: It sounds like we're then still a little bit in the infancy. The tools may be out there, some good, some are not, but really understanding the impact that technology can have in this space and assisting -- so if you think about that, the technologies that are out there, effective or not, whether there's evidence, what does this future look like? Is it about detecting, or is it about really providing assistance, like helping parents navigate this space?

Gregory: Well, it's both. You hit on an important distinction between the kinds of two different ways of using technology. You might use it to help one come to a diagnosis or a screening to alert to the likelihood that there's an individual that has special needs or some kind of delay or in other ways are different than our understanding of typical development. Tools that are there for that screening or diagnostic purpose is one class. The other one, and this is the one I think that's more challenging, is interventions. You have an identified need that you want to address and there are hundreds, thousands of opportunities for you to find a tool that claims to be useful for that particular purpose, whether that's speech or understanding of language or the ability to point effectively to objects, all these kinds of skills, or how they interview for a job, for example, if you have social anxiety. There are tools to help you do all of those things.

Ayanna: So, are there any gold standards with these tools? I mean, is there something that I can say that for this intervention X, this is the best tool that I can use?

Gregory: Well, there are research communities whose whole purpose is to be able to assess these kinds of uses of technologies, but there's this disconnect between what researchers establish effectiveness for and what is available to families. Families don't want to wait the 5-10 years for there to be a rigorous study in evidence for things if they feel something will work. Also, there's this struggle in some areas -- there's lots of clinical evidence from the perspective of individuals who treat or provide support for interventions, they know what has worked for them and their clients. Although, they're not strictly speaking doing research along those lines, they use their clinical experience to promote or advance particular kinds of interventions. It's very hard for a parent to try to figure out, 'Should I wait for the research-grade research? Should I take advice from someone who's been working in the field for 20 years? Or, should I look at reviews online in an app store?'

And the parents, also, don't have a lot of time to make these decisions. Because you're given the message that the earlier you understand there's a delay and the more proactive you are about addressing that as an intervention, the more important because we're given the idea that there's a short window of time. Before a child reaches school age, you want them to have certain functional communication capabilities. Well, that's 4 or 5 years old. If you're not diagnosed until you're 2.5 or 3, you've got 12-18 months you think to make progress.

Ayanna: So, what's the average age right now of detection?

Gregory: So, in the U.S., the average age is still unfortunately about 4.5-5 years, which basically means that by the time they get to kindergarten or pre-K, that's when an official kind of diagnosis occurs even though there may be a suspicion of a delay prior to that. In the research community, we are very confident that by the age of 2, there are gold standards for being able to establish a criterion on the autism spectrum, I'm talking about there. There's work to move that even earlier. So, we can know much sooner than 4.5-5 years.

Ayanna: So, one of the benefits of this mobile technology in pervasive is that getting this evidence-based research onto platforms that then parents can have access to.

Gregory: Yes, and there's a lot of work in that area of using technology. I've had a small role to play in some of that, where a lot of the bottleneck often is waiting to get an in-person meeting with someone with a diagnostic certification to make a decision. One way you might speed that up is by allowing parents to record the right kind of video evidence of a child's behavior and then forward that to a diagnostician, who would look at that instead of having an in-person interaction with a child. We've demonstrated on a small scale that that is an effective way to do a diagnosis of autism, particularly for the easy cases -- ones who clearly are not and ones who clearly are on the spectrum. So, you can focus your efforts for in-person interaction with individuals who kind of are in that gray area in between.

Ayanna: This is interesting. You think about where this started, family video, your dad just saying, 'Hey, I need to look at my brothers and sisters on film.' Very personal, but more family -- all the way to being able to give a tool to parents that can use these videos that are transcribed using technology for hopefully getting to see a clinician or diagnosis. Imagine how the world shapes us in these directions.

Gregory: Yeah, I mean, I've always felt that it's good to have your personal experiences kind of inspire what you do, and this was -- I don't know the right way to say it -- but a fortunate accident that I was able to stumble upon this particular way to record and observe a relevant change that is obvious over the periods of months, but day-to-day is not so obvious.

Ayanna: So, what's the most important thing that you've learned in autism awareness? And this is through your own experiences, your own research, research of others around you? Like, what is kind of the most important thing that you want our audience to take away?

Gregory: That's a pretty easy question for me to answer, what I think is most important, and it's not about the technology. It's about understanding these individuals. To understand that it is a spectrum, so it manifests itself in a wide variety of ways. And for many of these individuals, we often underestimate -- grossly underestimate -- their capabilities and their desires and ability to be like the rest of us and to integrate with the rest of society. My oldest son is a non-speaking individual, who over time has developed the ability to communicate by typing very simply with a single finger, and what he demonstrates is that at 21 years old, he's one of the smartest individuals I know. He has understood everything that has happened around his life, and he has not been able to communicate that understanding. Silly kinds of measures like traditional IQ tests put him at an IQ of around 40. Last night, I was doing homework with him, and he was expanding and contracting logarithms in his head faster than I could imagine doing it. And it's all because he's finally developed the ability to do the motor activity of moving his hand to say what's in his head. He's just one of a number of individuals that have tremendous potential, and we need to presume the competence of these individuals and appreciate their neurological differences, not that they have a disorder or even necessarily a disability. They just have a different way of perceiving and working through the world, and we need to figure out the best way to bring their strengths out. Because they're there.

Ayanna: I thank you for this personal story. And so before we go, I know that besides being a parent, you are also an advocate with regards to autism. Can you tell us just a little bit about what you do and how people out there can possibly get involved.

Gregory: Sure. As a result of working with many of my respected colleagues around Atlanta, in 2008 I started an informal group called the Atlanta Autism Consortium, which is intended to bring stakeholders from all different walks of life linked to autism. Today, this is a non-profit with about 150 paying members. You can look us up on Facebook, the Atlanta Autism Consortium. We hold monthly events, this month in particular because it's Autism Awareness Month. We're holding a number of social hours around Atlanta at various restaurants, a chance for people to get to come and find other families, individuals, researchers, teachers, clinicians. It's about bringing the Atlanta community together, because as a community of people we can identify and address a lot of these challenges more effectively if we just openly and honestly address them.

Ayanna: This is great. I learned quite a bit. I hope our audience has too. So, we appreciate Gregory for joining in this Interaction Hour and encourage our audience out there to find out more about the Atlanta Autism Consortium online. Be sure to head to our website and social media channels this month to see additional features on the research around autism and computing that takes place at the School of Interactive Computing.

You find us online at ic.gatech.edu and on Twitter and Facebook at @ICatGT.

Thanks for listening.