For our entire lives, we’ve been living through a Well here comes act two of the computing

computing revolution. revolution.

Many of you here in this room have contributed AI and XR are converging,
to it, unlocking radical new ways to interact with
with leaps forward in compute, technology on your terms.
connectivity, mobile technologies, Computers will become more lightweight and
and now AI. personal.
For my part, I've dedicated my entire career to They will share your vantage point,
augmented reality, understand your real-world context
fusing the real world with computing experiences. and have a natural interface that's both simple
I started this journey about 25 years ago for my and conversational.
PhD. Countless people and companies are innovating in
It might sound groundbreaking, but check out this space,
these early prototypes. including many on the TED stage this week.
The technology was really primitive back then, We're excited to contribute to this momentum
but the potential felt limitless with Android XR.
when we were experimenting in that university It's an operating system we're building with
lab. Samsung
What I didn't know at that time that brings XR hardware together with Gemini,
was many of the fundamental innovations for my our AI assistant,
work to augment human intelligence.
would come from AI researchers in a different It can support a broad range of devices,
lab. from headsets to glasses
While I was helping computers see the world, to form factors we haven't even dreamed of yet.
these AI researchers were helping computers OK, let me show you where we’re heading
reason about the world. by inviting my colleague Nishtha to the stage.
Since then, large language models and multimodal Welcome, Nishtha.
reasoning (Applause)
have enabled richer language and image Nishtha Bhatia: Hi.
understanding. Shahram Izadi: These ordinary-looking glasses
These models are now fast enough for live are packed full of technology.
conversations A miniaturized camera and microphones
where the AI can act on your behalf give the AI the ability to see and hear the world.
and, most importantly, with your permission. Speakers let you listen to the AI and play music
And augmented reality and virtual reality has or even take calls.
moved computing And these glasses also have a tiny,
from the rectangular screen to the 360 immersive high-resolution in-lens display
display that's full color
to now even the world itself becoming the display. that I'm holding in my hand.
We now refer to this broad collection of It's very, very small.
experiences as extended reality or XR. These glasses work with your phone streaming
Until now, back and forth,
these innovations have occurred separately and allowing the glasses to be very lightweight
in silos. and access all your phone apps.
And if you're wondering, I'm actually wearing the Let's actually step it up a notch with something
glasses too. we call Memory.
They're actually helping me see all of you in the For a rolling contextual window,
audience the AI remembers what you see
because they have prescription lenses inside without having to be told what to keep track of.
them. NB: So you guys may have noticed
But they’re also displaying my speaker notes for I snuck a peek back at the shelf a moment ago.
me as well. I wasn't paying attention, but let's see if Gemini
(Laughter) was.
For this demo, Hey, did you happen to catch the title of the
you'll see what Nishtha is seeing on the screen white book
behind her. that was on the shelf behind me?
And this is the very first time we're showing G: The white book is “Atomic Habits” by James
these glasses in public. Clear.
So this is going to be a live demo of conceptual NB: That is absolutely right.
hardware, early software. So let's try something harder.
What could go wrong? I keep losing my hotel key card.
Nishtha, best of luck. Do you know where I last left the card?
No pressure. G: The hotel key card is to the right of the music
NB: Amazing. record.
Alright, let’s just make sure that these glasses NB: Great.
are shown SI: For someone as forgetful as me, that's a killer
on the screen behind us. app.
OK. OK, this AI is inherently multimodal.
[Glasses screen off] So it's using visual data and natural language
Awesome. to digest complex queries in real time
(Applause) while remembering your past context.
Now I'm going to launch Gemini. Nishtha, let's see some more.
Gemini: Hello there. NB: Yeah, let’s bring this book back.
NB: Hi, Gemini. So I don't really have time to read it right now.
Can you start us off with something fun and But I wonder if Gemini might be able to help me
creative, summarize what's going on.
and write a haiku for what I'm seeing right now? Like, I don't know.
G: Sure. What does this diagram mean?
Faces all aglow. It looks complex.
Eager minds await the words. G: This diagram is labeled “The Habit Line”
Sparks of thought ignite. and illustrates the process of habit formation.
SI: Some anxious but happy faces as well, yeah. It shows how automaticity increases with
As you can see, the AI sees what Nishtha sees, repetitions
hears what she hears until a habit is formed.
and is responding in real time. NB: OK.
But that’s just table stakes. SI: Now imagine being able to communicate and
I think everyone in this audience has seen demos understand any language.
like this before. The AI can shift its output to speak different
languages (Music)
with dialects and accents that feel human and SI: Perfect.
natural. In a few seconds, the AI recognized the album,
NB: Yeah, let’s go to this sign over here. looked up the tracklist
Can you translate this to English for me? and took action using the phone apps.
G: The sign states: “Private property, no OK, Nishtha, it does look like the demo gods are
trespassing.” with us.
SI: OK, let's prove this is a live demo. Maybe with you more than me.
Does anyone in the audience have a suggestion Let's do one last demo
for a different language to translate to? I know you're keen to give.
Audience: Farsi. NB: Yes, this is my first time in Vancouver,
SI: Farsi. and I love going on walks.
We tried Farsi, it failed last time, but let's try it So why don't you navigate me to a park nearby
again. with views of the ocean?
NB: Do you want to try Farsi? G: OK, I am starting navigation to Lighthouse
SI: Because I do speak Farsi. Park,
It's my mother tongue, so thank you. which has magnificent views of the Pacific Ocean.
NB: Gemini, let’s just give this a shot. Is there anything else I can assist you with?
Can you translate this sign to Farsi for us? NB: Honestly, with these directions and a 3D map,
G: (Speaking Farsi) I should be all set, and hopefully I won’t look like
SI: Great, awesome. a tourist.
It speaks Farsi. Thank you all.
NB: That’s amazing. (Applause)
So, as Shahram mentioned,
you all may have seen translation demos like this
before,
but what's new now is that in addition to just
saying things
in a different language,
I can also speak to Gemini in another language.
I know Hindi, so let's give this a shot.
(Speaks Hindi)
G: (Speaks Hindi)
NB: So Gemini said you all look focused and
excited,
and it has a better accent than I do.
(Laughter)
SI: Alright, now let's see how the AI can connect
the physical world
with your digital content and take action.
NB: Yeah, let’s get some music going in here.
OK, Gemini, why don't you play us a track from
this record?
G: Sure, here’s “Bad Dreams” by Teddy Swims.