Why Ceo Matt Garman Is Willing To Bet Aws On Ai

Today, I’m talking pinch Matt Garman, nan CEO of Amazon Web Services, aliases AWS. Matt took complete arsenic CEO past June — you mightiness callback that we had his predecessor, Adam Selipsky, connected nan show conscionable complete a twelvemonth ago. That makes this conception terrific Decoder bait, since I emotion proceeding really caller CEOs find what to alteration and what to support erstwhile they’ve settled into their role.

Matt has a really absorbing position for that benignant of reside since he’s been astatine AWS for 20 years — he started astatine Amazon arsenic an intern and was AWS’s original merchandise manager. He’s now nan 3rd CEO successful conscionable 5 years, and I really wanted to understand his wide position of immoderate AWS and wherever it sits incorrect an manufacture that he had a pivotal domiciled successful creating.

You’ll comprehend Matt opportunity that astir companies are still hardly successful nan cloud, and that opportunity remains monolithic for AWS, moreover though it’s been nan marketplace leader for years. If you’re a merchandise caput aliases an aspiring merchandise manager, you’ll drawback Matt talking astir these things precisely for illustration nan merchandise caput he was from nan start, only now pinch a wide position from nan CEO chair.

But conscionable acquiring caller customers isn’t nan crippled immoderate longer: for illustration each unreality provider, Amazon is reorienting its afloat computing infrastructure for a world of generative AI. That includes overmuch than $8 cardinal successful backing for Anthropic, a immense push to build its ain AI chips to compete pinch Nvidia, and moreover atomic powerfulness investments arsenic nan powerfulness petition for AI continues to grow. After Matt and I talked earlier nan holidays, AWS announced an $11 cardinal finance to turn its accusation halfway operations successful Georgia. 

Matt’s position connected AI arsenic a exertion and a business is refreshingly chopped from his peers, including those overmuch incentivized to hype up nan capabilities of AI models and chatbots. I really pushed Matt astir Sam Altman’s state that we’re adjacent to AGI and connected nan precipice of machines that tin do tasks immoderate value could do. I too wanted to cognize erstwhile immoderate of this is going to commencement returning — aliases moreover justifying — nan tens of billions of dollars of investments going into it.

His answers connected immoderate subjects were beautiful candid, and it’s clear Matt and Amazon are acold overmuch focused connected really AI exertion turns into existent products and services that customers want to usage and small astir what Matt calls “puffery successful nan press.” 

One connection earlier we commencement — we recorded this conception conscionable earlier nan holidays, truthful I asked Matt astir Netflix, 1 of AWS’s biggest customers, and whether it would clasp up while streaming unrecorded events, peculiarly nan NFL games it streamed connected Christmas. Turns out, Netflix did conscionable bully pinch those, but nan answers coming were beautiful interesting. Matt still checks successful connected his ample customers, moreover arsenic CEO.

Okay, AWS CEO Matt Garman. Here we go.

This transcript has been lightly edited for magnitude and clarity. 

Matt Garman, you’re nan CEO of Amazon Web Services (AWS). Welcome to Decoder.

Thanks for having me.

I americium very excited to talk to you. You’re for illustration a cleanable Decoder guest. You are, I believe, nan first merchandise caput astatine AWS, you started arsenic an intern and now you’re nan CEO. We personification a batch of listeners who want to beryllium connected that journey, truthful there’s tons to talk to you astir conscionable successful that. 

You’re too nan caller CEO. We had your predecessor, Adam Selipsky, on nan show conscionable a mini complete a twelvemonth ago. You’re astir six months connected nan business now. So, there’s a batch of Decoder worldly successful location — really you’re changing nan connection and really you’re reasoning astir it. And then, obviously, we’re going to talk astir AI. It’s going to happen. I dream you’re caller for it.

I’m caller for it. Shoot, occurrence away. I’m happy to spell wherever you want.

All right. But I really want to commencement pinch a very hot-button, profoundly arguable topic. Are you ready?

Great. Fire away.

Okay, it’s Jake Paul. I want to commencement pinch Jake Paul. My knowing is Netflix is nan prototypical AWS customer, right? They started connected AWS, they made a ample liking connected AWS. They’re still nan customer, right? They haven’t adjacent AWS?

Yeah, Netflix is simply a awesome customer of ours. Absolutely.

They conscionable had nan live watercourse of Jake Paul fighting Mike Tyson. You tin deliberation point you want astir those 2 men fighting each other.

I was hoping Mike would win, honestly.

So was I.

I deliberation astir were, but that’s okay. It was nosy to spot him retired there.

You’ve conscionable group disconnected a cardinal overmuch conspiracy theories astir this fight. Anyhow, I told you it was controversial. All right, but nan watercourse was beautiful glitchy. I deliberation everybody agrees connected that. When I watched it, it degraded to 360p astatine immoderate constituent for me. Netflix CEO Ted Sarandos was conscionable connected style astatine a conference. Netflix said nan petition is 108 cardinal group globally, and here’s what Ted said astir that stream: “We were stressing nan limits of nan nett itself that night. We had a powerfulness room up successful Silicon Valley that was re-engineering nan afloat nett to support it up during this conflict because of nan unprecedented petition that was happening.” 

You’re nan CEO of AWS, you’re nan internet. Did they personification to re-engineer nan nett for nan Jake Paul fight?

You’ve sewage to inquire Ted astir that. I deliberation wherever they were stressed astir nan [content proscription network] they run, and you tin inquire Ted astir that too. Netflix has its ain homegrown CDN that it uses, and that’s nan information that I deliberation was stressed. I don’t cognize nan specifications of precisely wherever they were moving into barriers, but it wasn’t successful nan AWS infrastructure, it was successful nan Netflix-controlled information of their structure.

Yeah, their CDN is really fancy, right? They’ve sewage boxes and ISPs and everything. I was conscionable funny because what we’re astir to talk about, successful a immense way, is really providers for illustration AWS tin meet nan expanding petition for compute everyplace and past get it to nan group who petition it. And it feels for illustration astir group successful 2024 return video streaming for granted, but it’s still beautiful hard.

It is. And I deliberation successful particular, location are a mates of things astir that that are challenging, right? By nan way, it’s a ace difficult constituent that they did. Number one, it’s their first clip doing a big, scaled unrecorded watercourse for illustration that. The first clip is really what’s hard. Other group personification done that before. We’ll watercourse Thursday Night Football and different places for illustration that that personification figured retired really to do things astatine that scale, but it’s not nan first time. So, I’m judge that nan adjacent clip — I deliberation they personification a Christmas clip crippled — they’ll astir apt activity retired immoderate of those kinks and fig that information out.

The first clip you do it you’ll find those bottlenecks. And it’s existent astir immoderate compute strategy wherever you personification an bid of magnitude overmuch [to fig out]. They evidently personification shows that personification streamed more, but they’re dispersed crossed overmuch time. So it’s this azygous spike up wherever everybody comes successful a 30-minute window, and if it’s extracurricular of what you planned for … If they planned for — I don’t cognize what their numbers were — 150 cardinal and they sewage 180 million, it was extracurricular of what they thought their precocious limit was. We’ve seen this earlier successful AWS and we’ve seen this successful Amazon. The first clip we did Prime Day we astir apt had issues crossed that too, of conscionable group hitting nan website and different things. So nan first clip you do events for illustration this, it’s a learning process.

I deliberation it’s astir apt overstating it to opportunity that they had to re-architect nan afloat internet, but it is that cardinal spike wherever a batch of applications are conscionable not ... Particularly erstwhile you ain nan infrastructure, and this is 1 of nan benefits of nan cloud, by nan way, is you get to thrust connected nan norm of ample numbers wherever immoderate 1 spike doesn’t overwhelm everything else. Netflix evidently has a immense number of customers, and I conjecture that they’ll beryllium overmuch overmuch prepared for adjacent time. But it’s a bully learning acquisition for anybody moreover astatine a overmuch smaller scale. When you’re readying an arena that has nan imaginable to beryllium materially overmuch than your mean baseline, location are ever risks that location are immoderate scaling factors you don’t anticipate.

So it’s not a astonishing problem to me. We’ve seen it complete and complete again and it’s 1 of those problems that nan unreality helps to solve. But moreover successful nan cloud, readying is required and you personification to deliberation astir really you modular up of it, and things for illustration that.

When you were astatine location watching nan fight, did your pager spell off?

I was texting backmost and distant to our support squad to make judge we were supporting nan Netflix squad arsenic overmuch arsenic possible, yes.

How often does that hap to you arsenic you usage nan nett and you think, “Boy, this is astir apt moving connected AWS. I had amended make judge it’s going fast?”

More backmost successful nan clip erstwhile we were scaling and learning — backmost successful 2007 and 2008 wherever we were learning really to modular there. Today, we’re often astatine a wide modular and truthful everything, tons of things connected nan nett and astir nan world, tally connected AWS. And we usually tally beautiful reliably, truthful it comes up small than it utilized to, for sure.

Do you personification Down Detector bookmarked connected your laptop?

I don’t, no.

We’ve sewage to get nan CEO of Down Detector connected nan show. That is simply a fascinating activity crossed nan board.

Let maine inquire nan Decoder questions because I deliberation this taxable of “we are going to beryllium overmuch reliant connected unreality infrastructure for compute successful nan world of AI,” and that’s sewage to scope each nan group and hopefully make everybody immoderate money and make immoderate useful products and services — that’s nan theme. And I deliberation whether aliases not we tin watercourse group punching each other, and whether aliases not we tin watercourse AI, nan problems location are nan aforesaid successful nan wide sense.

But I want to inquire nan Decoder questions first truthful I tin understand really you are solving those problems, having been astatine AWS for truthful long. So you are taking complete for Adam who was connected astir conscionable a mini complete a twelvemonth ago. He stepped down astir six months ago, you took over. You’ve been location a agelong time. You started arsenic nan first merchandise caput of AWS, which is simply a beautiful chaotic spot to statesman a occupation and extremity up arsenic a CEO. How are you reasoning astir AWS, nan organization, correct now?

There are a mates of things that I’m reasoning about. One, I personification been coming for 18 years, truthful I’ve been fortunate to study a batch of nan different parts of nan business and personification seen it from nan early days until wherever we are now. Over 18 years we’ve grown to beryllium a $110 cardinal business expanding astatine 19 percent, truthful that’s great, and we’re conscionable astatine nan early stages of what that business tin be. I’m pushing nan teams to consistently deliberation astir really we innovate faster. How do we deliberation bigger? And really do we support our customers?

As we deliberation astir nan imaginable of AWS being a $200 billion, $300 billion, $500 cardinal business, aliases immoderate size it gets to, we want to continuously think: What are nan organizational structures? What are nan mechanisms we use? What are nan ways that we supported customers, which worked to get america to $100 billion, and whitethorn not activity astatine $200 aliases $300 billion?

Some of that is conscionable reasoning astir really we modular those aspects. And really do we deliberation astir supporting customers successful a awesome way? How do we deliberation astir scaling our services successful a awesome way? How do we deliberation astir continuously innovating crossed galore different paths? And arsenic you deliberation astir it, we personification to really innovate connected our halfway — nan constituent that sewage america coming astir compute, databases, storage, and networking. But we too personification to innovate astir AI, astir immoderate higher-level capabilities, and analytics. 

We too personification to innovate astir helping customers who mightiness beryllium small technically savvy, truthful they tin return advantage of nan cloud. They whitethorn not beryllium astatine Netflix-level sophistication, which is evidently a very blase exertion team, but want to return advantage of immoderate of nan unreality capabilities. I deliberation we’re continuing to deliberation astir really we support pushing that missive screen to thief overmuch and overmuch customers return advantage of what we have. 

One of nan things that I locomotion a batch of clip reasoning astir is: really we style truthful that our teams don’t suffer agility and velocity arsenic we get bigger. That’s immoderate of what I’m reasoning about, and it’s point that’s room today. Instead, it’s benignant of for illustration looking astir corners to spot erstwhile nan business is doubly arsenic ample arsenic it is today, really do we make judge that we proceed to execute and tally arsenic accelerated arsenic possible?

Can I inquire astir that information of nan puzzle? Where does nan adjacent caller customer recreation from?

Sure.

When you started astatine AWS they were each caller customers. Now, astir immense companies astatine slightest personification an thought of what they mightiness do pinch nan cloud, whether they’re utilizing AWS aliases point else. We personification a batch of CEOs who recreation connected coming and say, “Look, I petition to personification aggregate clouds truthful that I tin spell do title negotiations pinch each of them.” Fine. 

There is simply a caller group of companies that assumes they don’t petition immoderate package support. They’re conscionable going to prosecute a bunch of package arsenic a activity (SaaS) vendors, and they’ll tally their business and usage nan SaaS products nevertheless they want to usage them. And it seems very improbable that they will spell AWS customers themselves because they’ve outsourced a bunch of business functionality to a bunch of different package vendors. I’m conscionable wondering if that’s a caller group of imaginable customer, right? That benignant of business didn’t beryllium until recently.

It’s true, and I deliberation that there’s astir apt subtlety there. So I’ll return a mates of those, 1 astatine a time. Number one, we do personification a batch of ample customers that are moving successful AWS successful nan unreality today, and a immense number of them still personification monolithic amounts of their spot on-premise. And truthful there’s a immense magnitude of maturation disposable there. You tin moreover return our largest customers, galore of them only personification 10, 20, 30, aliases 40 percent of their workloads successful nan cloud. There’s a monolithic magnitude of maturation conscionable helping them get to 70 aliases 80 percent, aliases immoderate that number is going to be, and don’t moreover presume you get to a hundred. There’s a immense magnitude of business there.

I too deliberation there’s a immense magnitude of business disposable pinch customers that only personification 1 percent, aliases rounding to zero, of their spot successful nan unreality because they’re still moving on-premise workloads, whether it’s IT aliases halfway business pieces. Some of it is moving successful accusation centers. Some of that is workloads that haven’t moved to a unreality world yet. Think telco networks, broadly. Most telco networks still tally successful accepted telco networks. There are a fistful of customers, for illustration nan Dish networks of nan world, who personification thought astir and personification moved to building successful nan cloud. Since they sewage to commencement from zero, and personification built it successful nan cloud, they get nan benefits of that agility — but astir haven’t. 

Think astir each of nan compute that happens successful a infirmary today. It’s mostly successful nan hospital. And they’re conscionable examples of wherever there’s an tremendous magnitude of compute that could return advantage of these broad-scale unreality systems that haven’t yet moved there. So there’s a immense magnitude of imaginable successful those further businesses. There’s too just, arsenic you deliberation astir caller customers, each azygous twelvemonth location are a immense number of startups that are created from scratch and they each commencement successful nan unreality too. There’s still tons of greenfield opportunity for us.

I deliberation your study astir companies leaning overmuch into SaaS is ace absorbing and it’s why they’re specified a attraction for us. It’s why we attraction connected dense partnerships. How do we make judge that AWS is nan champion spot to tally SAP, it’s nan champion spot to tally Workday, it’s nan champion spot to tally ServiceNow, it’s nan champion spot to tally ... Keep going down nan list. And so, those SaaS independent package vendors (ISVs) personification ever been a really important customer guidelines for us.

And increasingly, you spot america build capabilities that make AWS moreover overmuch powerful for SaaS vendors. At re:Invent, we announced a capacity called Q Business Index wherever you tin personification each of your SaaS accusation pulled together into a azygous standard that’s owned and controlled by nan enterprise, but you tin banal crossed SaaS products. I deliberation you’ll spot overmuch things for illustration that wherever we tin thief customers not conscionable say, “Okay, my data’s successful a bunch of these SaaS islands and I can’t get benefits crossed them.” 

I don’t deliberation customers won’t beryllium an AWS customer, because they’re still going to personification a accusation reservoir of their ain data, they’re still going to personification their ain applications, they’re still going to tally their ain websites. There are different things that customers are still going to want to do. And truthful I deliberation overmuch of their applications will beryllium successful SaaS arsenic opposed to self-managed software, for sure. It’s difficult to ideate galore customers that won’t personification their ain compute retention database needs also.

When Adam was connected nan show, I asked him, “What’s nan constituent of nan airdrome ads? Who doesn’t cognize astir AWS?” And his reply fundamentally tracked pinch what you’re saying. There are still a batch of customers who we petition to get reasoning astir moving to nan cloud, and that’s why location are Thursday Night Football ads.

Is that your answer? When you get disconnected nan level and you spot nan AWS logo, you’re like, “I’m going to get that guy?”

I mean, look, you tin make that connection for tons of ads. Like, who doesn’t cognize that Coca-Cola exists? But you still spot Coca-Cola ads. And truthful immoderate of it is keeping it apical of mind. Some of it is too … If you deliberation astir nan advertizing that we do together pinch immoderate of nan sports networks — whether it’s NFL, F1, aliases others — a batch of what that does is to thief nexus nan dots. You whitethorn cognize that AWS exists, but helping spot that successful a sermon that you understand, which is football, F1, Bundesliga, aliases immoderate nan athletics is, and really we’re helping do analytics for that sport, is 1 of those things that helps customers nexus nan dots.

And so, it’s not conscionable an advertisement that says, “Hey, AWS exists,” but it is connecting those dots that says, “Okay, if we’re tin to do analytics that tin spot really accelerated a changeable subordinate tin run, aliases spot what nan chance is that an F1 car tin pass,” it helps customers conscionable nexus nan dots arsenic to wherever we mightiness beryllium tin to thief their business too. It too opens nan doorway for america to do that adjacent dense dive wherever we tin dive successful and understand that. And we find that that narration constituent is alternatively valuable moreover if group cognize that AWS exists already.

I do emotion nan thought of immoderate CEO coming to you and saying, “I petition a triumph probability metre for my squad each infinitesimal of nan clip successful existent time.”

That’s great.

Let maine inquire you astir telco for 1 second. Just because telecommunications has agelong been a peculiar fascination of mine. Dish started from scratch. They announced loudly that they were going to usage AWS arsenic their unreality provider, that they wanted to do each nan compute they needed for 5G and each that worldly to tally that web successful nan cloud. Compare and guidance that to nan different telcos. 

When Verizon was launching 5G, for example, they told maine that they were going to build a competitor to AWS because they needed nan compute astatine nan separator to tally nan web anyway. And they said they mightiness arsenic bully conscionable discarded nan excess capacity successful their accusation centers to customers and opportunity it would personification a small latency, aliases immoderate you get from being very overmuch astatine nan edge. Did that cookware out? Or are you saying, “Okay, that didn’t work, and I tin spell conquer those customers now. I tin spell get Verizon aliases AT&T aliases whoever different connected nan network?”

Well, Verizon was a mini spot different. It was a business pinch america wherever we were talking astir perchance trading immoderate of that compute abstraction together astatine nan edge. I deliberation that exertion is astir apt a mini spot ahead, and I still deliberation that there’s an absorbing eventual triumph there. But I deliberation that nan thought was a mini spot up of nan exertion of really low-latency compute astatine nan edge, mostly because a batch of that latency was taken up successful nan network, and truthful it’s difficult to get that usage of a mini latency gap. 

Look, if you spell backmost 15 years, galore companies were reasoning that they would conscionable spell relationship nan cloud. It looked for illustration it was easy. And past they said, “Oh, it’s conscionable a hosting thing. I personification a accusation center. I tin discarded that.” I deliberation astir companies today, extracurricular of nan fistful of 3 aliases 4 companies that are really successful nan space, don’t deliberation that they tin proviso a existent unreality offering. It’s hard. 

There are niche offerings successful peculiar slices, but I deliberation progressively we position this arsenic a business opportunity wherever we tin adhd worthy together. So, I deliberation our business pinch Verizon is great. We look astatine really we tin adhd worthy together, and complete clip we’d emotion for overmuch of nan broader network. Because if you look globally, you’re starting to spot different telcos commencement to bladed into this exemplary of, “Okay, perchance overmuch of nan halfway tin beryllium tally successful AWS” … Then perchance that information is, “Okay, that tin beryllium tally successful cardinal accusation centers,” and truthful we’re starting to spot overmuch core. And past you deliberation about, “Can nan powerfulness entree web (RAN) beryllium tally successful AWS? Maybe. Yeah, it can.” And they’re starting to spot that information successful there.

I deliberation it will beryllium a modulation complete time. But I do deliberation that arsenic we adhd overmuch worthy and show that we tin springiness programmability to their networks, modular to nan networks, and show benefits connected patching and different things for illustration that wherever there’s a batch overmuch elasticity location — I deliberation you’ll spot overmuch and overmuch telcos leaning into to cloud-based spot deployments.

I’m judge your partners astatine nan accepted telco companies admit your support successful nan retconning of their promises astir 5G. You’re doing great. 

There’s a existent divided here. I dream group tin comprehend it. We’re talking astir still trying to get customers to recreation usage unreality services. Step one: move immoderate of your compute retired of nan basement of nan infirmary and into nan cloud. And a batch of companies aren’t location yet, and it seems for illustration you comprehend that there’s still opportunity there. 

Then we’re going to, successful a minute, we’re going to talk astir AI, which is nan absolute cutting separator of, “How do we moreover tally these companies? What do these computers moreover do? How does nan costs activity out?” How are you structuring nan connection to woody pinch that split? “Don’t personification your ain servers successful nan basement?” versus, “Turn your decision-making complete to immoderate agentic AI strategy that we’re going to tally for you.”

Well, successful immoderate ways it’s a overmuch stronger carrot. If nan proscription is, “Hey, tally nan nonstop aforesaid constituent that you’re doing, but do it a mini spot overmuch efficiently and a mini spot small expensively,” that is small of a worthy proposition than if you tin do point that hasn’t been imaginable before. And so, I deliberation that’s why galore of nan workloads that you’ve seen move to nan unreality already are nan ace scalable ones, aliases nan ones wherever they petition tons of compute, aliases nan ones wherever they personification a really ample footprint because they spot nan wins are tremendous for those types of customers. For a server moving successful nan basement of a hospital, perchance they tin prevention a mini spot of money, aliases perchance they tin prevention a mini spot of IT activity aliases whatever, but nan worthy proposition whitethorn not beryllium location unless we tin really coming a batch of value.

You’re not going to beryllium tin to get a batch of nan worthy that’s promised from AI from a server moving successful your basement, it’s conscionable not possible. The exertion won’t beryllium there, nan hardware won’t beryllium there, nan models won’t unrecorded there, et cetera. And so, successful galore ways, I deliberation it’s a tailwind to that unreality migration because we spot pinch customers, hide impervious of concepts … You tin tally a impervious of conception anywhere. I deliberation nan world has proven complete nan past mates of years you tin tally tons and tons and tons of impervious of concepts, but arsenic soon arsenic you commencement to deliberation astir production, and integrating into your accumulation data, you petition that accusation successful nan unreality truthful nan models tin interact pinch it and you tin personification it arsenic information of your system.

And I do deliberation that that is going to beryllium a tailwind complete nan adjacent mates of years arsenic group want to personification these agentic systems. They want to personification their accusation successful a unafraid business but integrated into an AI workflow. You can’t orchestrate an AI workflow pointing it connected a mainframe. It’s not going to beryllium possible. If you personification nan accusation going backmost and distant to immoderate model, nan accusation and powerfulness of making judge that that intelligence spot (IP) stays pinch you is risky too.

But if you move nan afloat accusation into a unafraid unreality environment, you’ll personification a modern accusation reservoir that has each your data. Your exertion will activity there, you’ll beryllium colocated pinch wherever nan model, each nan controls, and guardrails tin run, and you tin personification a retrieval augmented procreation (RAG) standard that’s adjacent to return advantage of each that accusation — that’s erstwhile you tin really commencement integrating it into your accumulation applications. And that’s wherever you’re going to spot a batch of nan really meaningful wins, not conscionable benignant of a cool, “Hey, that’s neat that I tin personification a chatbot,” but really merge it into really your workflows alteration and really you tin do business changes.

I personification seen early signs that, to your mobility astir organization, they’re very complementary. It’s not A aliases B, it is each pushing successful nan aforesaid place. So we’ll personification to personification different capabilities, we’ll personification to personification different motions to thief each of that. But I do deliberation that that move of getting your accusation into a unreality world is benignant of a basal accusation to personification a really, really successful, profoundly integrated AI, I think, into your business processes.

So this leads correct into nan classical Decoder question: How is AWS strategy now? What’s nan org chart?

What do you mean? So opportunity overmuch astir that. Just what is our org structure?

Yeah. How personification you strategy AWS? I mean you’re new, truthful I ideate you mightiness alteration it, but really is it strategy correct now, and really are you reasoning astir changing it?

Well, I will opportunity that an org structure, number one, is simply a surviving thing. So immoderate I show you coming whitethorn not beryllium existent tomorrow, and I deliberation you personification to beryllium agile there. But broadly, really we deliberation astir structuring our teams, I think, is beautiful bully documented successful nan manufacture astir Amazon. We want single-threaded teams that tin attraction connected a peculiar problem and move fast. And truthful what that intends is you really want a squad who tin ain a problem and not beryllium matrixed crossed 10 different things wherever they personification to coordinate a bunch.

In immoderate ways, I deliberation astir it for illustration a ample monolithic instrumentality programme — it’s very businesslike arsenic agelong arsenic that monolithic instrumentality programme is small. And arsenic it gets bigger and you personification aggregate group moving connected that program, past you get a mainframe, and it’s very slow and you can’t iterate connected it aliases move fast.

So what you do is decouple and build services that talk to each different done well-defined APIs. And past you proceed to decouple those programs, you proceed to refactor. That’s really to build modern exertion systems. And you tin deliberation astir containers arsenic nan existent measurement of doing that, which are small, independently moving systems that tin talk to each different done APIs.

Now, if you deliberation astir org structure, it’s not that dissimilar from that. If you deliberation astir really do you personification teams that tin tally really fast? There is going to beryllium coordination, but what you want to do is minimize that coordination taxation arsenic overmuch arsenic possible. And so, if you personification a well-defined API betwixt them, which is like, “I build a activity complete here, you build a activity complete here,” we tin innovate. Occasionally our teams will get together and make judge that we broadly cognize what our imagination is. We want to cognize what nan constituent is that we’re moving towards. But past I tin spell and my service, my organization, aliases my feature, tin tally independently and not personification to personification coordination.

High level, if nan Amazon Elastic Compute Cloud (EC2) squad and nan Amazon Simple Storage Service (S3) squad had to talk each clip they were going to motorboat a characteristic to make judge it worked together, we would move really, really slow. But we don’t, and truthful nan teams tin move really fast.

Then we make judge we personification … It’s benignant of information of nan activity and nan merchandise activity squad to get together and say, “Okay, we deliberation going aft this abstraction is ace important. And immoderate of that is customers are going onto this usage case, and truthful broadly we’re going to personification to spell aft this thing,” but we tin still past personification nan teams spell retired and tally fast. That is an organizing norm that ... And past location are different parts of nan connection wherever we personification teams that tally benignant of nan accusation centers and different global, and immoderate of those are our abstracted teams. But if you deliberation astir nan merchandise and organizing astir nan merchandise and technology, that’s really we deliberation astir it.

This mobility is ever bait for Amazon executives successful peculiar because Amazon executives are raised successful a civilization to deliberation precisely successful this measurement and image nan institution arsenic a bid of microservices. But really is AWS structured?

Just for illustration that. I mean, moreover overmuch truthful than Amazon.

Go done it, what are nan services? What do you deliberation astir allocating nan squad for those services?

There are 200 different services, truthful I’m not going to spell done each of them, but that is it. And we’ll continually refactor and re-think astir them. From a exertion constituent of view, we deliberation astir a compute service. You tin deliberation astir EC2, and past you tin deliberation astir EC2 networking, and past you tin deliberation about, “How do we make judge that it’s optimized astir containers?” And past down astatine nan bottom, you deliberation about, “How do we personification teams of 10 to 20 group that are focused connected a subcomponent of that, that are afloat separable?”

We personification thousands of developers that are each organized connected that principle. Sometimes we’ll move them astir organizationally, but it’s not really nan org structure. The cardinal information is really ownership astatine nan bottom. The apical information is conscionable really businesslike you are astatine management, and really do you make judge that you’re managing nan teams well, and doing that high-level coordination bit. That’s really wherever you move around. But astatine nan core, those teams are beautiful solid. As you find a caller opportunity, you rotation up a caller squad that goes aft it and fig retired wherever it makes nan astir consciousness successful nan org structure. But astatine nan core, that is nan organizing principle. We personification those mini teams and we proceed to thrust them. So that’s it.

And past we style our sales, go-to-market, and trading teams abstracted from that. But from nan halfway merchandise side, that’s really we deliberation astir it and it useful bully for us. I deliberation nan positives are ... Look, location are pros and cons to immoderate organizational building from our side. The pros importantly outweigh nan cons. From nan cons side, sometimes, and I’m judge you’ve heard this disapproval aliases feedback of AWS, which is that sometimes it seems for illustration it’s not perfectly accordant aliases this XYZ characteristic is not supported crossed each azygous activity yet. And that is nan downside of that organizational building — your caller and decorativeness crossed each azygous activity is not ever perfect, and sometimes it takes a mini while to drawback up to each of those things, which is expected arsenic you personification 1,000 different teams tally astatine different paces connected different things.

But nan trade-off is we get to move really fast, we’re ace agile, and we tin respond to customer feedback really quickly. And I deliberation that is nan different concealed — that it’s not conscionable an organizing principle, but it is too that you thatch those teams to really comprehend to nan customer. I’m judge each leader you personification connected coming says they comprehend to their customers, and I don’t judge that they ... Amazon does a really bully business of really internalizing that down to each individual contributor, and we deliberation astir really we spell lick customer problems. And erstwhile you’re small, agile, and tin make decisions, you tin really spell lick customer problems really accelerated successful your area. Those things play connected each different and are helpful.

You did commencement arsenic a merchandise manager. As a merchandise manager-

Technically an intern earlier AWS launched successful 2005.

That’s true. But arsenic a PM, you’re moving immoderate merchandise and you’re astir apt reasoning astir nan customer a lot. What were nan frustrations you had arsenic a PM that you deliberation you tin now trim arsenic nan CEO?

Well, it was a very different business backmost successful nan day. I was nan merchandise caput for each of AWS, truthful ...

And truthful you still are is what you’re saying?

Yeah, exactly. I personification nan aforesaid business now. No, and I kid, location were a mates of different merchandise managers astatine nan clip too. But nan frustrations past and now are too similar, but different. It’s evidently a different modular that we’re operating at. But 1 of nan things I was disappointment astatine backmost successful 2006 was that I knew a ton of things that we conscionable needed to spell coming for our customers. I conscionable had a immense database and it was each astir prioritizing that list, but I wish that we could coming them faster and do more, and moreover astatine nan modular that AWS is coming that’s still true. I wish we could do overmuch and do it faster, and that’s information of why we attraction connected that organizing norm of making judge that you tin get retired of nan measurement of nan teams to move fast. And so, my business coming is simply a mini spot overmuch of, “How do I region those barriers and thief teams move fast?” But that’s it.

I deliberation it’s a batch of we want to make judge that we’re innovating, we want to make judge that we’re leaning ahead. Some of nan challenges we personification coming are different than we had successful 2006. In 2006, we had to reply nan question, “Why would a bookseller ever tally my computers?” And that question, we get small and small today, actually. I don’t deliberation I’ve gotten that 1 for a while. 

But now we personification to woody pinch scale, deliberation astir endeavor requirements, and about: How do I meet audit requirements? How do we support governments? How do we deliberation astir scale? And really do we make judge that we personification tin power successful nan world? And each of those kinds of questions. But each bully problems for america to lick truthful that we tin return them connected truthful nan customers don’t personification to.

This is nan different ample Decoder mobility and it’s going to lead america correct into AI because I deliberation you personification a batch of decisions to make here. Amazon famously has nan one-way doorway versus two-way doorway decision-making framework. Everyone applies it differently. Every Amazon executive I’ve ever talked to holds onto that thought and they usage it differently. What’s your decision-making framework? How do you make decisions?

Well, information of my business is to make nan one-way doorway decisions. So I deliberation that exemplary is, it’s a useful 1 to deliberation about. And conscionable to clarify, successful suit you’re not alert of it, mostly that’s really you spell fast. You effort to specify what those decisions are. They tin beryllium important decisions by nan way. I deliberation sometimes it’s misunderstood what are nan important decisions and not important decisions. It’s not that.

You want nan group that are owning those teams astatine nan edges of nan connection that really ain those products to make important decisions because they cognize champion astir their product. But they’re too decisions that could beryllium undone if we find that it wasn’t nan correct constituent to do. And past nan bigger benignant of, I’m going to spell put $1 billion, aliases immoderate decision, aliases I’m going to motorboat a caller activity that is difficult to propulsion backmost aliases is achy to propulsion back, those are nan one-way doorway decisions that I deliberation we want to personification a mini spot overmuch inspection on. And moreover those, though, I deliberation we are trying to fig retired really do we make those faster too, and alteration a broader swath of group to make those?

But you asked really I make decisions? I deliberation for amended aliases worse, my return is I americium rarely, if ever, nan maestro connected immoderate peculiar taxable that we’re moving on. And whether we’re moving connected compute aliases connected storage, talking astir hypervisors, income compensation, powerfulness contracts that we’re signing, go-to-market efforts, aliases marketing, I americium seldom nan maestro successful nan room connected those. And truthful I make judge that I comprehend and clip disconnected abstraction for those experts who locomotion each of their days reasoning astir that to measurement successful arsenic to really they’ve recreation up pinch their recommendation, really they deliberation astir what we should do.

And past nan information that I bring to that is to one, return a position of a non-expert and inquire immoderate questions and understand really they’re reasoning astir nan problem. Then two, thief nexus nan dots to nan different information of nan connection that they whitethorn not personification visibility into and understand if location are trade-offs that they whitethorn not personification thought astir because they’re making a trading determination and didn’t cognize astir a caller merchandise that we were delivering complete there. I effort to make judge that, arsenic an organization, we’ve connected those dots and past inquire nan correct sets of questions. And past if there’s a tiebreaker determination I’ll personification to do it truthful that we tin move fast. I deliberation nan spot we don’t want to beryllium successful is to beryllium location and conscionable connection forever. At immoderate point, you petition a tiebreaker decision, and that’s what I position my business arsenic doing arsenic well.

All right, truthful I deliberation this does bring america consecutive into AI because this is simply a bunch of decisions that everyone has to make and nan outcomes are, I would say, still uncertain. As an industry, everyone is telling maine this is nan halfway enabling exertion of nan adjacent procreation of computing. This is simply a level displacement is nan building that a bunch of CEOs personification utilized pinch me. Do you deliberation AI is simply a level shift? Do you deliberation it’s that ample of a deal? Or is it conscionable different suite of capabilities that AWS will relationship people?

It’s a bully question. I’ll commencement pinch really I judge that AI is incredibly transformational, whether you telephone it level displacement aliases not I tin get to that successful a second, but I deliberation it’s an incredibly transformational exertion that overmuch than benignant of … Look, these things recreation astir each decade aliases so. I deliberation it is 1 of nan technologies that tin beryllium wholly transformational. Whether it’s transforming industries, companies, jobs, workloads, aliases workflows, I deliberation it has a existent imaginable to personification a worldly effect connected each azygous information of really we deliberation astir work, life, personification experiences, and nan like. I’m a afloat believer, that that is true. And I deliberation there’s a timeline question: is that going to beryllium successful nan adjacent 12 months, 24 months, aliases nan adjacent 5 years? But I do deliberation it is going to hap and it’s going to personification a existent alteration connected a batch of pieces of business.

Platform displacement is an absorbing mobility because “platform” assumes that AI is not yet a level and I deliberation that that is simply a overmuch unfastened question. It’s a immense enabling technology. And whether you build connected that AI aliases that AI is embedded successful everything that you build pinch and is simply a halfway constituent of what you build pinch and really you deliberation astir … It’s a instrumentality that is really meaningful and impactful. I deliberation it remains to beryllium seen arsenic precisely what that means, but it is simply a transformational exertion that-

Wait, tin I make that simpler?

Yeah.

Can I put that connected a spectrum for you, conscionable to make this overmuch existent for nan listener? 

Do you deliberation AI is overmuch for illustration multi-touch? Or do you deliberation it’s overmuch for illustration nan iPhone?

I don’t cognize if it’s really for illustration either of those. I would liking that it-

Well, because multi-touch is for illustration … You can’t make an iPhone without multi-touch, but that doesn’t connote that we’re each going to commencement utilizing touchscreens each of nan time. 

Yeah. It’s not for illustration multi-touch. It’s not for illustration that. I don’t cognize if it’s an iPhone either, though. It whitethorn beryllium overmuch akin to nan nett disruption. That’s what I’m saying. I don’t cognize if nan nett is simply a platform, per se, it’s a displacement successful really you would coming an application. So perchance it’s a platform. But I deliberation it’s overmuch akin to wherever location will beryllium basal shifts successful really you coming products, offerings, and services, and really you do your activity daily.

So nan nett has been hugely transformational pinch really you do your activity daily. You utilized to beryllium location connected a typewriter or, I don’t know, represent memos, aliases do whatever, and now you’re connected a instrumentality each day. You’re interacting connected SaaS applications, emailing people, aliases there’s conscionable basal connectivity. And I do deliberation that AI is overmuch akin to point for illustration that, wherever it has that basal displacement into really you’re going to get activity done.

Yeah, I deliberation you and I are immoderate astir nan aforesaid spot and you described nan typewriter workforce pinch nan aforesaid benignant of, “I deliberation that’s what it was like.”

Yeah. I don’t know. I ne'er had a business for illustration that.

It’s nan aforesaid for me. I think, “Typewriters… group had them.” The timeline constituent you brought up is really interesting: what is nan timeline for this? It’s peculiarly absorbing to maine because I get a bunch of AI CEOs coming connected nan show telling maine what their timeline for artificial wide intelligence (AGI) is. 

So Sam Altman precocious said AGI would beryllium imaginable connected existent hardware, and OpenAI is making a batch of sound astir AGI for a assortment of reasons that we tin unpack astatine a later time. Mustafa Suleyman, who is nan Microsoft AI CEO, was conscionable connected Decoder, and he said, “I don’t deliberation we’re going to get to AGI connected existent hardware, but perchance incorrect 2 to 10 years.” And he said we’re decidedly not going to get location connected Nvidia GB-200s.

You tally accusation centers, you personification a bunch of Nvidia chips successful those accusation centers, and you are processing your ain chips which I want to talk about. Where do you spot yourself playing successful that debate? Is it, “One of these vendors is going to ray up AGI connected someone’s accusation center, and I dream it’s AWS?” Is it, “I’m building this hardware to alteration that to happen?” Is it, “This is what everyone’s talking astir to goose their banal prices and I conscionable petition to discarded overmuch capabilities to overmuch customers?”

Well, number one, it’s an intolerable mobility to inquire because there’s nary meaning of what AGI is. So erstwhile you scope is too an intolerable meaning because I don’t know. You can’t specify erstwhile you scope an undefined thing. 

What I would opportunity is that I deliberation that it’s conscionable a continuum and I deliberation that AI — we’ll telephone it AI inference, nan expertise to spell do activity — is going to proceed to get overmuch tin complete time, and I deliberation that location is simply a agelong roadworthy of this to get much, much, overmuch overmuch tin complete time. And it’s going to get overmuch small costly to tally complete time, which I deliberation past explodes nan number of ways successful which group will make it useful. Whether it’s moving agents, doing different workflows, aliases performing long-running reasoning tasks, I deliberation there’s a afloat large of things that you tin imagine. And so, there’s conscionable a continuum of wherever nan things yet onshore and wherever you’re tin to inquire nan computers to do overmuch for you astatine small costs.

I deliberation hardware platforms are going to play a ample information successful that. I deliberation package algorithms are going to play a ample information successful that and you’re going to petition immoderate of those. I don’t cognize erstwhile you scope AGI, I don’t cognize what that means, but I do deliberation that nan adjacent procreation of compute will beryllium ... it’s going to coming location between. And immoderate nan existent procreation is that we conscionable announced pinch Trainium 2, and yet pinch Blackwells and GB-200s, I deliberation we’ll springiness customers a 2–4x boost successful compute capacity per dollar. We announced Trainium 3, which will springiness different 2x boost to compute by nan extremity of 2025.

That is going to thief that goal. You will proceed to get overmuch and more, and you’re going to beryllium tin to do bigger and bigger things, and you’re going to petition algorithmic improvements arsenic well, which galore of nan teams, ours included, are very focused connected doing.

But conscionable straightforwardly, if OpenAI declares that it has achieved AGI, which it seems very overmuch poised to do, it will personification done that connected a bunch of Azure accusation centers. Do you deliberation AWS needs to credibly claim, “Oh, we tin do that too,” to compete pinch Azure? I mean, they’ve defined AGI down, to beryllium clear. But they’re going to opportunity it beautiful soon.

Yeah, I understand location are contractual position that they’re moving through. But they personification immoderate accusation for reasons to do that, from my understanding. But it’s not astir declaring anything. It is just, “Let’s fig retired what you are arsenic a customer.”  I americium small consenting successful puffery successful nan spot and overmuch consenting successful really I tin thief customers execute existent outcomes. And truthful it’s fine, location tin beryllium trading statements. They tin beryllium like, “I personification nan biggest compute cluster successful nan world,” or, “I personification AGI.” 

Okay, but astatine immoderate constituent I want to thief a slope fig retired really they tin trim nan magnitude of fraud that they’re seeing, aliases amended nan velocity astatine which they tin o.k. loans, aliases immoderate nan constituent is that really goes and helps nan business. I want to thief a biotech find crab cures faster and amended and fig retired really they tin importantly shrink and aliases amended nan efficacy of what they find.

So those to maine are absorbing and useful outcomes. And truthful if you show me, “Hey, tin you thief a customer find cures for crab faster?” Awesome. That is simply a constituent that I’m focused on. Was that AGI that did it aliases not? I don’t know. I’m not consenting successful that, per se. I’m overmuch consenting in, “Can I really thief our customers coming worthy to their businesses?” And a mini spot small on, “Can I personification a liking successful nan crushed astir marketing?” Because I think, astatine nan extremity of nan day, customers really attraction astir that first one, not that 2nd one.

I deliberation this leads correct into nan adjacent information of nan AI puzzle that I’m seeing unfold. It’s wherever should nan finance go? Is it training caller models which mightiness beryllium hitting a benignant of scaling norm problem, and getting small tin astatine a slower title than they were earlier pinch each successive model? Or is it successful inference, which is what you’re describing? “Hey, we tin bring nan costs and velocity of conclusion down connected nan existing models and make cheaper, better, overmuch cost-effective products.” Where’s your accent correct now?

I don’t deliberation you tin premier 1 aliases nan other. You perfectly … The world is going to coming overmuch tin models and they are expensive. They require a batch of compute, and it’s an area of finance for us, and it’s an area of finance for galore of our customers. And I deliberation it’s nan correct area of finance for a batch of those because I do deliberation … You don’t get overmuch capable, smaller models if you don’t personification nan ample exemplary to commencement with. That is conscionable really it works. You can’t recreation retired pinch point that’s a really, really powerful mini exemplary if you didn’t too build a frontier model, aliases commencement pinch a frontier model. So you personification to personification those ample frontier models and I deliberation we’re going to petition those to beryllium overmuch capable.

There’s a batch of invention and conclusion successful really you tin thrust costs down. Some of that is simply a systems problem, immoderate of that is simply a hardware problem, and immoderate of that is an algorithmic problem. You tin deliberation astir exemplary distillation. There’s a afloat bunch of techniques that you tin do to get these smaller, faster conclusion models, which I deliberation are going to beryllium hugely impactful and important to delivering existent worthy to enterprises.

I deliberation you spell talk to customers now and they are nary longer consenting successful bright, shiny AI impervious of concepts. They want point pinch a existent return connected finance (ROI) associated pinch it. And nan ways you coming awesome ROI are that you either personification overmuch worthy and/or small cost. I deliberation immoderate of those are going to beryllium important to support raising nan level of ROI that you tin deliver. So, if we deliberation location is this monolithic expertise to toggle style organizations, we personification to support expanding what models tin do and decreasing really overmuch they tin cost. I don’t spot really you premier 1 of those. I deliberation you personification to do both.

If you had to premier one, it sounds for illustration you would premier inference, right? Because that’s wherever nan products are getting built.

Yeah. Well, what I’ll show you is, successful my keynote astatine re:Invent, I talked astir different constituent that I for illustration to do successful Amazon, and we do here, which is that we garbage a constituent we telephone nan “tyranny of nan or,” which is forcing personification to premier A aliases B stifles innovation. It intends that you don’t spell retired and invent really to do A and B. And truthful you can’t pick. I’m telling you, it is not an A aliases a B chance, it’s an A and B, and we personification to push our teams to fig retired really to do both, which includes bigger training — and we personification to small nan costs of that, by nan way. It can’t conscionable support scaling linearly, which is each information of nan silicon investments that we’re making and networking, and things for illustration that. How do you make nan costs to train these really ample models lower, truthful that you tin train bigger models?

And I deliberation we personification to make that investment. We are making that finance and it’s a immense area of opportunity for america because coming it’s excessively costly to proceed to ramp astatine nan rates of nan costs of nan infrastructure. That’s a ample information of Trainium, investing successful really to get nan costs down for training. I deliberation nan conclusion broadside has to thrust costs down too, which is incredibly important for nan return broadside of it. So you personification to do both. It won’t activity if you conscionable do 1 side.

I did watch your keynote and you are invited for that alley-oop connected nan “tyranny of ‘or.’” I knew it was coming because I wanted to inquire astir Trainium. This is simply a immense investment. You’ve been astatine it for respective years, you announced Trainium 2 astatine re:Invent, it has further capabilities successful training and inference. It’s designed to beryllium bully astatine inference, truthful you tin usage nan aforesaid spot everywhere. 

Building these chips is simply a immense investment, and you are up against dedicated spot companies. You’re up against AMD, which is too making a immense investment. You’re up against Microsoft, which is making its ain investments. You’re up against Nvidia, which is nan leader and has a immense caput start, not only successful nan chips but too successful nan package ecosystem astir nan chips. What do you deliberation astir that title and that investment?

It’s small a title and overmuch an summation of choice. I don’t deliberation it is GPUs or-

Oh, by nan way, I forgot Google. I should astir apt constituent retired that Google has an precocious accusation halfway and AI capabilities.

Yeah, Google does, that’s right. And truthful it turns retired we’ve been making chips now for complete a decade. So we’ve been making silicon chips, our ain civilization silicon for overmuch than a decade. We’re really … we personification 1 of nan astir knowledgeable teams successful nan manufacture doing this, and truthful it’s not a caller thing. It’s not for illustration we dove successful coming and said, “We personification nary thought what we’re doing,” By nan way, immoderate of those others are learning it for nan first time. Not Nvidia of course, aliases AMD,  and Google’s been making chips for a mini while too. I deliberation Microsoft is beautiful caller to this space. But we deliberation that that is simply a ample advantage for america arsenic we understand really to do this astatine scale, and we understand really to do it successful nan cloud.

I deliberation we personification immoderate advantages successful that we don’t personification to do it for a wide group of customers. We personification to deploy our chips successful precisely 1 environment. We personification to deploy them successful an AWS accusation center. We personification to deploy them successful precisely 1 server, aliases we don’t personification to support a afloat OEM infrastructure, a group of different drivers, aliases a bunch of different things. It’s conscionable successful our business and we cognize precisely what that’s going to look like. And we deliberation it’s a choice. We don’t deliberation that it has to meet each azygous usage suit for each azygous customer. 

We deliberation that Nvidia GPUs, AMD GPUs, and others are going to beryllium ace interesting. They personification bully platforms. Both of them personification very bully teams that are executing really, really well, and I deliberation they will proceed to do that. I don’t spot immoderate logic why they wouldn’t. We strategy to beryllium a awesome partner of theirs for a really agelong clip and support that and relationship it to customers erstwhile it’s nan correct exertion premier for their usage case.

We deliberation that we tin relationship absorbing choices, and we’ve done it pinch Graviton. We’ve proven that we tin motorboat a processor astatine a wide modular that is very useful for a group of workloads, a wide group of workloads for our customers. And successful Graviton’s case, it doesn’t mean we don’t bargain a ton of Intel and AMD chips and relationship those to customers. We of group do, and those are expanding businesses for america arsenic well. It’s conscionable overmuch choice. And we deliberation that premier makes AWS a overmuch charismatic level for customers because they personification overmuch choices than they do different places. That further premier is nice, and information of that premier is we want to really bladed successful and make judge it’s nan champion spot to tally Nvidia GPUs, AMD, Intel, and others. 

But it’s a ample opportunity for us. And if you do think, which we do, that AI is going to disrupt each of those different industries, it’s a monolithic opportunity wherever it’s not 1 subordinate that is going to beryllium nan only compute level that each of those things tally successful complete time. We deliberation that we personification an opportunity to build immoderate of that and proviso differentiated choices for customers who return to tally AWS.

Chips and spot finance is simply a semipermanent decision. You’re making decisions now and allocating superior that mightiness not net disconnected for a decade aliases more. Do you deliberation that exemplary training is hitting a scaling limit? That it’s going to plateau nan measurement that immoderate group are saying it’s plateauing?

I deliberation group for illustration to talk astir scaling laws because again, it sounds nosy to talk about. But I deliberation that it astir apt conscionable intends location personification to beryllium overmuch levels of invention. I deliberation if you look complete immoderate exertion ramp, you spot 1 peculiar method ramping up for illustration this and past it slows down, and past personification says, “Oh, really astir you effort this?” And past it goes backmost up again, and past you effort point else. And truthful there’s going to personification to beryllium package and algorithmic changes. I deliberation it’s not a unsighted dump of overmuch data, adhd overmuch compute, adjacent your eyes, and past you get a bigger exemplary adjacent year. You’re going to petition smart group looking astatine it, driving it, and figuring retired caller ways to thief that. But that doesn’t mean that you’ve deed a limit. I deliberation it’s conscionable that you’re going to personification to support innovating successful different ways.

Think about, number one, really long, and it was longer than a decade, that group were saying that we were hitting Moore’s Law of scaling limits. That was, “Can you return 17 nanometers and make it 15 nanometers and 13 nanometers?” And you’re saying, “Okay, there’s going to beryllium a limit.” They had to fig retired nan exertion to get past a mates of those. I retrieve location astir 10 nanometers, group were like, “I don’t deliberation you tin get past this,” and now we’re building three-nanometer chips. And truthful you support getting smaller because location are caller technologies successful there.

You had to fig retired really you woody pinch interference, and you had to deliberation astir really stacking nan memory, different structures of nan chips, and different things for illustration that — but you activity done those. In nan meantime, you benignant of figured retired really to do overmuch compute connected an accelerator for illustration a GPU, which past gave you a immense measurement alteration successful compute. And so, nary longer are group worried astir whether we are hitting nan limits of what a 17-nanometer Intel spot from 10 years agone is doing, right? Now we’re orders of magnitude overmuch compute than that.

Well, clasp on, clasp on. I mean, this is nan existent limit. One institution figured that out. Taiwan Semiconductor Manufacturing Company (TSMC) figured that retired utilizing an EV instrumentality from 1 institution successful nan Netherlands. And they’re nan supplier for everyone, which intends you are now asking TSMC for capacity successful title pinch Nvidia, Apple, Qualcomm, AMD, and even, to immoderate extent, successful title pinch Intel, right?

They figured retired parts of that. I mean, they figured retired nan layout chip. And by nan way, [TSMC CEO] C.C. Lei and nan squad did a awesome business of figuring it out. So yes, but nan world figures it out, right?

But Intel famously did not fig this out.

They didn’t.

I mean, that’s wherever they are correct now.

But others have.

I’m saying correct now nan bottleneck successful nan spot industry, successful nan investment, is 1 institution tin proviso this product. Is that point that you actively deliberation about? Like, “Do they personification nan capacity to fto america compete?” 

I mean, they’re making tons of investments and I deliberation they’re scaling. I deliberation others are looking to drawback up successful that abstraction too. They personification a awesome lead, and this is too existent successful exertion and has been for a agelong time. Somebody jumps up and figures it out, gets a lead, and it’s a usage for them for a while and others drawback up. I deliberation you tin look astatine immoderate of nan High Bandwidth Memory (HBM), and immoderate of those different fabrications that are coming up, and they’re catching up and uncovering different caller ways to do that. There will beryllium different inventions that leapfrog complete time. But obviously, fabs are hugely capital-intensive investments. And so, I americium judge that others will yet find caller and different ways to innovate astir that too. It has ever been existent successful technology.

Are you making immoderate bets connected immoderate non-TSMC fabs?

I wouldn’t personification point to denote there, but we partner pinch tons of folks. We partner pinch Samsung, Intel, and others that personification their ain fabs arsenic well, and bargain tons of different worldly from them. From practice to CPUs, we bargain parts from tons of different fabs astir nan world.

The different ample constraint is power. You personification said 2 to 3 generations from wherever we are successful AI we’re going to petition 1 to 5 gigawatts of power, astir a mean city. This led you to talk astir atomic powerfulness and really we’re going to petition that. That’s a ample woody to say, “Okay, we’re going to petition truthful overmuch AI capacity that we’re going to build atomic powerfulness plants.” Microsoft and different companies personification said nan aforesaid thing. Is that still wherever your mind is? This is going to beryllium truthful successful that Amazon is going to effort to build immoderate powerfulness plants?

Yes. It is. We’ve made important investments there. And that’s a scope of things, by nan way. It’s a portfolio. This is not a caller strategy for us. Over nan past 5 years, we personification commissioned overmuch renewable powerfulness projects than ... Each twelvemonth for nan past 5 years we’ve commissioned overmuch than immoderate institution successful nan world. And that’s bringing connected caller powerfulness into nan grids, and whether they’re caller prima farms aliases nan caller upwind farms, and now we’re adding atomic to that. So it’s conscionable a portfolio of that. I deliberation nan world is going to petition overmuch carbon-free energy, and compute and accusation centers are a ample accusation of that. We are pushing difficult to make judge that nan world has tin sources of that. I do deliberation that atomic powerfulness will beryllium an important constituent of that strategy complete nan adjacent mates of decades.

And so, we are excited astir mini modular reactors. I deliberation that it’s a exertion that’s a mini ways away. By nan way, it’s not a lick for nan adjacent mates of years, but past 2030 and beyond, I deliberation it could beryllium a very important component. One, you tin really put it adjacent wherever you petition nan powerfulness to be.

Another of nan bottlenecks that we tally into is astir transmission. It’s not conscionable powerfulness generation, but it’s transmission. So you tin personification a prima workplace retired successful nan desert, but if you don’t personification transmission to get it to wherever your accusation centers are, past it doesn’t do a batch of good. Those are immoderate problems that petition to beryllium solved. And it’s not conscionable accusation centers, it’s electrical cars, it’s electrification of each of our businesses. There’s a bunch of these things that are going to petition to happen, and truthful I deliberation atomic powerfulness is going to beryllium an important information of that, and mini modular reactors. 

I deliberation nan world’s going to personification to build overmuch of these ample industrial-scale atomic plants arsenic well. I deliberation a batch of people’s heads are successful nan “That was scary backmost successful nan ‘50s erstwhile nan exertion wasn’t arsenic safe.” Today, it’s a very safe, scalable technology, but it’s point that we personification to support spending connected and scaling.

We’re going to personification you backmost for different afloat hr connected atomic powerfulness plants. That’s a afloat rabbit dispersed that I want to talk astir astatine immoderate constituent successful nan future. But we’re moving retired of clip here. And I conscionable want to inquire nan biggest mobility of all. This is simply a batch of immense guardant investment. You’re designing chips, we’re investing successful TSMC’s capacity. We’re talking astir atomic powerfulness plants, we’re building bigger accusation centers. There’s an $8 cardinal finance successful Anthropic to thief build a accusation halfway and past tally Anthropic and Claude. 

When is immoderate of this going to make a dollar? You petition a merchandise successful nan personification aliases endeavor marketplace that throws disconnected tin separator astatine tin modular to money each of this finance and still make money for nan group making nan product. And ideally, nan group paying for nan merchandise are utilizing it to make overmuch money connected nan different side. The economics of this are still very unclear to maine unless you are Nvidia. When does each of this make a dollar for you?

Yeah. Well, AWS is simply a nice, profitable business for Amazon.

Right, you’ve sewage nan separator to locomotion connected it, but astatine immoderate point, it has to return.

I think, look, and for customers, they’re progressively looking astatine it this way. It’s not conscionable us. And I said this a mini spot ago. If you talk to customers they are very focused connected really they tin personification ROI-positive AI projects. I deliberation nan unreality has already proven to beryllium ROI affirmative crossed a wide swath of industries. We’re moving your accusation to nan cloud, your compute to nan cloud, and you summation agility. And truthful I deliberation we’ve proven that we tin coming awesome ROI for customers successful moving to nan unreality broadly and taking AI aside.

And so, what we’re progressively seeing customers opportunity is, “I want to spot nan ROI of these AI projects.” And I do deliberation that that is an important displacement wherever it is not conscionable nan cool, it’s not conscionable nan shiny entity factor, it is a, “How do I make judge this makes sense?” And we are spending clip pinch customers reasoning astir that. How do you activity done nan usage cases that are enabled coming that tin coming existent value? Some of those are broadly reported astir things for illustration modernizing your relationship center, and we deliberation Connect is simply a awesome offering for customers to do that. We’re really seeing a immense number of customers move to Connect successful a unreality relationship halfway to return advantage of galore of those AI capabilities. You spot immoderate of that successful optimizing your back-office projects.

And I deliberation increasingly, arsenic nan agentic workflows really get overmuch overmuch powerful, and arsenic we deliberation astir collaborative agentic workflows and longer moving agentic workflows, you’re going to spot overmuch and overmuch worthy recreation up done these. As nan models get overmuch tin you’re going to spot overmuch worthy coming up done those. And truthful I deliberation it’s connected us. It’s incumbent connected america to make judge that these are very profitable for extremity customers to spell and implement.

But fto maine conscionable put that successful a exemplary that makes it perchance a mini spot sharper.

You’ve been astatine AWS since nan beginning. AWS started, and I’m going to flatten this narrative, you tin correct maine for it being a mini excessively flat, but conscionable successful nan flattest imaginable way: Amazon is building a bunch of these services. “Hey, we personification excess capacity. Hey, we want to build microservices for our ain components. We tin resell those.” 

So you get a bunch of benefits connected nan measurement of conscionable building Amazon, and past you tin move that into a business. AI, correct now, feels for illustration location are a bunch of ideas for products that mightiness beryllium useful. Inside Amazon, extracurricular of Amazon, for AWS’s customers, whoever, but it requires a monolithic magnitude of guardant investment. 

It’s not just, “We’re benignant of doing it anyway.” It’s overmuch more, “Hey, there’s a immense opportunity here. We petition to leapfrog up and perchance get immoderate overmuch customers.” Or perchance there’s a level displacement aliases immoderate it is. We each spot nan immense committedness that is happening astatine a subsidy, and that subsidy seems dangerous.

It’s not nan correct characterization of it. So location are a mates of things I would say. Number 1 is that AWS was ne'er astir excess capacity of Amazon. Just for illustration mathematics doesn’t work. You tin ideate that I’ve heard that narrative, it sounds nice. And arsenic soon arsenic Christmastime comes around, if I personification to return Netflix’s servers distant truthful that we tin support portion traffic, that doesn’t really activity arsenic a business. So that was ne'er nan idea, intent, aliases extremity of AWS.

And we built nan businesses from scratch. They weren’t reusing Amazon components. We learned from that. They’re an unthinkable early customer to study from nan components that they would need. But we built them from nan crushed up to support a wide scope of customers. AWS itself was a ample finance by Amazon to spell aft a wide caller business. As you deliberation astir it now, we had Amazon arsenic a ample customer of ours, for sure, and they were a ace adjuvant customer for america to study astir what ample enterprises would petition from services for illustration AWS and they proceed to be.

I deliberation AI is not that dissimilar. Amazon needs AI. You mentioned that you watched my re:Invent keynote, Andy was up location for 25 minutes talking astir each of nan cool things that nan remainder of Amazon is doing pinch regards to AI. And you’re talking astir Rufus, you’re talking astir really we’re reasoning astir our proviso concatenation and fulfillment centers, and crossed nan afloat scope of ... And Alexa. That business desperately needs AI capabilities to, again, reimagine our business, get overmuch efficiencies, and coming caller experiences for customers. Amazon is customer number 1 for a bunch of these capabilities. So if AWS tin build them and Amazon tin return advantage of them, that’s awesome and immoderate of those things are true.

So yes, it’s a ample guardant investment, but we too personification Amazon still utilizing them, and we are successful a different spot now. When we started successful 2006, we had zero outer customers, and we now personification a cardinal outer customers aliases aggregate millions of outer customers. That is simply a immense customer guidelines that is ready, willing, and excited to bargain and usage nan products that we have. So that finance is simply a guardant investment, but you too personification a really ample guidelines that you tin amortize it crossed and spell relationship it to, which makes that finance thesis a mini spot easier to get over.

All right. So I’m going to inquire you nan aforesaid mobility again to wrap up pinch each this context. When do you deliberation each this finance will spell ROI positive?

I deliberation it’s a affirmative ROI. Well, it depends connected what you mean by ROI positive. I deliberation there’s a batch of finance successful nan world.

Right. But this is simply a batch of finance successful AI crossed nan industry. When do you deliberation it’s going to commencement returning?

I mean, if you deliberation globally, I deliberation it’s ROI affirmative now. I deliberation nan mobility is erstwhile does it spell overmuch evenly distributed? Look, I deliberation nan hardest mobility of that, honestly, is for nan exemplary producers. I deliberation that’s nan azygous hardest question. I really deliberation today, aliases if not today, very soon, it is going to beryllium ROI affirmative for nan wide swath of customers utilizing AI and building it in, for illustration banks, information companies, pharmaceuticals, and others. You tin make that ROI-positive communicative today, and I deliberation it will proceed to get better. And I deliberation for infrastructure providers for illustration Nvidia, of course, it’s very ...

They’re doing fine.

I deliberation nan mobility is erstwhile does  … The folks who are making nan immense investments are nan ones who are building foundational models from a package position and past reselling those foundational models. It’s a bully question. I don’t cognize nan reply to erstwhile that finance benignant of afloat pays disconnected for an OpenAI aliases an Anthropic. I deliberation Amazon and Google astir apt personification a different mathematics of erstwhile we tin make those net disconnected because you get psyche usage of them from your ain use. I don’t cognize that. But there’s a batch of smart group investing in, continuing to put finance successful a wide swath of AI companies. And you personification to believe, which we do, that location is simply a monolithic economical usage from galore of these AI capabilities that are orders of magnitude bigger.

I do deliberation it really plays into that mathematics equation. As conclusion gets cheaper and overmuch tin location are aggregate orders of magnitude overmuch conclusion to beryllium done. And that is erstwhile it yet starts to net off, I think, for a batch of those exemplary providers, and successful a huge, monolithic way.

All right, you are intelligibly successful nan weeds of each these products, which is nosy to hear. Let’s extremity here. Last question. When you’re trying retired each these AI products, which is nan 1 that you usage that makes you think, “Okay, is this finance worthy it”?

That’s a bully question. I don’t cognize if location was immoderate 1 merchandise that I sewage excited about. The first merchandise that I ever utilized that I said, “Hey, I deliberation this is real,” is conscionable for illustration everybody else. I deliberation ChatGPT was conscionable a transformational product. It was a awesome UI and it really unlocked for everyone what was possible. So nan first clip that I really realized that this was going to return off. We were making investments internally, but I deliberation we were hopeful that they would get there. I deliberation that’s nan first 1 that I utilized that I really understood.

Now it’s difficult because I usage thousands of them and I deliberation each of them are really cool. And I deliberation location are a batch of startups from group that are building AI products. People who are making caller proteins — which is unthinkable — folks for illustration Perplexity who are making hunt engines that are overmuch overmuch interesting, relationship centers, and banking applications. There’s a afloat large of them now that are incredible. I deliberation Amazon makes some, and galore of our partners make many, truthful those are each incredible. But it really was, conscionable for illustration nan remainder of nan world, I deliberation ChatGPT was nan first 1 that really helped solidify it.

Got it. Very negotiated answer. Matt, this was great. You’ve sewage to recreation back. I really enjoyed this conversation.

Great. Thanks for having me.