Follow the CAPEX: Cloud Table Stakes

The capital expenditures (CAPEX) going into the cloud are Sagan-esque, with billions upon billions being spent on sprawling complexes of interconnected datacenters scattered across the planet. The hyper-scale public cloud operators (Amazon Web Services, Google, Microsoft) operate BFGCs (big, uh, freakin’ global computers) at immense industrial scale, with townships of well-ventilated warehouses that collectively hold millions of servers, connected by their own transoceanic cables.

Just to give a feel, here is a Microsoft datacenter complex in the eastern United States:

It seems pretty impressive until you see the expansion under way, which dwarfs the original complex (in the distance at the top of the following picture) and will result in a facility a mile long:

Beyond being mind-blowing to people like myself who come from the traditionally asset-light-to-the-point-of-no-assets-beyond-a-couple-laptops software business, the vast sums being spent to move dirt, pour concrete, bend metal, and sling electrons also help us handicap the race for the trillion dollar cloud “jackpot”.

CAPEX is not sufficient to win the cloud, but it is surely necessary. Not only is massive CAPEX outlay a prerequisite to offer cloud services globally at scale, but also a strong indicator of on-going success. Customer adoption and utilization necessitate further CAPEX spending.

And there are significant economies of scale operating at hyper-scale. You get efficiencies of operation, specify your own highly-optimized and cost-reduced servers and network equipment, get exclusivity on the latest CPUs (and probably soon will just design your own) and increasingly run a private global fiber network so you don’t have to pay telco retail (where there is no Moore’s Law). It is an unprecedented level of vertical integration.

And these accumulated CAPEX expenditures, given their sheer scale and the time required to translate balance sheet cash into a global footprint of fully operational and interconnected datacenters, constitute a significant competitive moat. This cumulative investment is a proxy for the size of those moats.

This post examines when the hyper-scale players had their “cloud inflection point”, i.e. when they began to devote CAPEX to their cloud, as well as the magnitudes of their cumulative and on-going CAPEX investment. From this analysis we get a sense of just how big a check new entrants will have to write if they want to play this game seriously.

Unhelpfully for our task, the cloud players don’t break out their cloud-specific CAPEX. It requires some work, speculation and reading tedious financial documents to glimpse the portion dedicated to cloud as opposed to other investments. Every company at this scale makes non-trivial investments on mundane things like office buildings and other facilities. But each also has CAPEX unique to their businesses. Amazon spends big on warehouses and the robots scurrying around inside. Google has (or at least had) investments in self-driving cars, a veritable air force of flying objects (balloons, drones, satellites), retail fiber networks and one still hopes the odd space elevator that all require some degree of CAPEX. All three companies build hardware products for which they may invest in manufacturing tooling which their outsourced manufacturing partners then operate.

Our analysis begins in 2001, as this is the first year for which we have Google CAPEX numbers (a whopping $13 million). We start with the investment section of the cash flow statements in their publicly reported financials. For Amazon and Microsoft we combine the Plant and Equipment line on the cash flow statement with their separately, contentiously and somewhat ambiguously reported capital leases (equipment that is financed and is paid for over time instead of up front, which maps very well to servers that have a limited useful life and are generating revenue over that time, plus we live in a zero interest rate world so why not). While bean counters and finance wonks can argue about the accounting impact of these leases, in real terms they constitute even more CAPEX. Including the capital leases makes the CAPEX numbers significantly higher. In 2016, Amazon did $5.7 billion in capital leases on top of $6.7 billion in CAPEX. We find an extra almost $1.1 billion in CAPEX for Microsoft in 2016 when we check their couch cushions for capital leases. Google does not appear to be using capital leases to fund their infrastructure. The Microsoft numbers have also been mapped from their July to June fiscal year to the calendar year, which the other two companies use as their fiscal year, for a true annual comparison.

All three companies show roughly similar trajectories, with annual CAPEX spending in 2016 between $10 and $12 billion. There is some variation from trend around the financial crisis, with Google pulling back sharply in 2009, Microsoft in 2010 and Amazon going parabolic out of the crisis.

The magnitude of these expenditures is even more impressive when compared to some of both the biggest companies on the planet and the biggest spenders on CAPEX. It is stunning that Microsoft now outspends Intel on CAPEX.

When we look at cumulative spend since 2001 through 2016, the curves are also similar. Google has spent $58 billion on CAPEX since their founding, while over the same time period Microsoft has spent over $48 billion, and Amazon almost $45 billion (though it is the most backloaded):

When we normalize against revenue, things are choppier:

Google is the biggest relative spender, though fluctuating dramatically from 4% in the aftermath of the financial crisis to as much as 18%, averaging 12% over the entire period.

The Microsoft numbers may be the most interesting as they illustrate the transition from an asset-light software company into one with a global cloud footprint, going from under 4% to 12% of revenue spent on CAPEX today. I’m also told these numbers understate the increase as the cloud build-out began while Microsoft was in the midst of a big office building spree, meaning the typical CAPEX for a pure software business is even lower (hmm, Oracle might be interesting to look at in this regard…).

Next we’ll drill down on each company.

Google

Google pioneered the cloud computing model to support their search and ad business, so their cloud inflection point is basically coincident with the company’s inception. It is only much more recently that they have begun to try to dumb down and expose a tiny fraction of their infrastructure externally as a public cloud.

Given this data cover almost the entire lifespan of the company, it also includes things like office space for over 70,000 employees (with slides a de rigueur if extraordinary facilities expenditure).

Google’s cloud infrastructure supports search, YouTube and a billion some odd Android devices, amongst other things. Google Search is the biggest application in the world, entailing trillions of annual queries against the many copies of the entire Web they store (which they size at a modest 60 trillion URLs). The ad business relentlessly tracks every click by billions of users. YouTube, which Google bought in 2006, serves up millions of hours of video daily and no doubt has required totally insane CDN and network investments, contributing to its continuing lack of profitability. Google Cloud Platform is basically a rounding error compared to these other applications, but nevertheless gets to leverage the underlying infrastructure.

Google’s extracurricular “Other Bets”, aka Google “X-cess”, also consume CAPEX. Google started to break out the CAPEX associated with “Other Bets” in 2014. Despite the sheer amount of metal involved in many of these activities, the CAPEX involved is actually a smaller percentage of the overall spend than might have been expected (and this is probably bad news for hopes of a grand societal bargain wherein we accept the all-surveilling eye of Google tracking our every move in exchange for a space elevator). However, CAPEX for “Other Bets” has grown rapidly from $501 million in 2014 to $1.39 billion in 2016 (over 13% of total CAPEX). It will be interesting to see what effect jettisoning various satellite and drone programs as well as approaching “put-up-or-shut-up” time for self-driving cars will have on this spend going forward.

It isn’t crazy to think Google may have spent upwards of $50 billion on their infrastructure over the lifetime of the company. No small part of that represents multiple generations of servers and exabytes of disks that have been replaced due to obsolescence and failure, as opposed to purely incremental capacity (so with a three year useful life, Google could have deployed six generations of hardware for some of its capacity). Broadly, Google’s public cloud efforts get to piggyback on these efforts, although they are also making some incremental investments just for Google Cloud Platform. They have had to deviate from their historic and highly centralized “just do it our way” attitude and are investing in smaller datacenters located in more geographies to address data sovereignty demands.

While Google faces a variety of challenges in its public cloud efforts, it is safe to say they are much more likely to involve fickle humans, product-market overshoot, and finding the right software face to put on their infrastructure as opposed to infrastructure itself.

Microsoft

Microsoft’s cloud inflection point is pretty easy to discern. We see two big accelerations in Microsoft’s CAPEX spending. One starts with an perhaps overly optimistic build-out for search in 2005-2009 that got the company cloud expertise and a global infrastructure footprint. The second begins in 2012 and continues to accelerate through 2016, driven presumably by Azure and Office 365. The bulk of that growth is for cloud CAPEX though some may also support the Surface and Xbox hardware businesses. It is notable that Nokia came and went with little or no discernable impact on Microsoft’s CAPEX spend (presumably Nokia had outsourced their manufacturing by that point).

Microsoft also has a search business. While they have only a fraction of Google’s queries, it still requires ginormous global infrastructure. Microsoft has spent $43 billion in CAPEX since 2006. Taking a stab and saying 80% of that is for cloud infrastructure suggests a cumulative cloud CAPEX investments of on the order of $30 billion (similarly summing total CAPEX above 2% of revenue yields similar number).

Amazon

As with all things Amazon, there is frenetic activity on multiple fronts. CAPEX spending barely registers before 2009 and in that year we see the knee of the proverbial hockey stick, with CAPEX growing 40-fold since. This CAPEX explosion includes AWS infrastructure (and Amazon isn’t leveraging an existing search-scale infrastructure, so this spend is driven by AWS), a massive expansion of e-commerce distribution centers to get ever closer to ever more of the buying public, and an office building frenzy that has turned Seattle into the “crane capital of America”.

Figuring out how much of this is for AWS is difficult. Amazon, not renowned for its transparency, actually provides a little more breakdown on its CAPEX spend than the other companies, breaking out both capital and build-to-suit leases (hat tip to The Next Platform for charting):

Unfortunately, like Amazon’s infamous charts with no units on the vertical axis, this additional information isn’t particularly useful. We can speculate that buildings (datacenters, distribution centers, office buildings, and/or biospheres) are constructed on build-to-suit leases and the computing stuff that goes into the AWS datacenters is purchased via capital leases. Amazon does say about their capital leases: “the increase reflecting investments in support of continued business growth primarily due to investments in technology infrastructure for AWS, which investments we expect to continue over time.” Amazon has spent about $11.4 billion via capital leases since 2009.

The challenge is teasing apart what has gone into AWS vs. the traditional Amazon e-commerce business (that AWS now subsidizes). We know AWS has seen explosive growth since its debut in 2006, but so too has the the rest of Amazon which has grown revenue by $115 billion (not including AWS) in the same period. And they have grown their distribution footprint at least five-fold in that time:

These warehouses are occupied by at least 30,000 Minion-esque Kiva robots. Even if these robots were funded by capital leases, at a cost on the order of $1000 per robot, they’re not likely to be material.

My guess is AWS’ infrastructure spend is on the order of $15 billion between servers and buildings. One area where AWS is behind both Google and Microsoft is owning their own network, which means they are paying a lot more to move bits. Presumably they are planning to rectify this and will add it to their CAPEX spending. We can expect Amazon’s CAPEX investment to continue to grow based on the three vectors above, plus they are opening a new vector as they expand their distribution business with 40 planned air freighters plus long haul and delivery truck fleets (FWIW, FedEx spends $5 billion a year on CAPEX, 10% of revenue). And maybe the Prime Air drones will be more than a PR stunt.

Cloud Table Stakes

This reading-between-the-lines analysis suggests the hyper-scale clouds collective CAPEX spend on their infrastructure could be approaching $100 billion ($50B for Google, $30B for Microsoft, $15B for Amazon). Given the cloud jackpot is a trillion plus dollar annual opportunity, that doesn’t seem crazy at all.

Are there are other estimates or disclosures of CAPEX spend on cloud infrastructure out there? Am I missing anything (e.g. I ignored amortization because I’m interested in the gross spend, not accounting values)?

In a future post, we’ll play follow the CAPEX for two companies that keep telling us they are leaders in the cloud contest: IBM and Oracle. The CAPEX will tell us if they have real clouds or are just clowns…