Video highlights

Nvidia could not build an independent compute market itself, Conrad said, because it would not be independent enough for people to trade on it.

Still, Nvidia benefits greatly from such a market existing.

SF Compute started as a company called Junnar that simply tried to sublease its cluster.

Each new customer request pulled it deeper into operations, including cleaning, room service and security, until it was effectively running the whole hotel.

To deliver compute reliably it had to build the operating capacity of a full cloud, which the company calls physical settlement.

The lesson it took from this is that the operations must exist before the market can be standardized.

In oil and power markets, physical settlement depends on pipelines and delivery networks that must already exist.

During COVID, oil prices went negative because storage was full and sellers paid buyers to take the oil.

Compute has the same kind of constraints, so SF Compute designed physical settlement and its order book together.

Conrad said the market is not yet mature, deep or liquid.

About 5% of SF Compute's total capacity is on the order book, and the company keeps moving more onto it.

Order book volume is small, but the liquidity on it is high, so the traded capacity is concentrated.

Conrad said the company measures liquidity by whether it solves the customer's problem, not by the number of market makers.

Two virtual machine types from one operator can differ by 40% in performance degradation due to GPU kernel launch time.

Orders attach properties such as performance grade and time window, and are matched against sell orders with those properties.

The trade-off is less concentrated liquidity and wider spreads, though buyers who can use many capacity types may find arbitrage.

Conrad said a good cloud tries to kill defective GPUs and nodes before they reach the customer, and replaces bad units through RMA.

Some problems are firmware bugs that are not necessarily Nvidia's fault, such as a server vendor's firmware that affects every unit.

Clusters are typically financed over three to five years, with payments spread across that period.

Order-book capacity blocks can be booked for an arbitrary length of time, and customers currently pay fully up front for a block.

Batch inference can be interrupted at almost any point and its owner does not much care, so short notice is enough.

Live inference customers expect the service to stay online, so interruption requires much longer notice and usually carries an SLA penalty.

The two are therefore priced differently, much like day-ahead and live markets in power.

Conrad compared expiring compute to milk: a block approaching its end cannot be sold later, so its price must fall to clear.

Current listings are unsold inventory, which pushes their price down, while a healthy curve has nearer compute priced higher than future compute.

Customers care about chip generation because of factors like VRAM and larger NVLink domains.

Oil also comes in many types, but refineries act as black boxes; compute's heterogeneity forces complexity onto the end customer, which hurts a market maker.

The goal is to identify a shared standard that reduces the dimensions, though some will remain.

Conrad said his team spent three years learning why physical settlement is needed.

Compute is less fungible than people assume, and the industry must build fungibility over time.

Premature financialization, he warned, makes the market easy to manipulate and creates basis risk between the hedged price and the capacity actually available.

The credit default swap market is reportedly about 12 times the size of the bonds it insures, which illustrates how cash settlement can drift from the underlying.

Silicon Data and Computer Change are described as early movers that chose cash settlement, while SF Compute does physical settlement.

Conrad said both approaches can exist and benefit each other.

Other companies are working on standards, residual value insurance and deposit lending to resolve the financial bottlenecks of compute.

SF Compute does all the things a cloud does without being the cloud, and it deliberately does not own the hardware.

Conrad said owning it would let the company manipulate the forwards market, so it must stay independent.

It also takes over backplane management, including power units, switches, network processors and the operating harness around settled nodes.

SF Compute is primarily US-focused but deploys clusters around the world.

Its pitch to sovereign compute players is to help them set up a cloud in their own country while bringing San Francisco off-take, expertise and clusters.

Conrad said the goal is to make those clouds financeable.

Conrad said Nvidia cannot make the market because it would not be independent enough for people to trade on it.

Without trading, there is not enough liquidity for lenders to finance against the forwards market.

The same reasoning applies to the big clouds and inference providers, which have too much demand on their side to run it neutrally.

Conrad described the company as virtualizing the data center, so customers get what they would have gotten by owning the hardware.

He compared the change to the arrival of the cloud itself.