This report has been made freely available to all readers thanks to Olas, the headline sponsor of Delphi’s Crypto x AI Month. Olas enables everyone to own a share of AI, specifically autonomous agent economies.

AI x Crypto (Part 3)

Part 1 – The Tower & the Square
Part 2 – Seizing the Means of Production (Infrastructure)
Part 3 – Composable Compute (Middleware)
Part 4 – The Agentic Economy (Apps)

Integrated vs Modular AI

More than any other super power – immutability, censorship resistance, etc – I have always felt crypto’s true promise lay in its composability. The DeAI thesis is quite similar to “money Legos” in DeFi, allowing developers market compensation for creating useful primitives which can be used and built upon permissionlessly at global scale. In short: to properly incentivize and unlock the compounding of software globally.

Software is eating the world, and AI is eating software.

As software creation moves from talented developers to mountains of GPUs trained on their prior output, we are seeing competing visions for how the tech stack of intelligence will evolve. If Google represents the extreme in the integrated approach: their own silicon, their own cloud, their own proprietary data sets, their own foundational models, their own distribution…

Then DeAI represents the other extreme – with fragmented data sets fed through often heterogeneous hardware pieced together by novel orchestration algorithms, applications underpinned by open source models and an end vision of composable agent collectives – networks of a million interchangeable components working in tandem to solve a given task as compute efficiently as possible.

In many ways, this is a classic integrated vs. modular tech stack debate. As Clayton Christensen reminds us, the early innings of any tech adoption cycle typically sees an integrated approach take the lead – dominating the entire value-chain to reduce friction and deliver a superior customer experience. AI is obviously quite early in its adoption curve, and, unsurprisingly, integrated approaches have jumped out to an early lead.

However, as industries mature, modular value chains often take share through standardization, composability, and commodification of various layers of the stack – often leading to comparable final solutions at a lower price.

Interestingly though, as Ben Thompson has pointed out, this theory of disruption has proven more prescient in analyzing B2B markets than consumer markets, citing the glaring counter-example: Apple. Apple keeps winning because of, not in spite of, its tight integration between hardware and software:

1. Delivering an incredible user experience
2. Providing users with greater trust and security within their tightly managed ecosystem

While today, intelligence clearly favors integration, it is less obvious whether, in the long run, integrated or modular approaches will win out.

Whether DeAI and the modular extreme can take share will be a test of coordination. Can all of these disparate pieces be strung together in a way which delivers a compelling user / developer experience, in a cost effective manner, in a highly-trusted and secure way?

Or… will it just be a mess?

In many ways, this is the original promise of blockchains themselves – global, trustless, permissionless coordination between unknown parties. DeAI may prove the ultimate test of blockchains themselves. A test in efficiently coordinating the inputs – compute, data and talent – set to dominate the 21st century. A test which will be decided by the integration of its middleware.

The test of composable compute.

Meta Goes Scorched Earth

Perhaps the biggest win for DeAI last month had nothing to do with crypto and everything to do with the release of the Llama 3.1 herd of models – including the 405b parameter behemoth, capabilities on par with leading closed-source models today.

As I mentioned in the Tower and the Square, DeAI’s biggest disadvantages lay within the infrastructure layer with hyperscalers providing tight integration between the cloud and model layers – using large balance sheets to corner the market for high-end hardware to power superior, closed-source models, with an eye on outsized profits through renting out intelligence.

Realistically, Meta was one of only a handful of players with the resources and distribution (RLHF) with a right to win in the foundation model space:

I, like many, was skeptical of Zuck’s long-term commitment to open source and worried DeAI and open source more broadly were developing an over-dependence on Llama models which may fade as the capex requirements continue to mount precipitously, and the ROI became ever more suspect.

Source: Sequoia: AI’s $600b Revenue Question

New-Zuck called my bluff. He is going all in – releasing what can only be called an open source manifesto and committing Meta to opening up subsequent generations in a bid to make Llama the industry standard.

Source: Meta Research

Open sourcing foundational models of similar performance to the existing SOTA has materially tilted the field in the direction of open source / DeAI by somewhat commoditizing the model layer and moving competition again further up the stack (at least for now – tbd on next gen superclusters).

While coordinating the buildout of massive super computers is arguably done better by authoritarian, hierarchical corporations, the tuning, experimentation, tooling and application layers almost certainly favor open, permissionless and compounding developer communities.

With Zuck and his massive cash pile and mountain of H100s committed to open source, we are likely to see 1000 flowers bloom – tuning and distilling models of all shapes and sizes on wide ranges of specialized data sets.

Nous Research is a great example – fine-tuning each generation of Llama and Mistral models to optimize for greater personalization: at first using a tightly curated data set of assistant specific examples to generate a low-cost, highly-personalized chat assistant – without worries of censorship or model creep all the way up to Hermes 3, an enhanced version of Llama 405b. Any entity with app-specific data can do the same – training the multi-trillion parameter model on say 100m specialized tokens, targeting their precise use case.

Llama is rapidly emerging as the standard for open source LLMs, but foundational models are set to be much broader than just text…

GNNs: Foundational Models for Blockchains

Where LLMs delineate patterns in large corpora of texts and learn to predict the next word, Graph Neural Nets (GNNs) process, analyze, and learn from graph-structured data. Graph databases (Neo4j etc) and GNNs are quite relevant in the web2 world today, popular in major use cases like social network analysis, recommendation engines, and fraud analysis.

As on-chain data primarily consists of complex interactions between users and smart contracts – in other words, a graph – GNNs appear a logical choice to underpin AI use cases on-chain. As GNNs process information and make predictions about the future state of the network, they are highly relevant in trading, DeFi and even social use cases.

Examples include:

• Price predictions: on-chain behavioral models to predict prices
  • Automated trading strategies
  • Sentiment analysis
• AI Finance: integrations with existing DeFi applications
  • Advanced yield strategies
  • Better utilization of liquidity
  • Security: better risk management / automated governance
• On-Chain Marketing:
  • More tailored airdrops / targeting
  • Combine user behavior on-chain with external web2 data sets
  • Recommendation engines based on on-chain behavior

All of these use cases pull quite heavily on organized data inputs which is why, as noted in this alpha feed post, I am so bullish on data warehousing / indexing plays like Space and Time, Subsquid, Covalent, Hyperline. Data indexing and verification will be essential to enabling this unlock, just as Snowflake, Databricks, etc have served as core enablers to enterprises becoming “AI ready”.

Source: Subsquid Whitepaper, David Attermann Substack

While I’m also bullish on GNN adoption in Defi, I’m a bit more cautious about the longer-term value capture of foundational models themselves given the explosion in LLMs and the intensity of competition. Leading model candidates will have to move quickly to retain a lead, only protected by:

1. Data Flywheel – as users and apps use a particular model, they can improve the underlying performance, attracting more users etc
2. First Mover Advantage – the first mover can establish a strong brand with users or better tooling / composability for developers making it more difficult for other models to gain traction – ideally re-enforcing point 1.

Realistically, we are still extremely early in the AI arms race – even on the LLM side and especially in new domains like blockchains, so any monetization / value capture predictions are still extremely precarious. However, I wouldn’t want to be a late arrival.

Examples: Pond, RPS

Given the fragmentation in models we are seeing – both on the LLM side, multi-model, GNN and more, the logical question becomes, how do we optimize this exploding supply for developers and users?

Routing: Harnessing the Cambrian Explosion

AI development is evolving rapidly – with more or less a new paper dropped daily flaunting new architectures or compute optimal syntheses between models. While the below contemplates primarily LLMs, the process should be broadly applicable as models of all types proliferate. I am far from an AI expert, but I do my best to distill complex ideas down for the average degen.

Below is a high-level overview of different approaches spanning from the most-integrated (i.e. “The God Model”) down to complete modularity (i.e. “Agent Collectives”) in orchestrating intelligences.

• The “God” Model – one model to rule them all
• Foundational Model + RAG – above but with user-specific context injected into the prompt
• Mixture of Experts – explores “search” within the model to enhance compute efficiency, routing the input to the optimal “expert”.
  • Pros: This can be scaled up to “millions of specialized experts” for extreme compute optimization
  • Cons: usually model and scenario specific, so more difficult to benefit from the explosion in external models
• LLM Synthesis – combines multiple LLMs, integrating their outputs into an enhanced final result
  • Pros: Strategic combination of smaller models can match or even outperform larger models
  • Cons: Requires at least two steps – generation and synthesis – increasing cost and latency
• LLM Routing – unlike synthesis, routing can select the optimal model for a given input without performing inference on the entire stable

Source: RouterBench: A benchmark for Multi-LLM Routing System

It generally comes in two types:

• Non-predictive: selects output without model assisted synthesis
• Predictive: selects the optimal LLM without requiring evaluation of the output, usually using supervised learning or a reward-based model

The extreme end of routing appears to be agent collectives – creating modular ecosystems of agents and agent components which can be interchangeable and routed through dynamically.

In reality, all of the above approaches will be used in different contexts depending on the complexity of the query, the performance / cost tradeoff preferences, and privacy / sensitivity of the data.

However, to me, the direction the world seems to be heading (ASSUMING LLAMA KEEPS UP WITH THE PACK…) is towards greater modularity – either within the model (Mixture of a Million Experts) or externally (LLM Routing / Agent Collectives) largely due to the gains in compute efficiency.

Source: RouterBench: A benchmark for Multi-LLM Routing System

Like LLMs, DeAI is undergoing its own Cambrian explosion of approaches for integrating and coordinating intelligences. At a high level, the projects appear split into several variations – from more integrated to more modular – but with a lot of cross-over:

1. Super models: coordinating disparate actors to build ever-better, open foundational models
2. Routing / synthesis: Identifying the right model to use at the right time
3. Agent collectives – interoperability protocols to combine together units of code to solve ever-more complex tasks

All three of the above categories will have a role to play, but the “balance of power” between them will shift with relative differentiation. If foundational models are commoditized, then routers and agent collectives will capture most of the value. If not, a handful of leading foundational models should be very well-positioned.

However, before diving into the up and coming ecosystem, its important to tip our hat to the DeAI OG whose sprawling network of subnets has taken a holistic approach.

Unbundling Bittensor

Today, Bittensor is almost synonymous with “DeAI Coordination Layer”. The protocol establishes a p2p marketplace which incentivizes the production of machine intelligence – enabling the exchange and collaboration of data, compute, models, and knowledge across its diverse array of subnets.

Source: Sami Kassab

The end goal is effectively a “digital hivemind” where various subnets can interact and learn from each other en route to ever more capable intelligences. To cover Bittensor comprehensively would require a report onto itself, which I feel has been covered ad nauseum.

For a great overview / bull case: Sami Kassab

For the preeminent critique: Iridium Eagle

My own view is not particularly sophisticated: Bittensor’s open design space and breadth which has been its greatest strength in sourcing talent and igniting a host of experimentation may also prove to be its biggest liability.

While a slightly odd metaphor, my mind keeps coming back to Craig’s List.

Source: Andrew Parker

Craig’s List was the OG online marketplace proving that indeed there was PMF for the exchanging of goods online. However, its breadth ultimately became unwieldy, allowing newer entrants to carve out standalone vertically-focused marketplaces through heavier focus, more tailored services etc – into large standalone platforms that each ultimately eclipsed the horizontal bundle.

Like Craig’s List, its not entirely clear to me why the diverse array of subnets on Bittensor should utilize the same token. In fact, many of the subnets active on Bittensor – from pre-training, to data scrapping, to compute, to storage, to tuning, to model ranking – have stand-alone competitors dedicated specifically to optimizing those use cases.

While providing a great canvas for experimentation, I fear the protocol’s ceiling may be capped – with the most consequential use cases picked off by more focused stand-alone protocols tailoring their offerings and incentives towards a single cause – and relying on the open market for exchange.

At every layer in the AI stack – from data to compute to training to tuning to orchestration to apps – the race is well underway.

The Race to AGI: Better Models

Given the high-end hardware and bandwidth requirements of scaled foundational model training, Big Tech has obvious advantages in a world where the scaling laws and current architectures hold.

Source: Sequoia: Steel, Servers, and Power

However, decentralized training algorithms are emerging from infancy – hoping to orchestrate distributed hardware using algorithms with significantly reduced data transfer needs.

Gensyn and Prime Intellect were discussed last month, along with exciting advances in lower latency training, but they are not alone. NeuroMesh is another new entrant with a novel approach using Predictive Coding Networks (PCN) which aim for parallel, fully localized, asynchronous learning. PCN is inspired by the human brain’s localized learning approach which would eliminate the need for back propagation altogether – the key bottleneck to distributed training.

With continued advances in decentralized training algorithms and growing GPU marketplace supply, we may have the early infrastructure ingredients for more distributed training. However, even with increasingly more capable open source models, their business model remains broken.

As Joey points out, the economics of open source models in today’s high-capex world make little sense. The field is literally dependent on Zuck’s good will and Apple / Google vendetta to keep up with the state of the art. The question many emerging projects aim to answer is: can capitalist incentives be baked into open source models so that talented researchers, developers, and compute / data providers can be properly rewarded via business models that makes sense?

One project attacking this problem head on is Pluralis. Pluralis aims to build “swarm locked models” with fractional ownership distributed based on compute contributed – without any one participant or group able to obtain the complete model weights without spending more computational power than would be required to train the model. In many ways, this design is extremely elegant because it would not require the massive up front capex of large centralized training runs but instead entice existing compute with fractional equity in (hopefully) useful models, receiving an ROI as the model becomes used in the wild. Swarm contributors would naturally be inclined to lend their compute to models they thought would have the highest utility and therefore command the highest ROI.

As Alex Long reminds us, 350k H100s running continually at peak power draws 0.24 GW. Bitcoin’s PoW mining consumption was estimated at 150 +/- 50 TWh in 2022, or 17.12 GW on average and approximately 0.5% of worldwide energy consumption. Assuming continued breakthroughs in decentralized training, pooled compute two orders of magnitude larger than the largest centralized clusters today has already been assembled in a single protocol.

Incentives matter.

Sentient is another ambitious project aiming to unlock latent talent through incentives. Recently raising US$85m to develop an “Open AGI Economy,” Sentient aims to bring ownership rights to open AI development: “establishing an ecosystem of researchers, developers, and users collaborating on an open AI platform to build AGI, transcending the boundaries of traditional monolithic and closed API-based AI platforms.”

Source: Sentient Foundation

One of the key breakthroughs Sentient plans to implement is “model finger-printing”: essentially putting “backdoors” into models which allow creators to use secret prompts to establish provenance, often resistant to changes during fine-tuning. This provenance would enable models to remain open and downloadable but with usage and enhancements tracked, allowing a sophisticated economy to form around compounding developer contributions – with all members of the decentralized value-chain rewarded proportionately as the model is continuously enhanced and tapped by the end user.

This vision could upend the current “altruistic” nature of open source and unlock the millions of talents outside of the top AI labs who want to contribute to AI development. If successful, this could fulfill OpenAI’s original promise before Sama concluded the open source model could not sustain the returns necessary to fulfill their vision: a vision of open AGI – whether in the form of more powerful foundational models or inter-linked collectives underpinning millions of agents and applications for billions of users. The essential innovation here is a business model one: open, but not free.

Compute is an essential ingredient to ever more powerful models, but as we move to the “reasoning” phase and applying AI to a host of different use cases, the collective tinkering of humanity may prove just as essential in the successful application of intelligence.

However, the decentralized AI value chain is complex and simply having more capable models does not automatically make them useful. Projects like the Mira Network are also focused on developer contributions but with greater emphasis on the tooling and infrastructure necessary to abstract AI workflows via dev-friendly SDKs which allow developers to more seamlessly integrate new models, data sets, and compute into their applications – both incentivizing and streamlining contributions. Like several “alt-layer 1s” we will discuss later, Mira aims to orchestrate the entire decentralized AI supply chain – providing a buffet of “flows”, models, data, and compute options which developers can choose from to dramatically enhance convenience and expand the market of AI development.

The vision of these projects is enormous: transforming the business model of intelligence. But they will also take longer to play out. Facebook’s commitment to capex intensive centralized training will set a very high bar in terms of what will be available to open source developers. And holistic AGI ecosystems / economies naturally would take much longer to form – especially if further advances in distributed training / routing are necessary to be competitive with Big Tech.

However, as Bitcoin has proven, incentives often prove transformative. Capitalist incentives unlocking latent compute and talent the world over could well end up being crypto’s primary use case, if executed properly.

But in this Cambrian explosion of open, incentivized contributions, how do we find the right model for the right task?

Routing: Finding the Needle in the Haystack

Given the exploding number and deflationary nature of models, routing is rapidly emerging as one of the most important functions in AI. How does one serve the right model for the right use case at the right price?

In DeAI, Nous Research’s subnet 6 on Bittensor was one of the earlier examples. Concerned the Hugging Face leaderboard could be gamed, Nous decided to release a subnet which was effectively a tournament to decide, using fresh synthetic data which could not be gamed, which open model could best approximate the outputs of the SOTA closed source models at the time.

Clearly, the ceiling in this tournament is capped by the standard – in this case GPT – further complicated because text-based outputs themselves are fairly subjective. However, the process of using dynamic competition to solve the routing problem is intriguing, especially in domains where “objective truth” vs. the prediction can be harvested.

Allora: Model Selection Reimagined (Supply Side)

Allora appears to be the next logical step in the evolution, hosting competitions between different models in different “topics” in a way that is “context aware” and self-improving over time.

Allora is designed to evaluate and combine the predictive outputs of ML models based on both historical performance and their relevance to current conditions. This contextual awareness is an essential differentiator – born of the requirement for “workers” to provide not only an inference but a forecast of the accuracy of each other’s inferences.

Source: Allora Litepaper

The process of condensing inferences and forecasted losses into a single inference is called “inference synthesis,” not too dissimilar from the “LLM synthesis” discussed above. However, in many domains in which Allora operates – like price predictions in Defi – the prediction accuracy is more quantifiable in nature. How far away was the actual price from the predicted price?

The network assesses forecasted losses and applies weights to workers’ inferences based on their expected accuracy and then combines top contributions into a single inference which is later compared by “reputers” with the “ground truth” as it becomes available (i.e. price), hopefully enabling more accurate predictions over time:

From a value-chain perspective, this is effectively an aggregator on the supply side – pulling in competing inputs from models like POND which compete to provide the most accurate inferences.

As there are multiple steps – meaning more expense and latency – this approach strikes me as most appropriate when the stakes are high: for instance in trading algorithms as opposed to more generic chat bot use cases.

However, in a world of fragmenting models, carving out a niche as the supply side aggregator / synthesizer across a wide range of topics feels like an attractive position to stake out. However, I still suspect it will pale in comparison to the ultimate value capture of demand side routers.

Morpheus: Your Interface to the Internet of Value (Demand Side)

Given the high-levels of competition we are seeing in frontier model capabilities and Meta’s well-funded ambitions to guarantee an open source standard…

As well as the continued shrinking of ever higher-quality models…

Source: ArtificialAnalysis.ai

It seems likely that many highly capable models will be run locally…

If the end goal on the enterprise side is a highly productive, low cost “remote worker,” then the goal on the consumer side is a highly capable personal assistant. Essentially, Siri, but… it works.

Apple is playing a different game from most of the big AI labs, betting that the value capture is ultimately in the distribution – as is so often the case.

Apple intelligence is a nod in this direction, on-device models which have user context, tap into latent hardware, and protect user privacy – answering most context dependent queries on device: “Hey Siri – give me three Italian restaurant reservation options tonight at 7pm within a 10 min drive” and can route to their private cloud or third party models, with user permissions, for more compute intensive queries.

This on-device agent effectively becomes one’s front page to the internet “by-passing” applications and Google search to accomplish the user’s intent directly. Google and Apple are clearly well-positioned here given the strong depth of user context, trust, touchpoints and stranglehold on the mobile OS duopoly.

While Apple and Google will release on device agents which liaise between private user data and the internet more broadly, Morpheus aims to perform a similar role between the user and the maturing web3 stack.

In many ways, I see the future as a fight between Apple Single Sign On, Google Single Sign On, and crypto wallets. Three ecosystems in an all out race to be the user’s trusted liaison – a race driven in equal parts by performance, convenience, trust, and security. Right now, Google and Apple, using a more integrated approach, are clearly the two front runners, but the web3 ecosystem – with the infusion of AI capabilities – is (almost) reaching a point where the various components can coalesce into a meaningful alternative ecosystem for highly determined users.

The goal of this coordination layer is to smooth the friction in using this alternative ecosystem.

Morpheus is one example of a crucial piece of infrastructure which needs to be built for this vision to come to fruition. The project aims to provide an open source on-device agent which can be customized to the user’s preferences and help users more effectively navigate the dismal web3 UX. The agent would have the user’s context and preferences but would also be imbued with algorithmic rankings to assist in navigating towards more trustworthy / useful smart contracts and the emerging network of agent services. Just as Google and Apple rose to prominence by filtering through the internet deluge to provide high signal information and secure applications, Morpheus and other demand side routers would aim to fill a similar niche but atop open rails and infrastructure.

• “Send 20% of my paycheck to Mom in USDC”
• “Help me source the cheapest available compute for this rendering job”
• “I have Ethereum, can you help me ‘stake’ it?”

All of these prompts which could be routed through a Solana transfer, the Render Network, or Lido, respectively – depending on algorithmic ranks.

Having a trusted on-device agent, whether in the world of web2 or web3, which liaises between a user’s private data and intents and routes them through the best path may end up being the most important relationship a company (or protocol) can have with a user.

Unsurprisingly, Morpheus is not the only game in town. The big boys are waking up to this opportunity…

But it doesn’t stop there. Morpheus aims to have a whole ecosystem of smart agent services with which users can interact.

And these emerging agents will need to work together.

Agent Interoperability

Agent collectives represent the gold standard of modular intelligence. In many ways, this would be the crowning achievement of composable compute – arguably the mission of web3.

While Google has integrated everything from the TPUs to the Cloud to the Models and the Apps, the promise of crypto is the opposite: an open and composable alternative – from the infrastructure, to the intelligence, to the applications – where competition within each layer of the stack, assuming strong standardization and interoperability, should lead to superior cost or performance over time.

Agent collectives apply this concept to the intelligence layer itself, betting that an ecosystem of composable and exchangeable components – behaviors, skills, full blown agents, up to completely automated services, standardized and interoperable – will outcompete more monolithic approaches to intelligence. Seeing the success of MoEs and LLM routing on the Pareto curve, this appears to be a logical end point to expect. DeFi may be less performant than traditional finance today, but the composability of open software primitives promises a compounding within the ecosystem which should supersede that of closed ecosystems as performance and regulatory roadblocks are navigated.

Agent collectives are making a similar bet. A bet that crypto remuneration can properly incentivize open source devs to develop useful components and that these components can be pieced together into highly cost-competitive services – operating as on/off chain hybrids – with limited overhead, capable of automating many DAOs and other services entirely.

Source: Theoriq Litepaper

Obviously, the “meta – routing / planning” agents are essential in this architecture, assisted by a combination of algorithmic and human feedback to ensure the most useful components are readily sourced and creators rewarded proportionately.

The architectures vary quite a bit, and while its early, I find myself sympathetic to on-chain, off-chain hybrids like Olas or Theoriq which recognize existing blockchains are ultimately not suitable for many AI applications but ultimately settle back to leading Layer 1s intermittently.

Source: Autonolas Whitepaper

While extremely competitive, Agent interoperability is intriguing because of the size of the potential end market. In the event one protocol becomes the standard, there is a meaningful chance of a compounding development flywheel coalescing around it. As developers receive sizable rewards to craft components, more components become available to build services, more services emerge to provide fees to owners and back to the ecosystem.

As the open libraries grow, the flywheel should accelerate.

If Google became a $2 trillion dollar business indexing the world’s information, then indexing agents and agentic components which not only retrieve information but can also accomplish fully automated tasks… should have a lot of room to run…

We will dive further into agents next month, and the above is far from comprehensive with others like Fetch, Naptha, WayFinder, Althea etc all having heavy cross-over here.

However, the bid to become the leading hub for AI apps and agents is not limited to “Layer 2s.” Many emerging projects believe a full-blown Layer 1, vertically integrating the data layer, the model layer, and the agent layer is the only real way to bring the promise of intelligence on-chain.

The Alt Layer 1 Bid

Just as Ethereum rose from Bitcoin’s shadow by offering a new platform with smart contracts and turing completeness, and Solana has now join the ranks of “the majors” by focusing maniacally on becoming “the world’s fastest state machine”, AI-first Layer 1s aim to join the power-law distribution through offering architectures that optimize for AI workloads.

In many respects, blockchains are the ultimate “coordination layer” but leading Layer 1s today were not built from the ground up with AI use cases in mind.

Emerging Layer 1s hope to ameliorate the following ills:

• Autonomous: Move on from “passive” smart contracts to active autonomous agents which can trigger transactions on-chain
• Integrated: Offer more seamless integrations to the off-chain resources required by AI workloads in data, compute, storage and APIs
• Attribution: Offer better tracking and attribution of crucial intelligence inputs – data, knowledge, models, compute – to better incentivize their onboarding

These L1s are effectively built on the premise that AI ecosystems need to be vertically integrated to be properly composable. In fact, Galadriel openly admits this in their litepaper:

“We started building Galadriel as a L2 but realized that vertical integration is necessary to enable fully decentralized & autonomous AI agents”

Source: Galadriel Litepaper

Many builders are starting with existing chains and looking to compensate for their short-comings in scalability and functionality with verifiable “co-processors”.

The emerging Alt-Layer 1 field is making the opposite bet: that a new Layer 1 – designed from the ground up – is the best architecture for coordinating the numerous inputs necessary for a thriving decentralized AI ecosystem.

Examples include: Talus Network, Galadriel, SaharaAI, Ambient

My own view is the community, liquidity, developer base, and proven decentralization of “the major” smart contract chains will be difficult to overcome. However, like Solana emerging from the pack of “alt-layer-1s” perhaps an AI-first upstart chain can also reach escape velocity. In my opinion, this is likely the most fruitful hunting ground to look for “the Solana” of the coming cycle. And yet, the graveyard will be full of also-rans.

More use-case specific Layer 1s are also emerging. Peaq – a Layer 1 blockchain designed to power DePIN networks – is one such example. It’s less of a traditional “Layer 1” than a specialized “IoT” chain that integrates with the others, providing “self-sovereign machine identities” which allow hardware – machines, robots, vehicles – to identify and connect with each other and other ecosystems.

The chain allows devs to configure machines and devices with specific access based on their role within the ecosystem verified by a combination of private key signatures, data pattern recognition, and oracles.

This could prove a crucial piece of “middleware” for the dataDAO thesis we discussed last month, allowing machine-to-machine payments and data feeds from DePIN networks integrated into the AI value chain, backed by proper on-chain incentives.

As of July 2024, peaq included over 850,000 machines, vehicles and robots / other devices

The OG Bid

Not to be outdone, the bid for “AI chain” bona fides is not only coming from upstarts but OGs as well.

[ICP](https://internetcomputer.org/docs/current/developer-docs/getting-started/overview-of-icp#:~:text=The Internet Computer Protocol (ICP,the security of decentralized execution.) is one of few chains with the throughput to allow AI models to run on-chain today. However, the project continues to be plagued by its poor initial launch and decentralization concerns, lagging the community and mind share of other Layer 1 ecosystems.

Near is another challenger who seems to be leaning more heavily into the community approach – kicking off an AI specific accelerator. In addition to the AI accelerator and scaling roadmap, Near is notable for its strong push into “account abstraction.” For a brief overview, please check out the “intents” and “chain abstraction” segments of our latest ETH CC report. In the declarative paradigm of intents and account abstraction, cross-chain orders are routed through specialized networks of solvers – largely market makers – who tap both on and off-chain liquidity and use other familiar tradfi liquidity tricks like netting out accumulative totals and “factoring” the remainder. Near and others aiming to usher in this new paradigm would effectively serve as account aggregators which is a privileged position to be in with the end user, but also surfaces some uncomfortable questions regarding value accrual to the underlying chains themselves:

If most future users liaise with blockchains through agents who largely use abstracted “intents” and solver networks to fill orders, then does the underlying blockspace start to become commoditized?

Do blockchains become primarily used only for settlement?

Is settlement enough to justify today’s valuations?

If not, what happens to the underlying security budget in proof of stake networks?

Will this effect their usage and builders willingness to build on them?

I don’t know the answers to the above questions, but they are worth considering.

The new and old Alt-1 ecosystems are making a hard push to be the home of AI on-chain, but many other projects remain committed to the leading smart contract ecosystems today – betting the liquidity, network effects, and security will ultimately prevail over new architectures; betting the power laws will continue unabated and the limitations in existing on-chain throughput can be solved by verifiability.

Verifiable Compute (& Privacy)

I almost see verifiable compute and privacy as two sides of the same coin: providing assurances to both the supply and demand side of data and models. Privacy protects data and model contributors from personal information leakage, IP infractions, and compliance faux paus. Verifiability provides blockchains with enhanced compute capacity while providing assurances to the demand side that they are getting what they asked for.

The nature of distributed systems like blockchains is that they provide certain properties like censorship resistance and immutability at the expense of others – namely performance.

If the purpose is decentralization and censorship resistance, then minimizing hardware requirements for nodes so as many users as possible can verify the blockchain’s integrity becomes an important attribute. In short, there are tradeoffs between hardware requirements, decentralization and performance.

Source: Florin Digital: Chapter 1: Verifiable Off-chain Compute

These constraints mean compute and bandwidth intensive activities like AI or other web2 scale applications are impossible on-chain today – assuming one would like to retain the decentralization properties which make blockchains valuable in the first place.

“Co-Processors” have emerged as an elegant solution to marry the censorship resistance and credibly neutral global computer with additional off-chain compute using various techniques which seek to minimize new trust requirements.

Think of these as verifiable oracles – for both compute and data – which can integrate web2 scale or AI applications with performance retarded blockchains.

This series by Florin Digital has many fantastic graphics which I will shamelessly use below but would recommend a detailed reading of the entire series:

Source: Florin Digital

To date, there have been four primary approaches to credibly bridging off-chain compute to the on-chain world:

Source: Florin Digital

Definitions:

zkML: combines zero-knowledge proofs with machine learning to ensure privacy and confidentiality of data and models, allowing verifiable computations without revealing certain underlying attributes

Source: Florin Digital

opML: leverages optimistic rollup principles to improve the scalability and efficiency of ML computations by verifying when disputes arise

Source: Florin Digital

TeeML: uses Trusted Execution Environments to securely execute ML computations, protecting data and models from tampering and unauthorized access

Source: Bagel: With Great Data Comes Great Responsibility II

Crypto-economic: instead of relying on cryptographic techniques to ensure verifiability, we can also use crypto-economic systems – similar to Ethereum’s Proof-of-Stake – which rely on accountability and financial incentives (e.g. AVS’ on EigenLayer)

Source: Florin Digital: Chapter 3

To date, there are a host of projects leveraging the above techniques to attack this problem (and often a combination of several):

And each technique has pro’s and con’s:

And, in reality, a combination of different techniques is likely to win out depending on the use case one is optimizing for and the trade-offs involved:

Source: Florin Digital

The initial enthusiasm around ZK-solutions has cooled somewhat primarily due to the cost vs. traditional compute. While the level of security is unmatched compared to the other techniques above, realistically we are ~5 years away from the cost reductions necessary make zk’s practical for many compute intensive use cases. TEEs have risen to fill the void. Despite potential vulnerabilities, TEEs are gaining mind share as a practical solution to both verifiability and privacy due to their cost effectiveness and readiness today.

Realistically, DeAI does not have the luxury of time as integrated solutions charge full steam ahead.

The Rise of TEEs

TEEs are secure and confidential computational environments within CPUs or GPUs that isolate code execution and data processing from the rest of the device – protecting the code from external influences even if the operating system or hardware environment is compromised.

Pros

• Confidential Execution: TEE ensures the data and model are protected from any external access or tampering
• Isolation: the TEE isolates the inference and training process from the rest of the system, ensuring security
• Verification: TEE provides attestation of untampered computing results
• Cost Effective: comparatively cheap, being only 20 – 30% more expensive compared to traditional compute (zk more like 1000x today)
• Flexible: More easily combined with other techniques like FHE, MPC, or ZKPs for additional layers of security

Cons:

• Requires trust in the manufacturer: in the attestation process, a certificate of the public key from the manufacturer is required
• Restricted resources: TEEs offer restricted computation resources inside their secure enclave leading to greater complexity
• Potential attack vectors: the ML process is still vulnerable. Previous research conducted by Grover et al. (2018) demonstrated methods to exploit access patterns in TEEs including “side-channel attacks” and other security breaches
• SDK support: most TEEs only provide basic low-level SDKs, and therefore make it hard to port all ML dependencies into TEEs

Looking at the above, the primary drawback in using TEEs in a “trustless” ecosystem like blockchains is a reliance on a centralized third-party. In this case, the manufacturer of the TEE.

However, this piece from David, Nukri, Rezo, and Jeremy makes a compelling case that TEEs, when combined with blockchain principles like open-source code, consensus mechanisms, smart contracts, and verifiable machine attestations, can achieve trustlessness while maintaining scalability, openness, and transparency.

Looking at scalability and cost effectiveness today, TeeML is the only one that is game-time ready.

Source: Omnida Substack

And when contemplating networks and adoption curves, being first to market can matter quite a bit.

To a meaningful degree, we already trust TEEs every day:

• TEEs are used in iPhones and Android devices, Signal messenger and even the popular hardware crypto wallet Ledger
• TEEs are now embedded in almost all CPUs and GPUs from major chip providers including Intel, Nvidia, AMD, ARM, Qualcomm, Samsung, IBM etc

Looking at several leading protocols like Super and Marlin, we can begin to see what the marriage of blockchain and TEE networks might entail. These protocols seek to combine smart contract governance and clustered TEE compute from a range of providers in an effort to “eliminate human administration from cloud governance.” Through the use of smart contract routing, these protocols combine TEEs to ensure the privacy of computation, attestations to verify the lack of tampering, and crypto rails for payment / rewards.

Source: Super Protocol Whitepaper

The end goal is a composable, scalable, trustless computing network at just a fraction above the cost of web2 cloud providers today.

Source: Omnida Substack

The network could enable a “decentralized hugging face” for sharing data, models and underpinning an “AI collaboration economy.”

Whether the final solution ends up being zk, op, Tee, crypto-economic or (more likely) some combination of the above, its clear co-processors will be essential to enabling web2 scale applications on-chain without sacrificing the decentralization which makes blockchains valuable in the first place.

Users have grown accustomed to a level of performance and personalization in the apps they use everyday which is only going to be super-charged as AI eats traditional software development.

Without co-processors, it is highly unlikely web3 applications will compete for mainstream users. For the trustlessness and composability to matter, you still need to meet a base level of UX and performance. Coprocessors are essential to this unlock, but they are not alone.

Privacy

If verifiability is crucial to induce demand, then privacy may prove just as critical in onboarding supply. Trillion-dollar industries with large data siloes are highly limited today when leveraging their data for AI training and inference due to privacy concerns. Without iron clad solutions, users and especially enterprises would be unlikely to contribute to open ecosystems, leaving the field fragmented, Big Tech licensing deals the only way to break down the data siloes.

For the definitive write up on the intersection of privacy and blockchains, I highly recommend Can Gurel’s “Everybody Needs ******“.

Given the breadth of this report, we will merely touch on the key highlights of privacy preserving machine learning (PPML) here and save a deep dive for a future report.

However, for those who can’t wait, I recommend Parts 1 and 2 of “With Great Data Comes Great Responsibility” from the guys over at Bagel summarized in the graphic below:

Source: Bagel: With Great Data Comes Great Responsibility II

From the diagram above, zkML and TeeML are the leading contenders in terms of verifiability while FHE and TEEs are the leading approaches to maximize privacy.

We have discussed the pro’s and con’s of TEEs, so lets briefly outline FHE:

Fully Homomorphic Encryption

FHE essentially permits computation on encrypted data, clearly useful in interacting with third parties during training or inference

Pros:

• Data Privacy: security guarantee that third-party access to data is prevented
• Model Privacy: as training and inference are carried out on private data, their is no need to publicize the neural net’s parameters
• Quantum Resistance: FHE encryption schemes are considered hard even for quantum computers

Cons:

• Verifiability: FHE does not offer proofs of correct encryption nor computation
• Speed & difficult to use: still quite slow and complex
• Memory requirements: the number of weights that need to be encrypted is proportional to the size of the network

Example projects: Zama, Fhenix, Inco

Similar to zkML being the “holy grail” in terms of verifiability, FHE is the current end goal on the privacy side but still suffers from performance and cost hurdles today. While mad scientists plug away at the frontiers of FHE, we should not let perfect be the enemy of good.

I am confident we will have cost effective ZK and FHE solutions in 3 – 5 years, but the modular stack requires useable components today. Protocols will need to be nimble, offering developers a range of solutions to choose from based on security, performance, and cost trade-offs for different use cases while being cognizant of the cost curves of potentially higher performing technologies.

The beauty of modular designs is that solutions can be swapped out as more effective versions become available.

Let 1000 flowers bloom.

The Crucible

In the near term, integrated approaches to manufacturing intelligence will continue to push out the frontier. The benefits of reduced friction in the value chain clearly outway the costs of coordination. For now.

However, we are already seeing early signs of fragmentation. Nvidia’s margins invite fierce competition. Single clusters will begin to run up against power constraints. Monolithic model architectures are compute inefficient, giving way to MoEs. The LLM explosion is already transforming routing into a hot area of research.

We still do not know how far the integrated approach can push performance. In many credible estimates, we still have a long way to go assuming the scaling laws hold…

Source: EPOCH AI: Can AI Scaling Continue through 2030?

However, we do know Big Tech will have to sprint to stay ahead of their rapidly decaying moats – especially in a world where Zuck is lighting their margins on fire.

If there is any slowdown at the frontier, modularity should gain share as absolute performance gives way to cost-performance tradeoffs for many use cases.

Whether blockchains have a role to play in this world will be a test of coordination and incentives. Can the longer tail of compute, data, and talent be assembled in a manner efficient and cost effective enough to provide an alternative tech stack from the more integrated approaches of Big Tech?

The pieces are not there today, but they are rapidly coalescing.

Software is eating the world. AI is eating software. AI is largely the combination of data, compute and talent. The entity or network which most effectively organizes these resources is set to own the 21st century.

Whether you or I have any role to play in this future may very well be decided by how efficiently the long-tail can coordinate. Distributed training, efficient routing, verifiability, privacy, and effective developer incentives may will decide the difference.

DeAI will live or die based on this test.

The test of composable compute.

‍