Do we need a new blockchain for generative AI?

Generative artificial intelligence (AI) has quickly become one of the hottest and arguably the most transformational technology trends of recent decades. The impact of generative AI is evident in all areas of the technology stack, from infrastructure to applications.

Since the release of ChatGPT and the subsequent GPT-4, the Web3 community has been speculating about the possible intersection of generative AI and Web3. While there are many obvious use cases such as conversation portfolios or language exploration, there are more advanced theorems worth exploring.

Jesus Rodriguez is the CEO of IntoTheBlock.

What if Generative AI Deserves Its Own Blockchain?

To analyze the viability of a generative AI blockchain, it is important to understand the current state of basic models, especially the emergence of open-source alternatives to API-based technology such as GPT-4, and the increasing concerns about centralized control of those foundation models.

Until a few months ago, the gap between API-based and open-source basic models was significant. Models such as OpenAI’s GPT-4, Anthropic’s Claude on language, DALL-E, and Midjourney on computer vision seemed significantly advanced compared to open-source alternatives. However, things began to change late last year with the surprise open-source release of Stable Diffusion, which provided a viable alternative to API-based text-to-image models. Despite this, large language models (LLMs) remained the centerpiece of generative AI, and in that domain open-source models paled in terms of quality to API-based alternatives.

Earlier this year, Meta AI Research published a paper introducing LLaMA, an LLM that matched the performance of GPT-3, but was significantly smaller. Initially, the model was not intended to be open-source, but something unexpected happened. A week after its publication, the model was leaked on 4chan and quickly downloaded by thousands of people. The LLaMA “accident” made a basic LLM available to everyone and created an unexpected momentum in open source innovation.

Shortly after the leak, new open-source foundation models with funny animal names started popping up all over the place. Stanford University released Alpaca, Databricks unveiled Dolly, Berkeley University open-source Koala, UC Berkeley, and Carnegie Mellon University collaborated on the release of Vicuna, Together announced the Red Pajama project, and the list goes on. Stable Diffusion and LLaMA have helped shift the scale of open-source generative AI and generated significant momentum. In addition, open-source base models are quickly closing the gap with commercial incumbents in terms of quality.

Another factor contributing to the emergence of a generative AI blockchain is the concern over the lack of transparency and centralized control of basic models. The size and complexity of the neural architectures that power basic models make exact interpretability almost impossible. As a result, the industry has to rely on intermediate steps, such as more open architectures and well-thought-out regulations. That a few centralized entities control the most powerful models on the market adds another concern regarding the feasibility of achieving true accountability, transparency, and interpretability in generative AI.

The combination of open-source innovation in base models and growing concerns about centralized control in the field creates a unique opportunity for Web3 architectures. The abundance of high-quality open source models reduces barriers to adoption in Web3 platforms. Resolving the transparency and control risks in generative AI is far from trivial, but there is little doubt that blockchain architectures hold key properties that can help in this area.

The explosion of innovation in open-source base models has significantly lowered the barrier to entry for Web3 platforms to integrate generative AI capabilities. The adoption of basic models in Web3 platforms can follow two fundamental and likely sequential paths:

In the first scenario, we are likely to witness tools such as exchanges, explorers, or wallets that include conversational capabilities powered by large language models. In addition, a new generation of DApps will be built with generative models as the cornerstone. In this scenario, Web3 primarily acts as a consumer of generative AI capabilities, with models running on traditional Web2 cloud infrastructures.

More intriguing alternatives emerge when considering Web3 platforms that can inherently support generative AI models. Imagine basic open-source models like LLaMA, Dolly or Alpaca running on nodes within a distributed blockchain. The ultimate realization of this vision is a blockchain designed specifically for generative AI.

The concept of a new blockchain optimized for a technology paradigm like generative AI may sound appealing, but it is undeniably controversial. After all, no new blockchains created solely for DeFi or NFTs have been created. What makes generative AI so different?

The answer lies in the architectural discrepancy between foundation model execution requirements and blockchain runtimes. A typical pre-trained base model consists of millions of neurons spanning tens of thousands of interconnected layers, running on clusters of GPUs or specialized deep learning hardware topologies. No smart contract in Web3 history comes close to that level of complexity. It is therefore logical to conclude that a new type of architecture is needed. Even Web2 infrastructures are evolving to support large-scale generative AI models, illustrating the magnitude of the required changes in Web3 architectures.

When considering a new blockchain for generative AI, the possibilities seem endless. But the simplest iteration of this idea should include a set of core capabilities. The ability to run nodes that run basic models is paramount for a blockchain dedicated to generative AI. The same goes for the ability to perform pre-training, fine-tuning, and inference workflows, the three primary stages in the base model lifecycle. Publishing and sharing datasets used for pre-training or refining models is also a desirable feature. Once we set up a blockchain runtime as a base layer, numerous transparency and interpretability capabilities can be enabled. For example, we can envision a proof-of-knowledge protocol that provides transparency regarding a model’s specific weights, validating that non-toxic or biased datasets were used for pretraining.

The concept of a specialized blockchain for generative AI is enticing, but is it really necessary? There is a valid value proposition in integrating generative AI capabilities into existing blockchain runtimes. However, the history of software shows a recurring trend of new architecture paradigms influencing infrastructure technologies. Recent trends such as cloud computing or big data serve as an example. Foundation models represent fundamentally different architecture paradigms that likely require more specialized blockchain infrastructures to operate effectively.

Moreover, we cannot overlook the potential of generative AI to transform the lower layers of the blockchain stack. It is not far-fetched to envision a proof-of-stake blockchain where validators process transactions based on natural language. Similarly, smart contracts could use language as the fundamental means of exchanging messages.

Generative AI has the potential to drive change across the entire blockchain stack. From this perspective, it seems logical to take a first-principles approach by enabling a new runtime with the flexibility to incorporate these changes.

Indeed, the idea of ​​a generative AI blockchain can be controversial and not without its challenges. However, I encourage exploring this idea using a negative argument.

What could happen if we fail to build new generative AI blockchains?

Currently, generative AI has created a significant technology gap between Web2 and Web3 architectures. This gap is widening due to the lack of native generative AI capabilities in Web3. Generative AI is reshaping fundamental aspects of software development, and new frameworks and platforms are rapidly appearing to support this paradigm shift.

Developing native generative AI capabilities is nothing short of an existential challenge for Web3, as it is critical to enabling new waves of innovation in the field. A native generative AI blockchain represents just one of many approaches that can facilitate this transition into the world of foundation models. Building a new blockchain presents numerous challenges, but the rapid evolution of L2 runtimes, platforms such as Cosmos, and the emergence of powerful L1 ecosystems such as Aptos or Sui make the possibility of a generative AI blockchain much more viable than in previous years.