Inference Node

To make the usage of AI models as seamless as possible, we modified the Ethereum Virtual Machine (EVM) to add native inference capabilities. Smart contract developers can use simple function calls to run models and use their results immediately in the same transaction. You can read more about this in inference execution.

The OpenGradient's network operates with heterogeneous inference nodes, each with the sole responsibility of computing inference and generating any associated attestations or cryptographic proofs. These proofs are then propagated across the network to full nodes for validation, ensuring the integrity and security of the network.

Inference nodes on the network are stateless and do not execute stateful EVM transactions or compute the state-transition function; we achieve the division of stateful vs stateless compute via Validation-Computation Separation (VCS).

Validation-Computation Separation

Our unique VCS architecture and seamless integration allow OpenGradient to provide fully scalable, secure, and verifiable inference to any smart contract and user on our network.

Inference nodes on our network are beefy and optimized machines that provide valuable hardware resources such as GPUs and are directly responsible for downloading and executing model inference. These nodes operate independently of the blockchain nodes. Therefore, their computation does not have to be replicated on all validators, greatly reducing costs and latency. Model inferences are instead secured and verified through cryptographic and cryptoeconomic methods such as ZKML, TEE, and optimistic execution. You can read more about these in Inference Verification.

Thanks to the unique VCS architecture and the separation of concerns it provides, we can offer inference that is cost-effective and scalable. Our blockchain ensures full verification and security, combining the best of both worlds - scalable execution and decentralized verification.

Private Inference Nodes

In the future, OpenGradient will support the ability to run private models. A user needs to spin off a private inference node to run a private model. Private nodes must share a model ID and register on the network, stating their intent to run a particular private model.