SolidML Onchain Inference

WARNING

SolidML and on-chain inference are currently only available on our alpha testnet. They are not yet available on the official testnet. For production LLM inference, see Secure LLM Inference which is available on the official testnet.

Overview

This guide walks you through how SolidML lets you run any ML or AI model directly from your smart contract thourgh the use of the OGInference precompile.

SolidML is tightly integrated with our Model Hub, making every model uploaded to the Hub instantly usable through SolidML. We've also made the process of uploading and using your model incredibly fast, allowing you to build AI-enabled apps without worrying about any of the underlying infrastructure.

Every inference executed through SolidML is natively secured by the OpenGradient blockchain validator nodes, allowing developers to focus on their application rather than complex verification techniques. Behind the scenes, OpenGradient relies on a range of inference verification techniques, such as ZKML, Trusted Execution Environments and cryptoeconomic security. This empowers developers to choose the most suitable methods for their use cases and requirements. To learn more about the security options we offer, go to ML Execution.

Inference Precompile

SolidML inference is provided through a Solidity interface (called OGInference) that any smart contract can call. The inference is implemented by a custom precompile on the OpenGradient network.

TIP

The SolidML inference precompile is accessible at 0x00000000000000000000000000000000000000F4.

The following function is exposed by OGInference:

• runModelInference: allows running inference of any generic ONNX model from the Model Hub.

ML Model Inferece

Function Definition

The runModel function is defined as follows:

interface OGInference {

    // security modes offered for ML inference
    enum ModelInferenceMode { VANILLA, ZKML, TEE }

    function runModelInference(
        ModelInferenceRequest memory request
    ) external returns (ModelOutput memory);
}

Calling this function from your smart contract will atomically execute the requested model with the given input and return the result synchronously.

Model Input and Output

The model input is defined as follows:

struct ModelInferenceRequest {
    ModelInferenceMode mode;
    string modelCID;
    ModelInput input;
}

Fields:

• mode: defines the inference execution and verification mode (ZK, TEE or VANILLA)
• modelCID: the Blob ID of the model to be inferenced. Can be retrieved from the Model Hub
• input: generic container for defining the model input

ModelInput and ModelOutput:

The input and output formats are implemented as generic tensor containers, providing flexibility to handle arrays of various shapes and dimensions for model inference. Each tensor has a unique name that must match up the expected ONNX input metadata and type. We currently support number and string input and output tensors.

TIP

Inspect your ONNX model metadata to find the model's input and output schema.

The definition of the input and output are as follows:

struct ModelInput {
    TensorLib.MultiDimensionalNumberTensor[] numbers;
    TensorLib.StringTensor[] strings;
}

struct ModelOutput {
    TensorLib.MultiDimensionalNumberTensor[] numbers;
    TensorLib.StringTensor[] strings;
    TensorLib.JsonScalar[] jsons;
    bool is_simulation_result; // indicates whether the result is part of simulation
}

Both the input and output consists of a list of number and string tensors. Number tensors can be multidimensional. The output also supports explicit JSON return types.

To read more about is_simulation_result in the output, please see Simulation Results.

NOTE

We use fixed-point representation numbers in the model input and output; see the TensorLib.Number type for more details. E.g., 1.52 is represented as TensorLib.Number{value = 152, decimals = 2}.

Full Definition:

enum ModelInferenceMode { VANILLA, ZK, TEE }

/**
 * Model inference request.
 */
struct ModelInferenceRequest {
    ModelInferenceMode mode;
    string modelCID;
    ModelInput input;
}

/**
 * Model input, made up of various tensors of either numbers or strings.
 */
struct ModelInput {
    TensorLib.MultiDimensionalNumberTensor[] numbers;
    TensorLib.StringTensor[] strings;
}

/**
 * Model output, made up of tensors of either numbers or strings, ordered
 * as defined by the model.
 *
 * For example, if a model's output is: [number_tensor_1, string_tensor_1, number_tensor_2],
 * you could access them like this:
 * number_tensor_1 = output.numbers[0];
 * string_tensor_1 = output.strings[0];
 * number_tensor_2 = output.numbers[1];
 */
struct ModelOutput {
    TensorLib.MultiDimensionalNumberTensor[] numbers;
    TensorLib.StringTensor[] strings;
    TensorLib.JsonScalar[] jsons;
    bool is_simulation_result;
}

TensorLib Reference

library TensorLib {

    /**
    * Can be used to represent a floating-point number or integer.
    *
    * eg 10 can be represented as Number(10, 0),
    * and 1.5 can be represented as Number(15, 1)
    */
    struct Number {
        int128 value;
        int128 decimals;
    }

    struct MultiDimensionalNumberTensor {
        string name;
        Number[] values;
        uint32[] shape;
    }

    struct StringTensor {
        string name;
        string[] values;
    }

    struct JsonScalar {
        string name;
        string value;
    }

Example ML Usage

The following smart contract uses SolidML to natively run a model from the Model Hub and persist its output.

import "opengradient-SolidML/src/OGInference.sol";

contract MlExample {

    // Execute an ML model from OpenGradient's model storage, secured by ZKML
    function runZkmlModel() public {
        // model takes 1 number tensor as input
        ModelInput memory modelInput = ModelInput(
            new NumberTensor[](1),
            new StringTensor[](0));

        // populate tensor
        Number[] memory numbers = new Number[](2);
        numbers[0] = Number(7286679744720459, 17); // 0.07286679744720459
        numbers[1] = Number(4486280083656311, 16); // 0.4486280083656311

        // set expected tensor name
        modelInput.numbers[0] = NumberTensor("input", numbers);

        // execute inference
        ModelOutput memory output = OGInference.runModelInference(
            ModelInferenceRequest(
                ModelInferenceMode.ZKML,
                "QmbbzDwqSxZSgkz1EbsNHp2mb67rYeUYHYWJ4wECE24S7A",
                modelInput
        ));

        // handle result
        if (output.is_simulation_result == false) {
            resultNumber = output.numbers[0].values[0];
        } else {
            resultNumber = Number(0, 0);
        }
    }
}