OpenAI-Compatible LLM Inference API

Alpha Release

🚧 Note: This project is currently in alpha stage

Overview

OpenGradient provides a set of OpenAI compatible endpoints for accessing verifiable LLM inference from traditional applications, such as agents.

NOTE

The OpenGradient LLM endpoints are currently in Alpha release.

We currently support the following endpoints:

• /v1/chat/completions: Chat completions, including tool calling (API Reference)
• /v1/completions: Completions (API Reference)

You can check our supported LLMs page to see all the models we support through the API.

These endpoints are a proxy for the OpenGradient blockchain - every inference request is sent to the OpenGradient network and has the same verifiable and trustless execution properties as sending a inference transaction request directly to the network.

Every inference is traceable in our Block Explorer.

Usage

The OpenGradient LLM endpoints are accessible through https://llm.opengradient.ai.

For example, to run an inference on Qwen/Qwen2.5-72B-Instruct from the Model Hub, you can use the following command:

curl -XPOST https://llm.opengradient.ai/v1/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer ${PRIVATE_KEY}" \
    -d '{"model": "Qwen/Qwen2.5-72B-Instruct", "prompt": "Who are you?"}'

TIP

Your OpenGradient private key should be exported to the PRIVATE_KEY environment variable.

In order to use OpenGradient's verifiable LLM in any application that uses OpenAI's client library, you can simply replace the URL of the OpenAI client. For example, using the OpenAI Python library, you can switch to OpenGradient using the following client config:

from openai import OpenAI

# initialize with OpenGradient
client = OpenAI(
    api_key=os.environ.get("OPENGRADIENT_KEY"),
    base_url='https://llm.opengradient.ai/v1'
)

# use as expected
chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="Qwen/Qwen2.5-72B-Instruct",
)

Upcoming Features

• Restricted API Keys for inference
• TEE secured API endpoint