OpenGradient Docs

Appearance

Python SDK Tutorial ​

Here is a simple tutorial on how to use the OpenGradient Python SDK. This tutorial will guide you through the installation process and show you some basic functionalities of the Python SDK.

Prerequisites ​

Before you begin, make sure you have the following:

  • Python is installed on your system; we support versions 3.10, 3.11, or 3.12.
  • pip (Python package installer) is available.

Make sure Python and pip are installed in the same environment.

NOTE

Windows Users: temporarily enable WSL when installing opengradient, fix in progress.

Set Up Accounts ​

To use OpenGradient's Python SDK, you'll need to create 2 accounts:

  • Model Hub account: You can create this using your email and password on the Hub Sign Up.
  • OpenGradient account: You will receive verifiable inference transactions on this blockchain account on the OpenGradient devnet. You can use any existing Ethereum-compatible wallet with an account (e.g., MetaMask) or create a new one using our SDK. See below.

We provide an account creation wizard in our SDK that guides you through this process. You can access it by running:

bash
opengradient config init

After you complete the accounts' set up, you can check your details by running the following command:

bash
opengradient config show

Import and Initialise the OpenGradient Python SDK ​

After installation, you can start using the SDK by importing it in your Python script:

python
import opengradient as og

og.init(email="<email>", password="<password>", private_key="<private_key>")

Using the SDK for Basic Operations ​

Here are some examples of what you can do with the OpenGradient SDK.

Example 1: Run an Inference ​

python
import opengradient as og

# initialize SDK
og.init(email="<email>", password="<password>", private_key="<private_key>")

# run inference
tx_hash, model_output = og.infer(
    model_cid='QmbUqS93oc4JTLMHwpVxsE39mhNxy6hpf6Py3r9oANr8aZ',
    model_input={
        "num_input1": [1.0, 2.0, 3.0],
        "num_input2": 10,
        "str_input1": np.array(["hello", "ONNX"]),
        "str_input2": " world"
    },
    inference_mode=og.InferenceMode.VANILLA
)

# print output
print(model_output)

After running this inference, you will see the response:

bash
{
    'num_output1': array([11., 12., 13.], dtype=float32), 
    'num_output2': array([10.], dtype=float32), 
    'str_output1': array(['hello', 'ONNX', ' world'], dtype='<U6'),
    'str_output2': array([' world'], dtype='<U6')
}

Example 2: Run LLM ​

python
import opengradient as og

# initialize SDK
og.init(email="<email>", password="<password>", private_key="<private_key>")

# run LLM inference
tx_hash, response = og.infer_llm(
    model_cid='meta-llama/Meta-Llama-3-8B-Instruct',
    prompt="Translate the following English text to French: 'Hello, how are you?'",
    max_tokens=50,
    temperature=0.0
)

# print output
print("Transaction Hash:", tx_hash)
print("LLM Output:", response)

After running this LLM, you will get the following response:

bash
LLM Output: .
The translation of the English text 'Hello, how are you?' to French is: 'Bonjour, comment allez-vous?'.
Here's a breakdown of the translation:
* 'Hello' is translated to 'Bonjour', which is a common way

Conclusion ​

With these easy steps, you should be able to get started using the OpenGradient Python SDK to run inferences and LLMs.

OpenGradient 2024