Google Colab 安装 Ollama

安装

使用开源项目

https://github.com/AITwinMinds/Ollama-in-Google-Colab

安装依赖

!sudo apt-get install -y pciutils

运行安装脚本

!curl https://ollama.ai/install.sh | sh

定义 Ollama 的函数

import os
import threading
import subprocess
import requests
import json

def ollama():
    os.environ['OLLAMA_HOST'] = '0.0.0.0:11434'
    os.environ['OLLAMA_ORIGINS'] = '*'
    subprocess.Popen(["ollama", "serve"])

使用

接下来就能够像本地一样使用 ollama

运行 ollama 服务

ollama_thread = threading.Thread(target=ollama)
ollama_thread.start()

拉取模型

!ollama run llama3.1

需要重新启动 ollama 服务

每次拉取新模型后，都需要重新启动一下 ollama 服务

ollama_thread = threading.Thread(target=ollama)
ollama_thread.start()

Rest API 调用

import requests

prompt = """
What is AI?
Can you explain in three paragraphs?
"""

url = 'http://localhost:11434/api/chat'
payload = {
    "model": "llama3.1",
    "temperature": 0.6,
    "stream": False,
    "messages": [
        {"role": "system", "content": "You are an AI assistant!"},
        {"role": "user", "content": prompt}
    ]
}

response = requests.post(url, json=payload)
message_str = response.content.decode('utf-8')
message_dict = json.loads(message_str)
print(message_dict['message']['content'])

Ollama Python

使用 ollama 官方库

!pip install ollama

聊天

import ollama
response = ollama.chat(model='llama3.1', messages=[
  {
    'role': 'user',
    'content': 'Why is the sky blue?',
  },
])
print(response['message']['content'])