Skip to main content


Petals runs 100B+ language models at home, BitTorrent-style.

This notebook goes over how to use Langchain with Petals.

Install petalsโ€‹

The petals package is required to use the Petals API. Install petals using pip3 install petals.

For Apple Silicon(M1/M2) users please follow this guide to install petals

!pip3 install petals


import os

from langchain.chains import LLMChain
from langchain_community.llms import Petals
from langchain_core.prompts import PromptTemplate

Set the Environment API Keyโ€‹

Make sure to get your API key from Huggingface.

from getpass import getpass


Create the Petals instanceโ€‹

You can specify different parameters such as the model name, max new tokens, temperature, etc.

# this can take several minutes to download big files!

llm = Petals(model_name="bigscience/bloom-petals")
Downloading:   1%|โ–                        | 40.8M/7.19G [00:24<15:44, 7.57MB/s]

Create a Prompt Templateโ€‹

We will create a prompt template for Question and Answer.

template = """Question: {question}

Answer: Let's think step by step."""

prompt = PromptTemplate.from_template(template)

Initiate the LLMChainโ€‹

llm_chain = LLMChain(prompt=prompt, llm=llm)

Run the LLMChainโ€‹

Provide a question and run the LLMChain.

question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"

Help us out by providing feedback on this documentation page: