Skip to main content

SambaNovaCloud

SambaNova's SambaNova Cloud is a platform for performing inference with open-source models

caution

You are currently on a page documenting the use of SambaNovaCloud models as text completion models. We recommend you to use the chat completion models.

You may be looking for SambaNovaCloud Chat Models .

Overview​

Integration details​

ClassPackageLocalSerializableJS supportPackage downloadsPackage latest
SambaNovaCloudlangchain_community❌beta❌PyPI - DownloadsPyPI - Version

This example goes over how to use LangChain to interact with SambaNovaCloud models

Setup​

Credentials​

To access ChatSambaNovaCloud models you will need to create a SambaNovaCloud account, get an API key and set it as the SAMBANOVA_API_KEY environment variable:

import getpass
import os

if "SAMBANOVA_API_KEY" not in os.environ:
    os.environ["SAMBANOVA_API_KEY"] = getpass.getpass()

Installation​

The integration lives in the langchain-community package. We also need to install the sseclient-py package this is required to run streaming predictions

%pip install --quiet -U langchain-community sseclient-py

Instantiation​

from langchain_community.llms.sambanova import SambaNovaCloud

llm = SambaNovaCloud(
    model="Meta-Llama-3.1-70B-Instruct",
    max_tokens_to_generate=1000,
    temperature=0.01,
    # top_k = 50,
    # top_p = 1.0
)
API Reference:SambaNovaCloud

Invocation​

Now we can instantiate our model object and generate chat completions:

input_text = "Why should I use open source models?"

completion = llm.invoke(input_text)
completion
"**Advantages of Open Source Models**\n\nUsing open source models can bring numerous benefits to your project or organization. Here are some reasons why you should consider using open source models:\n\n### 1. **Cost-Effective**\n\nOpen source models are free to use, modify, and distribute. This can significantly reduce the costs associated with developing and maintaining proprietary models.\n\n### 2. **Community Support**\n\nOpen source models are often maintained by a community of developers and users who contribute to their improvement. This community support can lead to faster bug fixes, new feature additions, and better documentation.\n\n### 3. **Transparency and Customizability**\n\nOpen source models provide complete transparency into their architecture and implementation. This allows you to customize and fine-tune the model to suit your specific needs.\n\n### 4. **Faster Development**\n\nBy leveraging pre-trained open source models, you can accelerate your development process. You can focus on fine-tuning the model for your specific use case rather than building one from scratch.\n\n### 5. **Improved Security**\n\nOpen source models are often reviewed and audited by a large community of developers, which can help identify and fix security vulnerabilities.\n\n### 6. **Interoperability**\n\nOpen source models can be easily integrated with other open source tools and frameworks, promoting interoperability and reducing vendor lock-in.\n\n### 7. **Access to State-of-the-Art Technology**\n\nMany open source models are developed by top researchers and institutions, providing access to state-of-the-art technology and techniques.\n\n### Example Use Cases\n\n* **Computer Vision**: Use open source models like TensorFlow's Object Detection API or OpenCV's pre-trained models for image classification, object detection, and segmentation tasks.\n* **Natural Language Processing**: Leverage open source models like spaCy or Stanford CoreNLP for text processing, sentiment analysis, and language translation tasks.\n* **Speech Recognition**: Utilize open source models like Kaldi or Mozilla's DeepSpeech for speech-to-text applications.\n\n**Getting Started**\n\nTo get started with open source models, explore popular repositories on GitHub or model hubs like TensorFlow Hub or PyTorch Hub. Familiarize yourself with the model's documentation, and experiment with pre-trained models before fine-tuning them for your specific use case.\n\nBy embracing open source models, you can accelerate your development process, reduce costs, and tap into the collective knowledge of the developer community."
# Streaming response
for chunk in llm.stream("Why should I use open source models?"):
    print(chunk, end="", flush=True)
**Advantages of Open Source Models**

Using open source models can bring numerous benefits to your projects. Here are some reasons why you should consider them:

### 1. **Cost-Effective**

Open source models are free to use, modify, and distribute. This can significantly reduce the costs associated with developing and maintaining proprietary models.

### 2. **Community Support**

Open source models are often maintained by a community of developers, researchers, and users. This community can provide support, fix bugs, and contribute to the model's improvement.

### 3. **Transparency and Reproducibility**

Open source models are transparent in their architecture, training data, and hyperparameters. This transparency allows for reproducibility, which is essential in scientific research and development.

### 4. **Customizability**

Open source models can be modified to suit specific use cases or requirements. This customizability enables developers to adapt the model to their needs, which can lead to better performance and accuracy.

### 5. **Faster Development**

Using open source models can accelerate development by providing a pre-trained foundation. This allows developers to focus on fine-tuning the model for their specific task, rather than starting from scratch.

### 6. **Access to State-of-the-Art Models**

Open source models often represent the state-of-the-art in their respective domains. By using these models, developers can leverage the latest advancements in AI research.

### 7. **Reduced Vendor Lock-in**

Open source models are not tied to a specific vendor or platform. This reduces the risk of vendor lock-in and allows developers to switch to alternative solutions if needed.

### Example Use Cases

* **Computer Vision**: Using open source models like YOLO (You Only Look Once) or SSD (Single Shot Detector) for object detection tasks.
* **Natural Language Processing**: Leveraging open source models like BERT (Bidirectional Encoder Representations from Transformers) or RoBERTa (Robustly Optimized BERT Pretraining Approach) for text classification, sentiment analysis, or language translation.
* **Speech Recognition**: Utilizing open source models like Kaldi or Mozilla DeepSpeech for speech-to-text applications.

**Getting Started**

To get started with open source models, you can explore popular repositories on GitHub or model hubs like the TensorFlow Model Garden or the PyTorch Model Zoo. These resources provide pre-trained models, documentation, and tutorials to help you integrate open source models into your projects.

By embracing open source models, you can tap into the collective knowledge and expertise of the developer community, accelerate your development process, and create more accurate and efficient AI solutions.

Chaining​

We can chain our completion model with a prompt template like so:

from langchain_core.prompts import PromptTemplate

prompt = PromptTemplate.from_template("How to say {input} in {output_language}:\n")

chain = prompt | llm
chain.invoke(
    {
        "output_language": "German",
        "input": "I love programming.",
    }
)
API Reference:PromptTemplate
'The translation of "I love programming" in German is:\n\n"Ich liebe das Programmieren."\n\nHere\'s a breakdown of the sentence:\n\n* "Ich" means "I"\n* "liebe" is the verb "to love" in the first person singular (I love)\n* "das" is the definite article for "Programmieren" (programming)\n* "Programmieren" is the verb "to program" in the infinitive form, but in this context, it\'s used as a noun to refer to the activity of programming.\n\nSo, "Ich liebe das Programmieren" is a common way to express your passion for programming in German.'

API reference​

For detailed documentation of all SambaNovaCloud llm features and configurations head to the API reference: https://python.langchain.com/api_reference/community/llms/langchain_community.llms.sambanova.SambaNovaCloud.html

Was this page helpful?