Skip to main content
Ctrl+K
🦜🔗 LangChain  documentation - Home 🦜🔗 LangChain  documentation - Home
  • Reference
  • Legacy reference
Ctrl+K
Docs
  • GitHub
  • X / Twitter
Ctrl+K
  • Reference
  • Legacy reference
Docs
  • GitHub
  • X / Twitter

Section Navigation

Base packages

  • Core
  • Langchain
  • Text Splitters
  • Community
  • Experimental

Integrations

  • AI21
  • Airbyte
  • Anthropic
  • AstraDB
  • AWS
    • agents
    • chat_models
    • embeddings
    • function_calling
    • graphs
    • llms
    • retrievers
    • utilities
    • utils
      • enforce_stop_tokens
      • get_num_tokens_anthropic
      • get_token_ids_anthropic
    • vectorstores
  • Azure Dynamic Sessions
  • Box
  • Chroma
  • Cohere
  • Couchbase
  • Elasticsearch
  • Exa
  • Fireworks
  • Google Community
  • Google GenAI
  • Google VertexAI
  • Groq
  • Huggingface
  • Milvus
  • MistralAI
  • MongoDB
  • Nomic
  • Nvidia Ai Endpoints
  • Ollama
  • OpenAI
  • Pinecone
  • Postgres
  • Prompty
  • Qdrant
  • Robocorp
  • Together
  • Unstructured
  • VoyageAI
  • Weaviate
  • LangChain Python API Reference
  • langchain-aws: 0.1.18

utils#

Functions

utils.enforce_stop_tokens(text, stop)

Cut off the text as soon as any stop words occur.

utils.get_num_tokens_anthropic(text)

Get the number of tokens in a string of text.

utils.get_token_ids_anthropic(text)

Get the token ids for a string of text.

© Copyright 2023, LangChain Inc.