LLMs#
Wrappers on top of large language models APIs.
- pydantic model langchain.llms.AI21[source]#
Wrapper around AI21 large language models.
To use, you should have the environment variable
AI21_API_KEY
set with your API key.Example
from langchain.llms import AI21 ai21 = AI21(model="j2-jumbo-instruct")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field base_url: Optional[str] = None#
Base url to use, if None decides based on model name.
- field countPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)#
Penalizes repeated tokens according to count.
- field frequencyPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)#
Penalizes repeated tokens according to frequency.
- field logitBias: Optional[Dict[str, float]] = None#
Adjust the probability of specific tokens being generated.
- field maxTokens: int = 256#
The maximum number of tokens to generate in the completion.
- field minTokens: int = 0#
The minimum number of tokens to generate in the completion.
- field model: str = 'j2-jumbo-instruct'#
Model name to use.
- field numResults: int = 1#
How many completions to generate for each prompt.
- field presencePenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)#
Penalizes repeated tokens.
- field temperature: float = 0.7#
What sampling temperature to use.
- field topP: float = 1.0#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.AlephAlpha[source]#
Wrapper around Aleph Alpha large language models.
To use, you should have the
aleph_alpha_client
python package installed, and the environment variableALEPH_ALPHA_API_KEY
set with your API key, or pass it as a named parameter to the constructor.Parameters are explained more in depth here: Aleph-Alpha/aleph-alpha-client
Example
from langchain.llms import AlephAlpha alpeh_alpha = AlephAlpha(aleph_alpha_api_key="my-api-key")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field aleph_alpha_api_key: Optional[str] = None#
API key for Aleph Alpha API.
- field best_of: Optional[int] = None#
returns the one with the “best of” results (highest log probability per token)
- field completion_bias_exclusion_first_token_only: bool = False#
Only consider the first token for the completion_bias_exclusion.
- field contextual_control_threshold: Optional[float] = None#
If set to None, attention control parameters only apply to those tokens that have explicitly been set in the request. If set to a non-None value, control parameters are also applied to similar tokens.
- field control_log_additive: Optional[bool] = True#
True: apply control by adding the log(control_factor) to attention scores. False: (attention_scores - - attention_scores.min(-1)) * control_factor
- field echo: bool = False#
Echo the prompt in the completion.
- field frequency_penalty: float = 0.0#
Penalizes repeated tokens according to frequency.
- field log_probs: Optional[int] = None#
Number of top log probabilities to be returned for each generated token.
- field logit_bias: Optional[Dict[int, float]] = None#
The logit bias allows to influence the likelihood of generating tokens.
- field maximum_tokens: int = 64#
The maximum number of tokens to be generated.
- field minimum_tokens: Optional[int] = 0#
Generate at least this number of tokens.
- field model: Optional[str] = 'luminous-base'#
Model name to use.
- field n: int = 1#
How many completions to generate for each prompt.
- field penalty_bias: Optional[str] = None#
Penalty bias for the completion.
- field penalty_exceptions: Optional[List[str]] = None#
List of strings that may be generated without penalty, regardless of other penalty settings
- field penalty_exceptions_include_stop_sequences: Optional[bool] = None#
Should stop_sequences be included in penalty_exceptions.
- field presence_penalty: float = 0.0#
Penalizes repeated tokens.
- field raw_completion: bool = False#
Force the raw completion of the model to be returned.
- field repetition_penalties_include_completion: bool = True#
Flag deciding whether presence penalty or frequency penalty are updated from the completion.
- field repetition_penalties_include_prompt: Optional[bool] = False#
Flag deciding whether presence penalty or frequency penalty are updated from the prompt.
- field stop_sequences: Optional[List[str]] = None#
Stop sequences to use.
- field temperature: float = 0.0#
A non-negative float that tunes the degree of randomness in generation.
- field tokens: Optional[bool] = False#
return tokens of completion.
- field top_k: int = 0#
Number of most likely tokens to consider at each step.
- field top_p: float = 0.0#
Total probability mass of tokens to consider at each step.
- field use_multiplicative_presence_penalty: Optional[bool] = False#
Flag deciding whether presence penalty is applied multiplicatively (True) or additively (False).
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Anthropic[source]#
Wrapper around Anthropic large language models.
To use, you should have the
anthropic
python package installed, and the environment variableANTHROPIC_API_KEY
set with your API key, or pass it as a named parameter to the constructor.Example
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field max_tokens_to_sample: int = 256#
Denotes the number of tokens to predict per generation.
- field model: str = 'claude-v1'#
Model name to use.
- field streaming: bool = False#
Whether to stream the results.
- field temperature: float = 1.0#
A non-negative float that tunes the degree of randomness in generation.
- field top_k: int = 0#
Number of most likely tokens to consider at each step.
- field top_p: float = 1#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- stream(prompt: str, stop: Optional[List[str]] = None) Generator [source]#
Call Anthropic completion_stream and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.
- Parameters
prompt – The prompt to pass into the model.
stop – Optional list of stop words to use when generating.
- Returns
A generator representing the stream of tokens from Anthropic.
Example
prompt = "Write a poem about a stream." prompt = f"\n\nHuman: {prompt}\n\nAssistant:" generator = anthropic.stream(prompt) for token in generator: yield token
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.AzureOpenAI[source]#
Azure specific OpenAI class that uses deployment name.
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field batch_size: int = 20#
Batch size to use when passing multiple documents to generate.
- field best_of: int = 1#
Generates best_of completions server-side and returns the “best”.
- field deployment_name: str = ''#
Deployment name to use.
- field frequency_penalty: float = 0#
Penalizes repeated tokens according to frequency.
- field logit_bias: Optional[Dict[str, float]] [Optional]#
Adjust the probability of specific tokens being generated.
- field max_retries: int = 6#
Maximum number of retries to make when generating.
- field max_tokens: int = 256#
The maximum number of tokens to generate in the completion. -1 returns as many tokens as possible given the prompt and the models maximal context size.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- field model_name: str = 'text-davinci-003'#
Model name to use.
- field n: int = 1#
How many completions to generate for each prompt.
- field presence_penalty: float = 0#
Penalizes repeated tokens.
- field request_timeout: Optional[Union[float, Tuple[float, float]]] = None#
Timeout for requests to OpenAI completion API. Default is 600 seconds.
- field streaming: bool = False#
Whether to stream the results or not.
- field temperature: float = 0.7#
What sampling temperature to use.
- field top_p: float = 1#
Total probability mass of tokens to consider at each step.
- field verbose: bool [Optional]#
Whether to print out response text.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) langchain.schema.LLMResult #
Create the LLMResult from the choices and prompts.
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Calculate num tokens with tiktoken package.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) List[List[str]] #
Get the sub prompts for llm call.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- max_tokens_for_prompt(prompt: str) int #
Calculate the maximum number of tokens possible to generate for a prompt.
- Parameters
prompt – The prompt to pass into the model.
- Returns
The maximum number of tokens to generate for a prompt.
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
- modelname_to_contextsize(modelname: str) int #
Calculate the maximum number of tokens possible to generate for a model.
text-davinci-003: 4,097 tokens text-curie-001: 2,048 tokens text-babbage-001: 2,048 tokens text-ada-001: 2,048 tokens code-davinci-002: 8,000 tokens code-cushman-001: 2,048 tokens
- Parameters
modelname – The modelname we want to know the context size for.
- Returns
The maximum context size
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
- prep_streaming_params(stop: Optional[List[str]] = None) Dict[str, Any] #
Prepare the params for streaming.
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- stream(prompt: str, stop: Optional[List[str]] = None) Generator #
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.
- Parameters
prompt – The prompts to pass into the model.
stop – Optional list of stop words to use when generating.
- Returns
A generator representing the stream of tokens from OpenAI.
Example
generator = openai.stream("Tell me a joke.") for token in generator: yield token
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Banana[source]#
Wrapper around Banana large language models.
To use, you should have the
banana-dev
python package installed, and the environment variableBANANA_API_KEY
set with your API key.Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field model_key: str = ''#
model endpoint to use
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.CerebriumAI[source]#
Wrapper around CerebriumAI large language models.
To use, you should have the
cerebrium
python package installed, and the environment variableCEREBRIUMAI_API_KEY
set with your API key.Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field endpoint_url: str = ''#
model endpoint to use
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Cohere[source]#
Wrapper around Cohere large language models.
To use, you should have the
cohere
python package installed, and the environment variableCOHERE_API_KEY
set with your API key, or pass it as a named parameter to the constructor.Example
from langchain.llms import Cohere cohere = Cohere(model="gptd-instruct-tft", cohere_api_key="my-api-key")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field frequency_penalty: float = 0.0#
Penalizes repeated tokens according to frequency. Between 0 and 1.
- field k: int = 0#
Number of most likely tokens to consider at each step.
- field max_tokens: int = 256#
Denotes the number of tokens to predict per generation.
- field model: Optional[str] = None#
Model name to use.
- field p: int = 1#
Total probability mass of tokens to consider at each step.
- field presence_penalty: float = 0.0#
Penalizes repeated tokens. Between 0 and 1.
- field temperature: float = 0.75#
A non-negative float that tunes the degree of randomness in generation.
- field truncate: Optional[str] = None#
Specify how the client handles inputs longer than the maximum token length: Truncate from START, END or NONE
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.DeepInfra[source]#
Wrapper around DeepInfra deployed models.
To use, you should have the
requests
python package installed, and the environment variableDEEPINFRA_API_TOKEN
set with your API token, or pass it as a named parameter to the constructor.Only supports text-generation and text2text-generation for now.
Example
from langchain.llms import DeepInfra di = DeepInfra(model_id="google/flan-t5-xl", deepinfra_api_token="my-api-key")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.ForefrontAI[source]#
Wrapper around ForefrontAI large language models.
To use, you should have the environment variable
FOREFRONTAI_API_KEY
set with your API key.Example
from langchain.llms import ForefrontAI forefrontai = ForefrontAI(endpoint_url="")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field base_url: Optional[str] = None#
Base url to use, if None decides based on model name.
- field endpoint_url: str = ''#
Model name to use.
- field length: int = 256#
The maximum number of tokens to generate in the completion.
- field repetition_penalty: int = 1#
Penalizes repeated tokens according to frequency.
- field temperature: float = 0.7#
What sampling temperature to use.
- field top_k: int = 40#
The number of highest probability vocabulary tokens to keep for top-k-filtering.
- field top_p: float = 1.0#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.GooseAI[source]#
Wrapper around OpenAI large language models.
To use, you should have the
openai
python package installed, and the environment variableGOOSEAI_API_KEY
set with your API key.Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field frequency_penalty: float = 0#
Penalizes repeated tokens according to frequency.
- field logit_bias: Optional[Dict[str, float]] [Optional]#
Adjust the probability of specific tokens being generated.
- field max_tokens: int = 256#
The maximum number of tokens to generate in the completion. -1 returns as many tokens as possible given the prompt and the models maximal context size.
- field min_tokens: int = 1#
The minimum number of tokens to generate in the completion.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- field model_name: str = 'gpt-neo-20b'#
Model name to use
- field n: int = 1#
How many completions to generate for each prompt.
- field presence_penalty: float = 0#
Penalizes repeated tokens.
- field temperature: float = 0.7#
What sampling temperature to use
- field top_p: float = 1#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.HuggingFaceEndpoint[source]#
Wrapper around HuggingFaceHub Inference Endpoints.
To use, you should have the
huggingface_hub
python package installed, and the environment variableHUGGINGFACEHUB_API_TOKEN
set with your API token, or pass it as a named parameter to the constructor.Only supports text-generation and text2text-generation for now.
Example
from langchain.llms import HuggingFaceEndpoint endpoint_url = ( "https://abcdefghijklmnop.us-east-1.aws.endpoints.huggingface.cloud" ) hf = HuggingFaceEndpoint( endpoint_url=endpoint_url, huggingfacehub_api_token="my-api-key" )
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field endpoint_url: str = ''#
Endpoint URL to use.
- field model_kwargs: Optional[dict] = None#
Key word arguments to pass to the model.
- field task: Optional[str] = None#
Task to call the model with. Should be a task that returns generated_text.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.HuggingFaceHub[source]#
Wrapper around HuggingFaceHub models.
To use, you should have the
huggingface_hub
python package installed, and the environment variableHUGGINGFACEHUB_API_TOKEN
set with your API token, or pass it as a named parameter to the constructor.Only supports text-generation and text2text-generation for now.
Example
from langchain.llms import HuggingFaceHub hf = HuggingFaceHub(repo_id="gpt2", huggingfacehub_api_token="my-api-key")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field model_kwargs: Optional[dict] = None#
Key word arguments to pass to the model.
- field repo_id: str = 'gpt2'#
Model name to use.
- field task: Optional[str] = None#
Task to call the model with. Should be a task that returns generated_text.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.HuggingFacePipeline[source]#
Wrapper around HuggingFace Pipeline API.
To use, you should have the
transformers
python package installed.Only supports text-generation and text2text-generation for now.
- Example using from_model_id:
from langchain.llms import HuggingFacePipeline hf = HuggingFacePipeline.from_model_id( model_id="gpt2", task="text-generation" )
- Example passing pipeline in directly:
from langchain.llms import HuggingFacePipeline from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline model_id = "gpt2" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id) pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=10 ) hf = HuggingFacePipeline(pipeline=pipe)
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
- field model_id: str = 'gpt2'#
Model name to use.
- field model_kwargs: Optional[dict] = None#
Key word arguments to pass to the model.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- classmethod from_model_id(model_id: str, task: str, device: int = - 1, model_kwargs: Optional[dict] = None, **kwargs: Any) langchain.llms.base.LLM [source]#
Construct the pipeline object from model_id and task.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Modal[source]#
Wrapper around Modal large language models.
To use, you should have the
modal-client
python package installed.Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
- field endpoint_url: str = ''#
model endpoint to use
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.NLPCloud[source]#
Wrapper around NLPCloud large language models.
To use, you should have the
nlpcloud
python package installed, and the environment variableNLPCLOUD_API_KEY
set with your API key.Example
from langchain.llms import NLPCloud nlpcloud = NLPCloud(model="gpt-neox-20b")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field bad_words: List[str] = []#
List of tokens not allowed to be generated.
- field do_sample: bool = True#
Whether to use sampling (True) or greedy decoding.
- field early_stopping: bool = False#
Whether to stop beam search at num_beams sentences.
- field length_no_input: bool = True#
Whether min_length and max_length should include the length of the input.
- field length_penalty: float = 1.0#
Exponential penalty to the length.
- field max_length: int = 256#
The maximum number of tokens to generate in the completion.
- field min_length: int = 1#
The minimum number of tokens to generate in the completion.
- field model_name: str = 'finetuned-gpt-neox-20b'#
Model name to use.
- field num_beams: int = 1#
Number of beams for beam search.
- field num_return_sequences: int = 1#
How many completions to generate for each prompt.
- field remove_end_sequence: bool = True#
Whether or not to remove the end sequence token.
- field remove_input: bool = True#
Remove input text from API response
- field repetition_penalty: float = 1.0#
Penalizes repeated tokens. 1.0 means no penalty.
- field temperature: float = 0.7#
What sampling temperature to use.
- field top_k: int = 50#
The number of highest probability tokens to keep for top-k filtering.
- field top_p: int = 1#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.OpenAI[source]#
Generic OpenAI class that uses model name.
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field verbose: bool [Optional]#
Whether to print out response text.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) langchain.schema.LLMResult #
Create the LLMResult from the choices and prompts.
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Calculate num tokens with tiktoken package.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) List[List[str]] #
Get the sub prompts for llm call.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- max_tokens_for_prompt(prompt: str) int #
Calculate the maximum number of tokens possible to generate for a prompt.
- Parameters
prompt – The prompt to pass into the model.
- Returns
The maximum number of tokens to generate for a prompt.
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
- modelname_to_contextsize(modelname: str) int #
Calculate the maximum number of tokens possible to generate for a model.
text-davinci-003: 4,097 tokens text-curie-001: 2,048 tokens text-babbage-001: 2,048 tokens text-ada-001: 2,048 tokens code-davinci-002: 8,000 tokens code-cushman-001: 2,048 tokens
- Parameters
modelname – The modelname we want to know the context size for.
- Returns
The maximum context size
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
- prep_streaming_params(stop: Optional[List[str]] = None) Dict[str, Any] #
Prepare the params for streaming.
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- stream(prompt: str, stop: Optional[List[str]] = None) Generator #
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.
- Parameters
prompt – The prompts to pass into the model.
stop – Optional list of stop words to use when generating.
- Returns
A generator representing the stream of tokens from OpenAI.
Example
generator = openai.stream("Tell me a joke.") for token in generator: yield token
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.OpenAIChat[source]#
Wrapper around OpenAI Chat large language models.
To use, you should have the
openai
python package installed, and the environment variableOPENAI_API_KEY
set with your API key.Any parameters that are valid to be passed to the openai.create call can be passed in, even if not explicitly saved on this class.
Example
from langchain.llms import OpenAIChat openaichat = OpenAIChat(model_name="gpt-3.5-turbo")
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field max_retries: int = 6#
Maximum number of retries to make when generating.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- field model_name: str = 'gpt-3.5-turbo'#
Model name to use.
- field prefix_messages: List [Optional]#
Series of messages for Chat input.
- field streaming: bool = False#
Whether to stream the results or not.
- field verbose: bool [Optional]#
Whether to print out response text.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Petals[source]#
Wrapper around Petals Bloom models.
To use, you should have the
petals
python package installed, and the environment variableHUGGINGFACE_API_KEY
set with your API key.Any parameters that are valid to be passed to the call can be passed in, even if not explicitly saved on this class.
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field client: Any = None#
The client to use for the API calls.
- field do_sample: bool = True#
Whether or not to use sampling; use greedy decoding otherwise.
- field max_length: Optional[int] = None#
The maximum length of the sequence to be generated.
- field max_new_tokens: int = 256#
The maximum number of new tokens to generate in the completion.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- field model_name: str = 'bigscience/bloom-petals'#
The model to use.
- field temperature: float = 0.7#
What sampling temperature to use
- field tokenizer: Any = None#
The tokenizer to use for the API calls.
- field top_k: Optional[int] = None#
The number of highest probability vocabulary tokens to keep for top-k-filtering.
- field top_p: float = 0.9#
The cumulative probability for top-p sampling.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.PromptLayerOpenAI[source]#
Wrapper around OpenAI large language models.
To use, you should have the
openai
andpromptlayer
python package installed, and the environment variableOPENAI_API_KEY
andPROMPTLAYER_API_KEY
set with your openAI API key and promptlayer key respectively.All parameters that can be passed to the OpenAI LLM can also be passed here. The PromptLayerOpenAI LLM adds two optional :param
pl_tags
: List of strings to tag the request with. :paramreturn_pl_id
: If True, the PromptLayer request ID will bereturned in the
generation_info
field of theGeneration
object.Example
from langchain.llms import PromptLayerOpenAI openai = PromptLayerOpenAI(model_name="text-davinci-003")
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- create_llm_result(choices: Any, prompts: List[str], token_usage: Dict[str, int]) langchain.schema.LLMResult #
Create the LLMResult from the choices and prompts.
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Calculate num tokens with tiktoken package.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) List[List[str]] #
Get the sub prompts for llm call.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- max_tokens_for_prompt(prompt: str) int #
Calculate the maximum number of tokens possible to generate for a prompt.
- Parameters
prompt – The prompt to pass into the model.
- Returns
The maximum number of tokens to generate for a prompt.
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
- modelname_to_contextsize(modelname: str) int #
Calculate the maximum number of tokens possible to generate for a model.
text-davinci-003: 4,097 tokens text-curie-001: 2,048 tokens text-babbage-001: 2,048 tokens text-ada-001: 2,048 tokens code-davinci-002: 8,000 tokens code-cushman-001: 2,048 tokens
- Parameters
modelname – The modelname we want to know the context size for.
- Returns
The maximum context size
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
- prep_streaming_params(stop: Optional[List[str]] = None) Dict[str, Any] #
Prepare the params for streaming.
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- stream(prompt: str, stop: Optional[List[str]] = None) Generator #
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction. Once that happens, this interface could change.
- Parameters
prompt – The prompts to pass into the model.
stop – Optional list of stop words to use when generating.
- Returns
A generator representing the stream of tokens from OpenAI.
Example
generator = openai.stream("Tell me a joke.") for token in generator: yield token
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.PromptLayerOpenAIChat[source]#
Wrapper around OpenAI large language models.
To use, you should have the
openai
andpromptlayer
python package installed, and the environment variableOPENAI_API_KEY
andPROMPTLAYER_API_KEY
set with your openAI API key and promptlayer key respectively.All parameters that can be passed to the OpenAIChat LLM can also be passed here. The PromptLayerOpenAIChat adds two optional :param
pl_tags
: List of strings to tag the request with. :paramreturn_pl_id
: If True, the PromptLayer request ID will bereturned in the
generation_info
field of theGeneration
object.Example
from langchain.llms import PromptLayerOpenAIChat openaichat = PromptLayerOpenAIChat(model_name="gpt-3.5-turbo")
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field max_retries: int = 6#
Maximum number of retries to make when generating.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- field model_name: str = 'gpt-3.5-turbo'#
Model name to use.
- field prefix_messages: List [Optional]#
Series of messages for Chat input.
- field streaming: bool = False#
Whether to stream the results or not.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Calculate num tokens with tiktoken package.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Replicate[source]#
Wrapper around Replicate models.
To use, you should have the
replicate
python package installed, and the environment variableREPLICATE_API_TOKEN
set with your API token. You can find your token here: https://replicate.com/accountThe model param is required, but any other model parameters can also be passed in with the format input={model_param: value, …}
Example
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.SagemakerEndpoint[source]#
Wrapper around custom Sagemaker Inference Endpoints.
To use, you must supply the endpoint name from your deployed Sagemaker model & the region where it is deployed.
To authenticate, the AWS client uses the following methods to automatically load credentials: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
If a specific credential profile should be used, you must pass the name of the profile from the ~/.aws/credentials file that is to be used.
Make sure the credentials / roles used have the required policies to access the Sagemaker endpoint. See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field content_handler: langchain.llms.sagemaker_endpoint.ContentHandlerBase [Required]#
The content handler class that provides an input and output transform functions to handle formats between LLM and the endpoint.
- field credentials_profile_name: Optional[str] = None#
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which has either access keys or role information specified. If not specified, the default credential profile or, if on an EC2 instance, credentials from IMDS will be used. See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
- field endpoint_kwargs: Optional[Dict] = None#
Optional attributes passed to the invoke_endpoint function. See `boto3`_. docs for more info. .. _boto3: <https://boto3.amazonaws.com/v1/documentation/api/latest/index.html>
- field endpoint_name: str = ''#
The name of the endpoint from the deployed Sagemaker model. Must be unique within an AWS Region.
- field model_kwargs: Optional[Dict] = None#
Key word arguments to pass to the model.
- field region_name: str = ''#
The aws region where the Sagemaker model is deployed, eg. us-west-2.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.SelfHostedHuggingFaceLLM[source]#
Wrapper around HuggingFace Pipeline API to run on self-hosted remote hardware.
Supported hardware includes auto-launched instances on AWS, GCP, Azure, and Lambda, as well as servers specified by IP address and SSH credentials (such as on-prem, or another cloud like Paperspace, Coreweave, etc.).
To use, you should have the
runhouse
python package installed.Only supports text-generation and text2text-generation for now.
- Example using from_model_id:
from langchain.llms import SelfHostedHuggingFaceLLM import runhouse as rh gpu = rh.cluster(name="rh-a10x", instance_type="A100:1") hf = SelfHostedHuggingFaceLLM( model_id="google/flan-t5-large", task="text2text-generation", hardware=gpu )
- Example passing fn that generates a pipeline (bc the pipeline is not serializable):
from langchain.llms import SelfHostedHuggingFaceLLM from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline import runhouse as rh def get_pipeline(): model_id = "gpt2" tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained(model_id) pipe = pipeline( "text-generation", model=model, tokenizer=tokenizer ) return pipe hf = SelfHostedHuggingFaceLLM( model_load_fn=get_pipeline, model_id="gpt2", hardware=gpu)
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
- field device: int = 0#
Device to use for inference. -1 for CPU, 0 for GPU, 1 for second GPU, etc.
- field hardware: Any = None#
Remote hardware to send the inference function to.
- field inference_fn: Callable = <function _generate_text>#
Inference function to send to the remote hardware.
- field load_fn_kwargs: Optional[dict] = None#
Key word arguments to pass to the model load function.
- field model_id: str = 'gpt2'#
Hugging Face model_id to load the model.
- field model_kwargs: Optional[dict] = None#
Key word arguments to pass to the model.
- field model_load_fn: Callable = <function _load_transformer>#
Function to load the model remotely on the server.
- field model_reqs: List[str] = ['./', 'transformers', 'torch']#
Requirements to install on hardware to inference the model.
- field task: str = 'text-generation'#
Hugging Face task (either “text-generation” or “text2text-generation”).
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) langchain.llms.base.LLM #
Init the SelfHostedPipeline from a pipeline object or string.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.SelfHostedPipeline[source]#
Run model inference on self-hosted remote hardware.
Supported hardware includes auto-launched instances on AWS, GCP, Azure, and Lambda, as well as servers specified by IP address and SSH credentials (such as on-prem, or another cloud like Paperspace, Coreweave, etc.).
To use, you should have the
runhouse
python package installed.- Example for custom pipeline and inference functions:
from langchain.llms import SelfHostedPipeline from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline import runhouse as rh def load_pipeline(): tokenizer = AutoTokenizer.from_pretrained("gpt2") model = AutoModelForCausalLM.from_pretrained("gpt2") return pipeline( "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=10 ) def inference_fn(pipeline, prompt, stop = None): return pipeline(prompt)[0]["generated_text"] gpu = rh.cluster(name="rh-a10x", instance_type="A100:1") llm = SelfHostedPipeline( model_load_fn=load_pipeline, hardware=gpu, model_reqs=model_reqs, inference_fn=inference_fn )
- Example for <2GB model (can be serialized and sent directly to the server):
from langchain.llms import SelfHostedPipeline import runhouse as rh gpu = rh.cluster(name="rh-a10x", instance_type="A100:1") my_model = ... llm = SelfHostedPipeline.from_pipeline( pipeline=my_model, hardware=gpu, model_reqs=["./", "torch", "transformers"], )
- Example passing model path for larger models:
from langchain.llms import SelfHostedPipeline import runhouse as rh import pickle from transformers import pipeline generator = pipeline(model="gpt2") rh.blob(pickle.dumps(generator), path="models/pipeline.pkl" ).save().to(gpu, path="models") llm = SelfHostedPipeline.from_pipeline( pipeline="models/pipeline.pkl", hardware=gpu, model_reqs=["./", "torch", "transformers"], )
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
- field hardware: Any = None#
Remote hardware to send the inference function to.
- field inference_fn: Callable = <function _generate_text>#
Inference function to send to the remote hardware.
- field load_fn_kwargs: Optional[dict] = None#
Key word arguments to pass to the model load function.
- field model_load_fn: Callable [Required]#
Function to load the model remotely on the server.
- field model_reqs: List[str] = ['./', 'torch']#
Requirements to install on hardware to inference the model.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) langchain.llms.base.LLM [source]#
Init the SelfHostedPipeline from a pipeline object or string.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.StochasticAI[source]#
Wrapper around StochasticAI large language models.
To use, you should have the environment variable
STOCHASTICAI_API_KEY
set with your API key.Example
from langchain.llms import StochasticAI stochasticai = StochasticAI(api_url="")
- Validators
build_extra
»all fields
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field api_url: str = ''#
Model name to use.
- field model_kwargs: Dict[str, Any] [Optional]#
Holds any model parameters valid for create call not explicitly specified.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.
- pydantic model langchain.llms.Writer[source]#
Wrapper around Writer large language models.
To use, you should have the environment variable
WRITER_API_KEY
set with your API key.Example
from langchain import Writer writer = Writer(model_id="palmyra-base")
- Validators
set_callback_manager
»callback_manager
set_verbose
»verbose
validate_environment
»all fields
- field base_url: Optional[str] = None#
Base url to use, if None decides based on model name.
- field beam_search_diversity_rate: float = 1.0#
Only applies to beam search, i.e. when the beam width is >1. A higher value encourages beam search to return a more diverse set of candidates
- field beam_width: Optional[int] = None#
The number of concurrent candidates to keep track of during beam search
- field length: int = 256#
The maximum number of tokens to generate in the completion.
- field length_pentaly: float = 1.0#
Only applies to beam search, i.e. when the beam width is >1. Larger values penalize long candidates more heavily, thus preferring shorter candidates
- field logprobs: bool = False#
Whether to return log probabilities.
- field model_id: str = 'palmyra-base'#
Model name to use.
- field random_seed: int = 0#
The model generates random results. Changing the random seed alone will produce a different response with similar characteristics. It is possible to reproduce results by fixing the random seed (assuming all other hyperparameters are also fixed)
- field repetition_penalty: float = 1.0#
Penalizes repeated tokens according to frequency.
- field stop: Optional[List[str]] = None#
Sequences when completion generation will stop
- field temperature: float = 1.0#
What sampling temperature to use.
- field tokens_to_generate: int = 24#
Max number of tokens to generate.
- field top_k: int = 1#
The number of highest probability vocabulary tokens to keep for top-k-filtering.
- field top_p: float = 1.0#
Total probability mass of tokens to consider at each step.
- __call__(prompt: str, stop: Optional[List[str]] = None) str #
Check Cache and run the LLM on the given prompt and input.
- async agenerate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- async agenerate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) Model #
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data. Default values are respected, but no other validation is performed. Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
- copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) Model #
Duplicate a model, optionally choose which fields to include, exclude and change.
- Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating the new model: you should trust this data
deep – set to True to make a deep copy of the model
- Returns
new model instance
- dict(**kwargs: Any) Dict #
Return a dictionary of the LLM.
- generate(prompts: List[str], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Run the LLM on the given prompt and input.
- generate_prompt(prompts: List[langchain.schema.PromptValue], stop: Optional[List[str]] = None) langchain.schema.LLMResult #
Take in a list of prompt values and return an LLMResult.
- get_num_tokens(text: str) int #
Get the number of tokens present in the text.
- get_num_tokens_from_messages(messages: List[langchain.schema.BaseMessage]) int #
Get the number of tokens in the message.
- json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) unicode #
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
- save(file_path: Union[pathlib.Path, str]) None #
Save the LLM.
- Parameters
file_path – Path to file to save the LLM to.
Example: .. code-block:: python
llm.save(file_path=”path/llm.yaml”)
- classmethod update_forward_refs(**localns: Any) None #
Try to update ForwardRefs on fields based on this Model, globalns and localns.