ReaderConfig

class langchain_community.llms.titan_takeoff.ReaderConfig

Bases: BaseModel

Configuration for the reader to be deployed in Titan Takeoff API.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

param consumer_group: str = 'primary'

The consumer group to place the reader into

param device: Device = Device.cuda

The device to use for inference, cuda or cpu

param max_batch_size: int = 4

The max batch size for continuous batching of requests

param max_seq_length: int = 512

The maximum sequence length to use for inference, defaults to 512

param model_name: str [Required]

The name of the model to use

param tensor_parallel: int | None = None

The number of gpus you would like your model to be split across