ReaderConfig

class langchain_community.llms.titan_takeoff.ReaderConfig

Bases: BaseModel

Configuration for the reader to be deployed in Titan Takeoff API.

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param consumer_group: str = 'primary'

The consumer group to place the reader into

param device: Device = Device.cuda

The device to use for inference, cuda or cpu

param max_batch_size: int = 4

The max batch size for continuous batching of requests

param max_seq_length: int = 512

The maximum sequence length to use for inference, defaults to 512

param model_name: str [Required]

The name of the model to use

param tensor_parallel: int | None = None

The number of gpus you would like your model to be split across