JSONLoader#

class langchain_community.document_loaders.json_loader.JSONLoader(file_path: str | Path, jq_schema: str, content_key: str | None = None, is_content_key_jq_parsable: bool | None = False, metadata_func: Callable[[Dict, Dict], Dict] | None = None, text_content: bool = True, json_lines: bool = False)[source]#

Load a JSON file using a jq schema.

Setup:
pip install -U jq
Instantiate:
from langchain_community.document_loaders import JSONLoader
import json
from pathlib import Path

file_path='./sample_quiz.json'
data = json.loads(Path(file_path).read_text())
loader = JSONLoader(
         file_path=file_path,
         jq_schema='.quiz',
         text_content=False)
Load:
docs = loader.load()
print(docs[0].page_content[:100])
print(docs[0].metadata)

{"sport": {"q1": {"question": "Which one is correct team name in
NBA?", "options": ["New York Bulls"
{'source': '/sample_quiz
.json', 'seq_num': 1}
Async load:
docs = await loader.aload()
print(docs[0].page_content[:100])
print(docs[0].metadata)

{"sport": {"q1": {"question": "Which one is correct team name in
NBA?", "options": ["New York Bulls"
{'source': '/sample_quizg
.json', 'seq_num': 1}
Lazy load:
docs = []
docs_lazy = loader.lazy_load()

# async variant:
# docs_lazy = await loader.alazy_load()

for doc in docs_lazy:
    docs.append(doc)
print(docs[0].page_content[:100])
print(docs[0].metadata)

{"sport": {"q1": {"question": "Which one is correct team name in
NBA?", "options": ["New York Bulls"
{'source': '/sample_quiz
.json', 'seq_num': 1}

Initialize the JSONLoader.

Parameters:

  • file_path (Union[str, Path]) – The path to the JSON or JSON Lines file.

  • jq_schema (str) – The jq schema to use to extract the data or text from the JSON.

  • content_key (str) – The key to use to extract the content from the JSON if the jq_schema results to a list of objects (dict). If is_content_key_jq_parsable is True, this has to be a jq compatible schema. If is_content_key_jq_parsable is False, this should be a simple string key.

  • is_content_key_jq_parsable (bool) – A flag to determine if content_key is parsable by jq or not. If True, content_key is treated as a jq schema and compiled accordingly. If False or if content_key is None, content_key is used as a simple string. Default is False.

  • metadata_func (Callable[Dict, Dict]) – A function that takes in the JSON object extracted by the jq_schema and the default metadata and returns a dict of the updated metadata.

  • text_content (bool) – Boolean flag to indicate whether the content is in string format, default to True.

  • json_lines (bool) – Boolean flag to indicate whether the input is in JSON Lines format.

Methods

__init__(file_path, jq_schema[, ...])

Initialize the JSONLoader.

alazy_load()

A lazy loader for Documents.

aload()

Load data into Document objects.

lazy_load()

Load and return documents from the JSON file.

load()

Load data into Document objects.

load_and_split([text_splitter])

Load Documents and split into chunks.
__init__(file_path: str | Path, jq_schema: str, content_key: str | None = None, is_content_key_jq_parsable: bool | None = False, metadata_func: Callable[[Dict, Dict], Dict] | None = None, text_content: bool = True, json_lines: bool = False)[source]#

Initialize the JSONLoader.

Parameters:

  • file_path (Union[str, Path]) – The path to the JSON or JSON Lines file.

  • jq_schema (str) – The jq schema to use to extract the data or text from the JSON.

  • content_key (str) – The key to use to extract the content from the JSON if the jq_schema results to a list of objects (dict). If is_content_key_jq_parsable is True, this has to be a jq compatible schema. If is_content_key_jq_parsable is False, this should be a simple string key.

  • is_content_key_jq_parsable (bool) – A flag to determine if content_key is parsable by jq or not. If True, content_key is treated as a jq schema and compiled accordingly. If False or if content_key is None, content_key is used as a simple string. Default is False.

  • metadata_func (Callable[Dict, Dict]) – A function that takes in the JSON object extracted by the jq_schema and the default metadata and returns a dict of the updated metadata.

  • text_content (bool) – Boolean flag to indicate whether the content is in string format, default to True.

  • json_lines (bool) – Boolean flag to indicate whether the input is in JSON Lines format.

async alazy_load() → AsyncIterator[Document]#

A lazy loader for Documents.

Return type:

AsyncIterator[Document]

async aload() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

lazy_load() → Iterator[Document][source]#

Load and return documents from the JSON file.

Return type:

Iterator[Document]

load() → List[Document]#

Load data into Document objects.

Return type:

List[Document]

load_and_split(text_splitter: TextSplitter | None = None) → List[Document]#

Load Documents and split into chunks. Chunks are returned as Documents.

Do not override this method. It should be considered to be deprecated!

Parameters:

text_splitter (Optional[TextSplitter]) – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns:

List of Documents.

Return type:

List[Document]

Examples using JSONLoader

On this page