Document compressor that uses a pipeline of Transformers.

Example

const compressorPipeline = new DocumentCompressorPipeline({
transformers: [
new RecursiveCharacterTextSplitter({
chunkSize: 200,
chunkOverlap: 0,
}),
new EmbeddingsFilter({
embeddings: new OpenAIEmbeddings(),
similarityThreshold: 0.8,
k: 5,
}),
],
});
const retriever = new ContextualCompressionRetriever({
baseCompressor: compressorPipeline,
baseRetriever: new TavilySearchAPIRetriever({
includeRawContent: true,
}),
});
const retrievedDocs = await retriever.getRelevantDocuments(
"What did the speaker say about Justice Breyer in the 2022 State of the Union?",
);
console.log({ retrievedDocs });

Hierarchy

Constructors

Properties

transformers: (BaseDocumentTransformer<DocumentInterface<Record<string, any>>[], DocumentInterface<Record<string, any>>[]> | BaseDocumentCompressor)[]

Methods

  • Abstract method that must be implemented by any class that extends BaseDocumentCompressor. This method takes an array of Document objects and a query string as parameters and returns a Promise that resolves with an array of compressed Document objects.

    Parameters

    • documents: DocumentInterface<Record<string, any>>[]

      An array of Document objects to be compressed.

    • query: string

      A query string.

    • Optional callbacks: Callbacks

    Returns Promise<DocumentInterface<Record<string, any>>[]>

    A Promise that resolves with an array of compressed Document objects.

Generated using TypeDoc