Summarize Text
Synopsis
This operator can be used to summarize a text via an LLM.
Description
The operator is used for submitting the contents of the selected nominal column to an LLM to create a short summary. It can optionally be enriched with rag input data for grounding the model and providing additional context.
Input
Connection
The LLM connection which contains the LLM provider to use, together with the relevant credentials. This port is mandatory, and a valid LLM connection must be provided.
Data
This table input port expects the data which contains the column to submit to the LLM for classification.
Prompt
OPTIONAL. A preconfigured prompt (created by the Create Prompt operator) that will be used to submit the task to the LLM, together with its config. In other words, no prompt and no settings need to be configured in this operator if a prompt object is passed here. If this port is connected, the respective settings of the operator will vanish and the prompt and settings contained within the prompt object will be used instead. Advanced: It is also possible to use the output of a record output port of any of the Prompt Engineering operators, which will re-use the prompt that was used to generate the record.
RAG
OPTIONAL. These optional input ports are used to provide the LLM with reference data (the R in RAG). To use the RAG data at specific places within the prompt/system prompt, you can use special placeholders: a set of 2 curly brackets around the ref name together with the RAG port number the data is supplied at, e.g. {{RAG 1}}. Reference numbering starts at 1, referencing the first RAG port. If no placeholders are used, each RAG data is appended as a simple additional message to the LLM as-is. Note: For now, the implementation is very simplistic: The entire objects at these ports will be converted to a textual (e.g. JSON-based) representation and submitted as-is as part of the LLM query. Files are read and submitted in the same way as if you were to open a file on your computer with a text editor and using whatever is shown as part of the LLM query.
Output
Result
The data table containing the new target column with the summary.
Parameters
Input Column
The column which to summarize.
Summary Length
The number of sentences the summary should have.
Target Column
The name of the new column which should contain the summary.
System Prompt
An optional system prompt, can be left empty. The system prompt is an initialization prompt which can be sent to the LLM. It is often used to let the LLM use a certain persona or style in answering such as "You are a factual chatbot which prefers short answers."
Provider
The provider of the model. Availability depends on the provided LLM connection.
Model
The model to submit the prompt to. Availability depends on the LLM provider specified in the provided connection.
Override Prompt Config
This parameter becomes available in case a Prompt object is provided at the prompt input port. By default, and with this override not being selected, the configuration is taken from the Prompt object. If this override is active, the configuration is taken from the parameters of this operator instead. This allows to create a prompt with one LLM, but use it with a different LLM.
Max Output Tokens
The maximum amount of output tokens that should be returned by the LLM. Note that different LLMs may have different maximum output token capabilities. A token is usually between 2-4 characters, but the exact amount depends on the model, the language, and the content.
Response Format
The response format of the LLM. Some models may not be able to deliver all available options.
Temperature
Controls the randomness used in the answers. Lower values will lead to less random answers. A temperature of 0 normally represents a fully deterministic model behavior.
Top P
Controls diversity via nucleus sampling. A value of 0.5 means that half of all likelihood-weighted options would be considered. A value of 1.0 means all options are considered.
Frequency Penalty
How much to penalize new tokens based on their frequency in the answer so far. Can be used to reduce repetitiveness.
Presence Penalty
How much to penalize new tokens based on their presence in the answer so far. Increases the model's likeliness to talk about new topics.
Use Seed
If selected, the seed can be specified to have more deterministic results.
Seed
Some LLMs (not all support this functionality!) will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed. Behavior depends on the selected model.
Enable Parallel Execution
This parameter enables the parallel execution of this operator and will cause multiple concurrent queries to the LLM. Please disable the parallel execution if this is not desirable.