AI Agent
Synopsis
This operator behaves as an AI Agent within the Altair AI Cloud.
Description
The operator behaves as an AI Agent within the Altair AI Cloud and can interact with the platform via human language. Since it is backed by an LLM, the exact interactions are determined via reasoning of the LLM. It can access APIs of the platform that would normally only be available via the UI, as well as any deployments available to the current project - this allows the user the capability to define agentic functions via workflows.
Input
Connection
The LLM connection which contains the LLM provider to use, together with the relevant credentials. This port is mandatory, and a valid LLM connection must be provided.
Chat
OPTIONAL. A previous conversation history with an LLM (created by the Create Chat operator) can be added here, which will be submitted to the LLM before the new prompt to pass a conversation history along. If this optional port is connected, the LLM will be given the historical context of the conversation. Note: The history is submitted as-is, no RAG or any other replacements will take place before submission.
Prompt
OPTIONAL. A preconfigured prompt (created by the Create Prompt operator) that will be used to submit the task to the LLM, together with its config. In other words, no prompt and no settings need to be configured in this operator if a prompt object is passed here. If this port is connected, the respective settings of the operator will vanish and the prompt and settings contained within the prompt object will be used instead. Advanced: It is also possible to use the output of a record output port of any of the Prompt Engineering operators, which will re-use the prompt that was used to generate the record.
RAG
OPTIONAL. These optional input ports are used to provide the LLM with reference data (the R in RAG). To use the RAG data at specific places within the prompt/system prompt, you can use special placeholders: a set of 2 curly brackets around the ref name together with the RAG port number the data is supplied at, e.g. {{RAG 1}}. Reference numbering starts at 1, referencing the first RAG port. If no placeholders are used, each RAG data is appended as a simple additional message to the LLM as-is. Note: For now, the implementation is very simplistic: The entire objects at these ports will be converted to a textual (e.g. JSON-based) representation and submitted as-is as part of the LLM query. Files are read and submitted in the same way as if you were to open a file on your computer with a text editor and using whatever is shown as part of the LLM query.
Output
Response
A data table with a single cell, containing the LLM response to the AI agent request. The response can vary drastically based on the prompt and system prompt provided, and can range from e.g. a simple URL pointing to a result to a long answer detailing every step the agent took and their outcomes.
Data Output
An optionally returned data table that the agent either created based on calculations, or directly returned from the Data Catalog. Can be used to e.g. do things like "give me the latest data added to the project". Note: If the agent was not instructed to output data, this port will be empty and not return any data object!
Record
A complete representation of the prompt that was used to query the LLM, including all input prompt messages, the response, utilized function calls including their arguments, as well as config and meta information. This can be used for auditing the LLM interaction and result.
Parameters
Provider
The provider of the model. Availability depends on the provided LLM connection.
Model
The model to submit the prompt to. Availability depends on the LLM provider specified in the provided connection.
Prompt
The prompt for the AI agent with optional rag references (via {{RAG x}} placeholders) that will be filled in before being sent to the LLM. Rag references are referencing the rag input data at the specified port (starting at 1), and they consist of a set of 2 curly brackets around the ref name, e.g. {{RAG 1}}.
System Prompt
An optional system prompt, can be left empty. The system prompt is an initialization prompt which can be used to define the behavior of the agent. It can be used to make the agent more or less verbose, e.g. "Comment on each step you did to explain it to the user." or "Provide only the actual result of the relevant function call, no prose."
Override Prompt Config
This parameter becomes available in case a Prompt object is provided at the prompt input port. By default, and with this override not being selected, the configuration is taken from the Prompt object. If this override is active, the configuration is taken from the parameters of this operator instead. This allows to create a prompt with one LLM, but use it with a different LLM.
Max Output Tokens
The maximum amount of output tokens that should be returned by the LLM. For the agent, this includes tool invocations, so it needs to be significantly higher than for prompts normally. Note that different LLMs may have different maximum output token capabilities. A token is usually between 2-4 characters, but the exact amount depends on the model, the language, and the content.
Response Format
The response format of the LLM. Some models may not be able to deliver all available options.
Temperature
Controls the randomness used in the answers. For the agent, you'd normally want a temperature of 0 to as much as possible reduce randomness of the executed functions. Note that even with a temperature of 0, the functions executed can vary between runs, this depends on the used LLM and how well it is constrained via prompts. Lower values will lead to less random answers. A temperature of 0 normally represents a fully deterministic model behavior.
Top P
Controls diversity via nucleus sampling. A value of 0.5 means that half of all likelihood-weighted options would be considered. A value of 1.0 means all options are considered.
Frequency Penalty
How much to penalize new tokens based on their frequency in the answer so far. Can be used to reduce repetitiveness.
Presence Penalty
How much to penalize new tokens based on their presence in the answer so far. Increases the model's likeliness to talk about new topics.
Use Seed
If selected, the seed can be specified to have more deterministic results.
Seed
Some LLMs (not all support this functionality!) will make a best effort to sample deterministically, such that repeated requests with the same seed and parameters should return the same result. Determinism is not guaranteed. Behavior depends on the selected model.