Get Started
Introduction
Core Concepts
Advanced Usage
Deploy with Python

Core Concepts

AI Functions Explained

Stateless Transformation

An AI Function performs a stateless transformation from input data to output data. It processes inputs—such as text or images—and produces structured, strongly typed outputs without maintaining any internal state between calls.

Query of an AI Function

Instructions

The instructions, the static part of the prompt, are feed to the system message to an OpenAI style LLM API.

Input

Inputs are provided as part of the user message to the LLM APIs. These inputs define the data the function will process.

Output

  • Assistant's Response: The LLM generates outputs based on the combined instructions and function inputs. Depending on the provider's API design, outputs are returned as the assistant's message or as part of function call arguments.
  • Structured Outputs: Outputs conform to a predefined schema, ensuring consistency and reliability. This structure facilitates easy parsing and integration into other systems.

Components of an AI Function

1. Prompt Engineering

AI Agent Assistance

The AI Function Builder includes an AI agent that generates an initial prototype prompt based on your task description. This prototype incorporates best practices and relevant techniques to serve as a solid starting point.

Advanced Techniques Included

  • Chain-of-Thought Prompting: Enables the model to perform step-by-step reasoning, which is useful for complex tasks requiring logical deductions.
  • Structured Prompts: Utilize formats like XML or Markdown to make prompts more machine-readable, aiding the model in parsing and understanding instructions.

The AI Function Builder analyzes your natural language description and automatically suggests appropriate prompting add-ons, simplifying the prompt engineering process.

User Refinement and A/B Testing

  • Editing: Users can edit and refine the generated prompt or add on to better suit specific needs.
  • A/B Testing: The platform supports A/B testing, allowing users to compare different prompts on multiple example inputs, when iterating on prompts.

2. LLM Selection

Understanding Model Differences

No single model excels at all tasks. Models vary in:

  • Inference Speed: Faster models are preferable for real-time applications.
  • Intelligence: Some models offer more advanced reasoning capabilities.
  • Cost: Important for balancing performance with budget constraints.
  • Features: Access to the internet, support for multimodal inputs, etc.

AI Agent Recommendations

The AI agent selects a suitable model based on your task description, usually opting for the most cost-effective and low latency model that can solve the task.

Flexibility and Optimization

  • Model Switching: You can switch between models across different providers.
  • Pareto Efficiency Curve: Configuration editors recommend models that illustrate trade-offs between factors like speed and intelligence.
  • A/B Testing: Allows you to test different models and compare performance directly, facilitating optimization for cost, speed, and output quality.

3. Output Schema

Automated Schema Generation

The AI agent automatically generates an output schema based on your task description. This schema defines:

  • Data Types: Specifies the type of data each field should contain.
  • Nested Structure: Allow hierarchical organization of the output data.
  • Required Fields: Identifies which fields are mandatory.
  • Field Descriptions and Examples: Provides additional context to inform the LLM during generation.

Ordering matters

Unlike standard JSON Schema definitions, the order of keys in the output schema affects the sequence in which the LLM generates the output fields. Since LLMs generate text autoregressively—producing one token at a time based on previously generated tokens—the model generates each property in the output sequentially. This means that when the LLM is generating later properties, it has already generated the earlier ones and can use them as context.

Implications:

  • Intermediate Steps First: Place intermediate steps or foundational data that can assist in generating later fields at the beginning of the schema. This helps the LLM use earlier outputs to inform subsequent ones.
  • Logical Flow: Arrange the fields in a logical order that reflects the natural progression of the task or reasoning process.