AI agent & RAG frameworks

The @tool decorator wraps a Python function as a Tool: the function name becomes the tool name, the docstring becomes the description, and the type hints become a JSON argument schema. The model sees only those three things.

invoke runs the runnable once on one input and returns the final output. batch runs many inputs, concurrently by default. stream yields output in chunks as it is produced. Each has an async twin: ainvoke, abatch, astream.

A chat message history holds one conversation's ordered messages, keyed by a session id, through the messages property plus add_messages and clear. InMemoryChatMessageHistory keeps that list in process RAM, so a restart or a second replica loses the conversation.

A ChatModel takes a list of role-tagged messages and returns an AIMessage object; the LLM interface takes a plain string and returns a plain string. Tool calling, structured output and multimodal content exist only on the ChatModel path.

PromptTemplate formats one string with declared variables and suits plain text-completion models. ChatPromptTemplate formats an ordered list of role-tagged messages (system, human, ai) and is what you use with chat models, which is nearly always today.

Define a state schema, usually a TypedDict; create StateGraph(State); register functions with add_node; wire them with add_edge from START to END; then call compile() to get a runnable graph you invoke with an initial state dict.

graph.invoke() blocks and returns only the final state. graph.stream() yields a chunk as the graph advances, one per superstep, so a UI can show progress long before the run finishes.

A checkpointer makes LangGraph save a snapshot of the graph's state after every super-step. The thread_id you pass in config names the conversation those snapshots belong to, so the next invoke resumes that thread's accumulated state instead of starting empty.

interrupt() pauses the graph in the middle of a node, persists a checkpoint, and surfaces a payload to the caller under the interrupt key. You resume by invoking the same thread_id with Command(resume=value); the interrupt() call then returns that value.

A Command return carries both a state update and a goto, so one node writes state and names its successor in a single value. That is the handoff idiom in multi-agent graphs; a plain dict return leaves routing entirely to the declared edges.

Instantiate the Flow subclass and call kickoff(), or kickoff_async() to await it. Pass a dict as the inputs argument and those values are applied to the flow state before the start methods run. kickoff returns the last executed step's output.

It appends coworker tools — delegate work and ask a question — to that agent's toolset, built from the crew's other agents and addressed by role string. The cost is extra LLM turns, longer prompts, and lossy handoffs, since only text crosses between agents.

Role, goal and backstory are prompt text: CrewAI interpolates them into the system prompt sent to the model for that agent. Role additionally names the agent for delegation and logs. None of the three enforces anything — tools and task assignment do that.

A hierarchical CrewAI crew needs a manager: either manager_llm, from which CrewAI builds a default manager agent, or manager_agent, your own Agent. Supplying neither fails validation, and the manager agent must not also appear in the crew's agents list.

Process.sequential executes the Crew's tasks in the order you listed them, each task seeing earlier output. Process.hierarchical inserts a manager agent that decides which worker gets which task and reviews the result, adding LLM calls, latency and nondeterminism.

LocalCommandLineCodeExecutor runs model-generated code as a subprocess on the host machine, with your files, environment variables and network. DockerCommandLineCodeExecutor runs the same code inside a container instead, isolating it from the host. Local is for trusted prototyping only.

RoundRobinGroupChat cycles participants in the exact order passed to its constructor, one response per turn, with no model deciding anything. Every response is broadcast to all participants, so each agent sees the whole shared conversation.

Pass an explicit model_info dictionary alongside model and base_url. The client keeps a capability table for models it recognizes; for anything outside it — a self-hosted or newly released model behind an OpenAI-compatible endpoint — it cannot infer capabilities and raises unless you declare them.

AssistantAgent makes one model call with its system message plus its context, executes every requested tool, then by default returns a ToolCallSummaryMessage holding the raw tool results. It only sends those results back to the model when reflect_on_tool_use is True.

autogen-core is the event-driven actor runtime plus base abstractions. autogen-agentchat is the opinionated agent-and-team API built on top of it. autogen-ext holds the concrete integrations: model clients, tool adapters, code executors and the gRPC runtime.

instruction is the agent's own prompt — it tells that agent how to behave. description is a one-line summary other agents read when deciding whether to hand work over, so it drives delegation rather than behaviour.

adk web starts a local dev UI over your agent folder, with chat plus Events, Trace and Eval tabs. adk run drives the same agent from the terminal. adk api_server serves it as a local HTTP API.

Nothing survives with InMemorySessionService — sessions, event history and state live in the process and vanish on restart. DatabaseSessionService writes them to a SQL database given by db_url, so the same session_id resumes afterwards.

Google ADK wraps any callable placed in an agent's tools list into a FunctionTool. The function name, its type-hinted parameters and its docstring become the declaration the model sees, so the docstring is functional code rather than a comment.

Use LlmAgent when the model must decide what happens next. Use ADK's SequentialAgent, ParallelAgent or LoopAgent when the order is already known: they run their sub_agents on fixed control flow and call no LLM of their own.

A plain Koog AIAgent already runs a built-in loop: send the prompt, execute any tool calls, return the final message. You pass a strategy when you need explicit control flow — branching, custom Kotlin steps, history compression, or subgraphs.

Koog's prompt("id") { } builder returns an immutable Prompt made of typed messages added by system(), user() and assistant() calls. The id is a stable label for that prompt, used to identify it in logs, traces and tooling — not sent as content.

ToolRegistry is the single list of tools a Koog agent may call. Koog sends each registered tool's name, description and parameter schema with the prompt, then resolves incoming calls back to Kotlin. An unregistered tool is invisible and unresolvable.

Send nodeLLMRequest to nodeExecuteTool on onToolCall, nodeExecuteTool straight to nodeLLMSendToolResult, and nodeLLMSendToolResult back to nodeExecuteTool on onToolCall. Both LLM nodes also need an onAssistantMessage edge to nodeFinish, or the loop never exits.

A PromptExecutor is Koog's single suspending entry point for running a Prompt against an LLModel, hiding provider HTTP details behind an LLMClient. SingleLLMPromptExecutor wraps one client; MultiLLMPromptExecutor holds several and routes each call by the model's provider.

QueryEngineTool.from_defaults wraps an existing query engine in LlamaIndex's tool interface with a name and description. That description is the only thing the model reads when deciding whether to route a question to that corpus.

Both are token counts, not characters. chunk_size (default 1024) caps how many tokens a node may hold, measured with the tokenizer LlamaIndex is configured with; chunk_overlap (default 200) repeats that many trailing tokens at the start of the next node.

VectorStoreIndex.from_documents() splits each document into nodes, calls the embedding model once per node, and writes those vectors plus the node text into a vector store — by default an in-memory SimpleVectorStore that vanishes when the process exits.

SimpleDirectoryReader walks a folder, picks a reader based on each file's extension, and returns Document objects with file metadata attached. You narrow what it touches with input_files, required_exts, exclude, exclude_hidden, recursive and num_files_limit.

In LlamaIndex, as_retriever() returns an object whose retrieve() call gives back scored nodes and makes no LLM call. as_query_engine() wraps a retriever with a response synthesizer, so query() also calls the LLM and returns an answer plus its source nodes.

Haystack's Agent packages the whole tool-calling loop into one component: it sends messages to the chat generator, executes requested tools through an internal ToolInvoker, feeds the results back, and stops on its exit conditions or step limit.

ChatPromptBuilder renders a Jinja2 template into a list of ChatMessage objects. Every placeholder in the template becomes an input it expects at run time, and the rendered messages leave on its prompt output, ready for a chat generator.

A Haystack indexing pipeline runs a converter (file bytes to Document), then DocumentCleaner (strip whitespace and boilerplate), DocumentSplitter (cut into smaller Documents), a document embedder (attach vectors), and DocumentWriter (persist to a DocumentStore).

Pipeline.run() takes a dict keyed by component name, whose value is that component's run() keyword arguments. It returns a dict, also keyed by component name, holding the outputs no other component consumed. Use include_outputs_from to also get intermediate outputs.

InMemoryBM25Retriever takes the raw query string on its query input and scores lexically over document text. InMemoryEmbeddingRetriever takes a query_embedding, so a text embedder must run first in the pipeline and feed its vector into that socket.

RAGFlow ships as a Compose stack: the ragflow server container (web UI plus API, with a task executor beside it), a document engine (Elasticsearch by default, or Infinity) for chunks and vectors, MySQL for metadata, MinIO for the original files, and Redis for the task queue.

The chunk method is a per-document-type parsing template — General, Q&A, Manual, Table, Paper, Book, Laws, Presentation or One — that decides where the document is cut and what a chunk represents. It is a knowledge-base default you can override per file, and changing it requires re-parsing.

{knowledge} is the placeholder RAGFlow fills with the retrieved chunks before calling the model. Remove it from the assistant's system prompt and retrieval still runs, but no document text ever reaches the LLM, so answers stop being grounded.

Generate an API key in the RAGFlow UI and send it as Authorization: Bearer <key> to /api/v1. Create a dataset, upload documents into it, then trigger parsing explicitly — parsing is asynchronous, and a document is only searchable once its run state reaches completion.

DeepDoc runs vision models over each page: OCR for scanned text, layout analysis that labels titles, paragraphs, figures, captions and headers, and table-structure recognition that rebuilds tables as HTML. Plain extraction returns only a flat text stream.