Skip to content

Understanding Agentic AI and Tool Calling: A Simple Chatbot Demo

An AI agent combines a language model with software that can choose tools, act on their results, and continue through several steps. Products such as OpenAI's Operator and Claude's Computer Use illustrate parts of this pattern.

I built an interactive chatbot to make the control loop visible. It can retrieve document context, call web APIs, and create GitHub issues. This post explains tool calling, how the loop decides what to do next, and why permissions and stopping conditions matter.

Navigate this post

From Passive Answers to Autonomous Tool Use

Standard Large Language Models (LLMs) operate as text-in, text-out generation engines. When given a prompt, an LLM predicts the most likely sequence of tokens based entirely on patterns learned during pre-training. While effective for drafting prose or answering general queries, static language models cannot access real-time information or interact with external software systems.

Agentic AI extends language models by using the model as a reasoning engine inside a broader application loop. Instead of generating a direct textual answer immediately, the agent evaluates the user prompt, determines what external information or actions are required, and issues structured commands called tool calls.

Large Language Model (LLM)

A text generation engine trained on massive language datasets that predicts the next likely words in a response, but cannot natively perform real-world actions or access live external data.

Tool Calling

A mechanism where an AI model outputs formatted commands (JSON requests) instructing host software to run specific functions on its behalf - like an assistant asking a calculator to perform exact math.

Agentic AI

An application design where an AI model acts autonomously - setting step-by-step goals, choosing tools, evaluating outputs, and fixing its own errors until a multi-step objective is accomplished.

I created an interactive demonstration project to showcase how these three concepts interact in practice. You can test the live application via the Interactive Agentic AI Chatbot Demo or inspect the implementation details in the Agentic AI GitHub Repository.

Defining Tool Calling vs Retrieval

Standard Retrieval-Augmented Generation (RAG) fetches relevant text chunks from a pre-indexed vector database before generating an answer. Tool calling goes a step further: the model dynamically selects which API endpoint or utility function to invoke based on user input, enabling both read and write operations across external environments.

Comparing Traditional LLMs vs Agentic Workflows

To evaluate how agentic architectures expand system capabilities compared to traditional language model deployments, consider the key operational dimensions summarized below:

Capability Dimension Standard LLM RAG System Agentic AI Chatbot
Knowledge Scope Static training weights External document database Dynamic multi-source search
Execution Ability Read-only text generation Read-only context retrieval Read-write external API actions
Control Loop Single-pass prediction Single-pass prompt retrieval Iterative reasoning and tool execution
Error Recovery Cannot self-correct Fixed retrieval pipeline Evaluates tool output and retries

Standard LLMs are restricted to knowledge contained in their static training weights and can only produce text output. RAG systems improve knowledge coverage by retrieving relevant documents from a database, but remain read-only pipelines. An Agentic AI chatbot introduces two-way interaction: it reads live web data and writes changes to external services, such as creating repository issues.

Explore technical depth

How the Chatbot Architecture Executes Actions

The demonstration chatbot follows an iterative execution loop. When a user submits a request, the central language model inspects the available tool schemas - including local document search, web search, Wikipedia lookup, and GitHub issue creation.

---
title: "Agentic Chatbot Execution Loop"
---
flowchart TB
    accTitle: Agentic chatbot execution loop
    accDescr: The model selects a tool, receives its output, evaluates whether the goal is complete, and either responds or chooses another tool.
    A["User Prompt"] --> B["LLM Reasoning Engine"]
    B --> C{"Select Tool?"}
    C -->|"Document Search"| D["Retrieve Local Context"]
    C -->|"Web/Wiki Search"| E["Fetch External API Data"]
    C -->|"Create Issue"| F["Execute GitHub API Call"]
    D --> G["Return Tool Output"]
    E --> G
    F --> G
    G --> B
    B --> H["Final User Response"]

The execution pipeline consists of four main phases:

  1. Input Evaluation: The user prompt enters the system execution layer where the language model analyzes the requested objective.
  2. Tool Selection: The model evaluates whether internal knowledge is sufficient or if an external tool call is required.
  3. Execution & Feedback: If a tool is selected, the application layer runs the function (such as calling the GitHub API) and returns the output back to the model context.
  4. Synthesis & Response: The model evaluates the tool output to determine if further action is needed before producing the final response to the user.

Implementing Tool Calling in Python

Implementing tool calling requires defining Python functions with clear docstrings and type annotations. Frameworks like LangChain parse these annotations into JSON schemas that describe the tool's purpose and expected arguments to the language model.

agent_chatbot.py
from langchain_community.agent_toolkits import create_python_agent
from langchain_core.tools import tool
import requests

@tool
def create_github_issue(title: str, body: str) -> str:
    """Creates a new issue in the designated GitHub repository."""
    url = "https://api.github.com/repos/kanad1323/agentic-ai-output/issues"  # (1)
    headers = {"Authorization": "token GITHUB_TOKEN"}
    payload = {"title": title, "body": body}
    response = requests.post(url, json=payload, headers=headers)  # (2)
    if response.status_code == 201:
        return f"Issue created successfully: {response.json().get('html_url')}"
    return f"Failed to create issue: {response.status_code}"

tools = [create_github_issue]  # (3)
  1. Defines the target GitHub API endpoint for issue creation within the destination repository.
  2. Transmits an authenticated HTTP POST request carrying the structured title and body generated by the model.
  3. Registers the tool function within the agent execution array, providing schema metadata to the model prompt.

Managing API Safety and Side Effects

Tools that perform write operations (such as creating issues, sending emails, or updating databases) introduce state changes in external systems. Production agent implementations should enforce rate limiting, authentication scopes, and user confirmation steps to prevent unintended API calls during model retry loops.

Key Takeaways for Building Agentic Systems

Designing effective agentic applications requires balancing reasoning flexibility with execution safety:

  • Reasoning Over Memory
    Agents use LLMs to make dynamic decisions about tool selection rather than relying strictly on fixed training data or static search indexes.

  • Structured Tool Interfaces
    Clear function docstrings and typed parameters allow language models to output precise API payloads without custom syntax parsing.

  • Autonomous Action
    Extending read-only query processing to write-enabled API integrations enables models to perform real-world tasks like issue creation.

  • Safety & Oversight
    Production agent workflows require rate limits and verification steps to prevent unintentional API calls during iterative execution.

Conclusion

Agentic AI transforms language models from isolated text generators into active components of modern software architectures. By combining reasoning capabilities with structured tool calling, developers can build systems that gather real-time data, navigate external APIs, and execute tasks autonomously.

Understanding the mechanics of tool calling - from function schema definition to iterative execution loops - provides the foundation for building reliable AI assistants while maintaining proper safeguards around automated write operations.

References and further reading

Open the complete reference catalog

Primary Sources