Skip to content

MultiAI-Query: Query Multiple AI Models with a Single Prompt

Comparing large language models (LLMs) is awkward when each provider uses a separate interface and response format. MultiAI-Query sends the same prompt and selected settings to several model APIs - including OpenAI, Mistral, Llama through Groq, and Google Gemini - then writes the responses to one Markdown file.

This post explains the dispatch pipeline, the configuration that must remain comparable, and the limits of treating one side-by-side response as a model evaluation.

Navigate this post

What is MultiAI-Query?

MultiAI-Query is an open-source Python tool for collecting model responses in one place. Instead of copying a prompt across several developer consoles, it sends the request concurrently, formats each completion, and writes them to response.md.

Multi-Provider Access

Evaluating outputs across diverse model architectures - comparing proprietary models against open-weights models served via high-throughput LPUs (Language Processing Units) - provides immediate visibility into cost, latency, and reasoning trade-offs.

Comparing model responses under identical prompt conditions helps developers spot hallucination risks, evaluate reasoning variations, and select the optimal model for specific production workloads.

Multi-Model Architecture & Execution Flow

The MultiAI-Query execution pipeline follows four distinct stages: prompt ingestion, configuration setup, concurrent API dispatch, and Markdown aggregation.

When a user submits a prompt, the engine loads API credentials from environment variables, attaches configurable system instructions, and constructs standard request payloads for each selected provider endpoint. Responses are collected asynchronously and formatted into structured Markdown sections.

---
title: "MultiAI-Query Request Dispatch Pipeline"
---
flowchart TB
    accTitle: MultiAI-Query request dispatch pipeline
    accDescr: One prompt and its settings are sent to several model APIs, then their responses are collected into one comparison document.
    A["User Prompt Input"] --> B["MultiAI-Query Engine"]
    B --> C["System Message & Hyperparameters"]
    C --> D1["OpenAI API"]
    C --> D2["Groq LPU API"]
    C --> D3["Mistral AI API"]
    C --> D4["Google Gemini API"]
    D1 --> E["Response Aggregator"]
    D2 --> E
    D3 --> E
    D4 --> E
    E --> F["Formated response.md Output"]

The pipeline ensures that network delays from a single slow API provider do not block the processing of other completed model responses. The final aggregator orders outputs deterministically based on the user's initial configuration list.

Key Features & Configuration Parameters

MultiAI-Query exposes several key configuration options that let users control model behavior and output formatting:

Temperature

Controls creativity versus predictability in model generation. Lower values (e.g., 0.2) make responses focused and deterministic, while higher values (e.g., 0.8) produce more creative and varied answers.

Top-p (Nucleus Sampling)

Limits the model's word choices to the top percentage of likely options (e.g., 0.9 cuts out the bottom 10% unlikely words), preventing strange or off-topic phrases.

Max Tokens

Defines the maximum response length (in words/word fragments) generated per model call, preventing runaway output and controlling API costs.

Optimizing Token Budgets

When running batch evaluations across multiple models, set max_tokens between 500 and 1,500 tokens. This prevents unexpected credit spend on verbose completions while providing sufficient depth for reasoning tasks.

Provider Comparison & Technical Benchmarks

Each supported provider offers distinct advantages depending on context length, inference throughput, and task specialization. The table below summarizes key parameters for the default models integrated into MultiAI-Query:

Provider API Default Model Context Window (Tokens) Max Output (Tokens) Primary Technical Strength
OpenAI gpt-4o 128,000 4,096 Complex logic & multi-step coding
Groq llama-3.3-70b-versatile 128,000 8,192 Ultra-fast LPU inference speed
Mistral AI mistral-large-latest 128,000 8,192 Concise reasoning & European language support
Google Gemini gemini-1.5-pro 2,000,000 8,192 High-context document ingestion

While gemini-1.5-pro leads in context capacity with support for up to 2 million tokens, Groq's custom LPU architecture delivers superior token generation speed for open-weights models like llama-3.3-70b. OpenAI's gpt-4o maintains high accuracy on structured output generation. Using MultiAI-Query allows developers to select models based on verifiable empirical metrics rather than theoretical claims.

Rate Limits & API Key Security

Always store API credentials in a .env file rather than hardcoding keys into scripts. Additionally, be aware of per-minute rate limits (RPM/TPM) when dispatching high-frequency prompt requests to free-tier provider accounts.

Getting Started & Usage Example

Setting up MultiAI-Query requires setting your provider API keys and calling the query execution function. The snippet below demonstrates how to initialize the engine and query four models simultaneously:

multiai_query.py
import os
from multiai import MultiQueryEngine

# Initialize the multi-model evaluation engine # (1)
engine = MultiQueryEngine(
    models=["gpt-4o", "llama-3.3-70b-versatile", "mistral-large-latest", "gemini-1.5-pro"],
    temperature=0.7,
    max_tokens=1000,
)

# Dispatch prompt payload concurrently to all provider endpoints # (2)
results = engine.query(
    prompt="Explain quantum entanglement using an everyday analogy.",
    system_message="You are a clear and concise science communicator.",
)

# Aggregate responses into a side-by-side Markdown document # (3)
engine.save_markdown("response.md", results)
  1. Configures the active model list, sampling temperature, and token generation ceiling.
  2. Dispatches the prompt and system instructions to all configured API endpoints concurrently.
  3. Formats all model outputs into response.md with headers, metadata, and syntax highlighting.

For complete installation steps and setup instructions, visit the kanad13/MultiAI-Query GitHub repository.

Architectural Takeaways

  • Unified API Abstraction
    Queries multiple LLM endpoints through a single interface, eliminating boilerplate code for each provider.

  • Concurrent Dispatch
    Sends prompts asynchronously across providers to minimize latency during multi-model evaluation runs.

  • Structured Markdown Output
    Aggregates completions into response.md with side-by-side formatting for evaluation and auditing.

  • Extensible Provider Schema
    Allows developers to register new model endpoints or custom system messages with minimal configuration changes.

Why Open Source?

MultiAI-Query is published under an open-source license to provide developers, prompt engineers, and AI researchers with a transparent, customizable tool for multi-model experimentation. Community contributions enable rapid additions of new provider APIs, custom output exporters, and benchmarking integrations.

Conclusion

MultiAI-Query reduces the overhead of evaluating multiple LLMs by providing a lightweight, concurrent querying interface. By standardizing prompt inputs across OpenAI, Groq, Mistral, and Gemini, developers can evaluate speed, accuracy, and output quality side by side. While API rate limits and individual provider pricing structures remain operational constraints, using a unified dispatch script enables informed decisions when choosing AI models for application development.

References and further reading

Open the complete reference catalog

Primary Sources