Blog/Inside KalshiTrader: How We Built an Agentic Architecture for Automated Prediction Market Trading
Agentic AITradingArchitecturePrediction MarketsKalshiTrader

Inside KalshiTrader: How We Built an Agentic Architecture for Automated Prediction Market Trading

April 26, 202610 min readBy Brad McCorkle, Founder & CEO, Lesos AI

An agentic architecture is a system design pattern where autonomous AI agents perceive their environment, make decisions, and take actions in a continuous loop — without requiring a human to approve each step. Applied to financial trading, this means an AI system that monitors markets, identifies opportunities, executes trades, and manages positions on its own. KalshiTrader is built on this pattern, connecting to the Kalshi prediction market exchange to trade automatically on behalf of users.

Most trading bots are rule-based: if price crosses a threshold, buy. If it crosses another, sell. These systems are brittle. They cannot adapt to new information, they cannot reason about why a market is moving, and they fail on event types they were not explicitly programmed to handle. Prediction markets are particularly hostile to rigid rule systems because the underlying events — Fed rate decisions, CPI releases, weather outcomes, election results — each have fundamentally different dynamics.

KalshiTrader takes a different approach. Instead of encoding rules, we built an agentic system where a large language model (Google Gemini) acts as the reasoning core. The agent observes market state, reasons about event probabilities using real-world context, and decides whether to enter, hold, or exit positions. This article explains how that architecture works, why we made the design choices we did, and what we learned building it.

What Makes an Architecture "Agentic"

The term "agentic" gets thrown around loosely, so it is worth being precise. An agentic system has four properties that distinguish it from a standard API integration or rule-based bot:

  • Perception — The agent continuously observes its environment. In KalshiTrader, this means polling the Kalshi API for current market prices, portfolio positions, and account balance.
  • Reasoning — The agent interprets observations and forms a plan. Gemini analyzes market data, event context, and probability distributions to decide whether an opportunity exists.
  • Action — The agent executes decisions in the real world. KalshiTrader places buy and sell orders on the Kalshi exchange via their API, using the user's linked brokerage account.
  • Autonomy — The loop runs without human intervention. Once the user enables trading and sets their preferences, the system operates continuously until paused.

The critical difference between an agentic system and a chatbot or copilot is that the agent acts. A copilot suggests trades for a human to execute. An agent executes them. This distinction has profound implications for architecture — when the system takes real financial actions, every component must be designed for reliability, safety, and auditability.

System Architecture Overview

KalshiTrader's architecture has five major components, each handling a distinct responsibility in the agent loop:

  • Mobile Client (React Native / Expo) — The user interface. Handles authentication via Google Sign-In, displays portfolio positions and P&L, and provides controls for trading preferences and the kill switch.
  • Backend API (Azure-hosted) — The coordination layer. Manages user accounts, stores encrypted credentials, orchestrates the agent loop, and exposes endpoints for the mobile client.
  • AI Reasoning Engine (Google Gemini) — The decision-making core. Receives market data and returns structured trade signals with confidence scores and rationale.
  • Kalshi Integration Layer — Handles all communication with the Kalshi exchange API. Executes trades, fetches portfolio state, and monitors positions.
  • Notification Service (Firebase Cloud Messaging) — Pushes real-time alerts to users for trade entries, exits, and P&L updates.

These components form a closed loop: the backend polls Kalshi for market state, sends relevant data to Gemini for analysis, receives trade signals back, executes orders on Kalshi, updates the user's portfolio, and notifies them via push notification. The entire cycle runs autonomously.

The Agent Loop: Perception → Reasoning → Action

Step 1: Market Perception

The agent's perception layer continuously polls the Kalshi API for market data. But prediction markets have a unique challenge compared to equity or crypto markets: there are hundreds of active markets across fundamentally different categories. A Fed rate decision market behaves nothing like a weather market, which behaves nothing like a crypto price market.

KalshiTrader addresses this with user-configurable market category filters. Users choose from 10 categories — Fed rates, CPI, GDP, employment, crypto, weather, politics, earnings, mentions, and other — and the agent only monitors markets in enabled categories. This serves two purposes: it respects user preferences and it reduces the search space for the reasoning engine, improving signal quality.

Step 2: AI Reasoning

When the perception layer identifies potentially interesting markets, it packages the relevant data and sends it to Google Gemini for analysis. Critically, no user personal information is sent to Gemini — only market data (tickers, prices, event descriptions, expiration dates). The LLM acts as a pure reasoning engine over public market information.

Gemini's analysis produces structured trade signals that include: a directional recommendation (buy yes, buy no, or no action), a confidence score, a rationale explaining the reasoning, and suggested position sizing. The structured output format is enforced programmatically so the downstream execution layer can parse it reliably — a critical requirement when the output drives real financial actions.

Why Gemini and not a fine-tuned model? Prediction markets span dozens of event types with constantly changing context. A fine-tuned model trained on historical data would struggle with novel event types (e.g., a new type of government report or an unprecedented weather event). Gemini's broad world knowledge and reasoning capability let it analyze event types it has never seen specific training data for — which is exactly what prediction markets demand.

Step 3: Trade Execution

When the reasoning engine returns an actionable signal, the execution layer takes over. This is where the agentic pattern carries the most risk — the system is about to spend the user's money. The execution layer enforces several safety constraints before placing any order:

  • Kill switch check — Is automated trading still enabled? The user can toggle this at any time from the app.
  • Category check — Is this market's category still enabled in the user's preferences?
  • Balance check — Does the user have sufficient balance for the position?
  • Duplicate check — Does the user already hold a position in this market?

Only after all checks pass does the system submit the order to Kalshi's API. The order response (fill price, quantity, order ID) is recorded and the user receives a push notification with the trade details.

Position Management and Exit Strategies

Entering a trade is only half the problem. The agent must also monitor open positions and decide when to exit. This is arguably the harder task — market conditions evolve, new information arrives, and the optimal exit timing depends on factors that change continuously.

KalshiTrader's exit strategy evaluation runs as a separate agent loop. The backend periodically reviews all open positions, checking current market prices against entry prices, evaluating time remaining until market expiration, and re-querying Gemini for updated analysis when conditions have materially changed. Exit signals can be triggered by profit targets, stop-loss thresholds, or the AI determining that the original trade thesis is no longer valid.

When an exit is triggered, the system sells the position, records the P&L, and pushes a notification to the user. The full lifecycle — entry, monitoring, exit — runs autonomously.

Security Architecture

An agentic system that executes financial transactions demands a security model that assumes breach. If any single component is compromised, the blast radius must be limited. Here is how KalshiTrader implements defense in depth:

  • Credential encryption — Kalshi brokerage credentials are encrypted with AES-256-GCM at rest using a derived master key. They are never stored in plain text and are only decrypted in memory during API calls.
  • On-device security — Sensitive tokens (session JWTs, authentication state) are stored via Expo SecureStore, which uses the Android Keystore on Android devices. This is hardware-backed encryption that the OS manages.
  • Transport security — All API communication uses HTTPS. Backend requests from the mobile client are signed with HMAC to prevent tampering and replay attacks.
  • Rate limiting — All API endpoints are rate-limited to prevent abuse, both from external attackers and from bugs in the client that might cause runaway requests.
  • Authentication — Users authenticate via Google Sign-In (OAuth 2.0). The backend issues self-signed JWTs for session management with appropriate expiration.

The most important security feature in an agentic trading system is the kill switch. No matter how robust your encryption or authentication, users must be able to immediately halt all automated activity with a single tap. KalshiTrader's kill switch is a server-side flag — disabling it immediately prevents any new trades from being placed, regardless of what the AI recommends.

Observability: Crash Reports, Analytics, and Audit

Autonomous systems that take real-world actions need comprehensive observability. When something goes wrong at 3 AM, you need to reconstruct exactly what happened. KalshiTrader uses three layers of observability:

  • Sentry — Crash reporting and error monitoring. Captures unhandled exceptions, stack traces, and device context. Configured to exclude financial data from error reports — crash logs contain device info and code-level errors, never portfolio data or credentials.
  • PostHog — Product analytics. Tracks anonymized usage patterns: which features are used, how often users check positions, which market categories are most popular. No personally identifiable information is sent.
  • Activity logs — Server-side event log of every trade placed, every exit executed, every AI analysis run. This is the audit trail. It records what happened, when, and why (including the AI's rationale for each trade signal).

Lessons Learned Building Agentic Trading Systems

After building and operating KalshiTrader, several lessons stand out that apply to any agentic system that takes real-world actions:

  • Structured output is non-negotiable — When an LLM's output drives financial actions, you cannot tolerate format variability. Enforce output schemas programmatically and reject responses that do not conform.
  • The kill switch is a first-class feature — Not an afterthought, not a debug flag. It must be prominently accessible in the UI, instantly effective server-side, and tested regularly.
  • Separate the perception loop from the action loop — Observing markets and executing trades should be decoupled. This lets you update observation frequency without changing execution logic, and it lets you pause execution while continuing to observe.
  • Never send PII to the reasoning engine — Design the data pipeline so the LLM only receives market data. This is both a privacy requirement and a simplification — the LLM does not need to know who is trading, only what the market looks like.
  • User control is table stakes — Users must be able to configure what the agent trades (category filters), whether it trades (kill switch), and verify what it has traded (activity history). An autonomous agent that cannot be constrained by its principal is not a product — it is a liability.

The Agentic Future of Financial Applications

KalshiTrader represents a specific application of a general pattern: autonomous AI agents operating in real markets with real money. The same architecture — perception, reasoning, action, with safety constraints and human override — applies to portfolio rebalancing, options hedging, arbitrage detection, and any domain where decisions can be automated but must be governed.

The key insight from building KalshiTrader is that the hard problems are not in the AI reasoning — Gemini handles market analysis well. The hard problems are in everything around it: secure credential management, reliable execution, graceful degradation, user control, and observability. Getting the architecture right matters more than getting the model right.

Try KalshiTrader

AI-powered automated prediction market trading on the Kalshi exchange. Set your preferences, enable trading, and let the agent work for you.

Learn More