Sara AI
Personal AI assistant focused on workflow automation
Overview
Sara AI is an active engineering project focused on building an assistant that performs real actions. It interfaces directly with third-party software, executing tasks like calendar scheduling and reminder settings via natural language processing.
The Problem
Generic conversational agents lack integration with personal productivity tools, requiring users to manually copy and paste details between browser tabs to execute simple operations.
Objectives
Project Goals
- Translate plain text instructions into reliable external API payloads.
- Manage conversation memory to maintain state across multi-turn prompts.
- Provide streaming text responses for interactive user engagement.
Success Criteria
- Validate tool-calling JSON outputs against defined schemas before executing webhooks.
- Minimize response latency by streaming LLM text output.
Constraints
- Handling rate limits on external service APIs and ensuring reliable fallback execution.
Planning & Research
I selected FastAPI to handle asynchronous request volumes and selected OpenAI's structured outputs API to ensure the assistant's arguments consistently validate against my application schemas.
Architecture
Component Responsibilities
The frontend sends queries to the FastAPI service, which coordinates with the LLM to verify if a tool needs to be called, runs the local function block, and returns the result.
Data Flow
User types command -> FastAPI passes query to LLM -> LLM returns tool call request -> FastAPI runs local action script -> FastAPI returns formatted confirmation to user.
Infrastructure
The prototype runs on an asynchronous Python server and interfaces directly with cloud LLM endpoints.
Engineering Decisions
FastAPI Async Architecture
Asynchronous I/O handling is critical when managing multiple concurrently open HTTP connections to external APIs and LLM streaming portals.
Schema-Driven Tool Execution
Defining executable actions as Pydantic models ensures that input arguments conform strictly to type requirements before calling third-party services.
Implementation
I wrote the core assistant engine using Python, configured system prompts for predictable workflow execution, and built custom handlers to execute external calendar and task webhook requests.
Challenges & Trade-offs
LLM arguments occasionally drifting from target API structures
Enforce strict JSON schema parsing at the LLM level.
Configuring the API call to enforce strict JSON structure prevents invalid arguments from being generated, eliminating redundant network calls.
Results
Established a functional backend core capable of processing natural language commands into verified API operations.
Maintained tool schema validation failure rates under 1% during active testing.
Performance Metrics
- Successful multi-turn task state tracking
- Asynchronous API integration with zero blocking threads
- Under 100ms backend processing overhead prior to LLM response
Lessons & Reflections
Prompt engineering is a developer discipline that requires rigorous validation setups rather than simple ad-hoc testing.
Technology Stack
Gallery
