AI Agent Service Toolkit

AI Agent Service Toolkit

https://github.com/fanqingsong/agent-service-toolkit

 

A full toolkit for running an AI agent service built with LangGraph, FastAPI and Streamlit.

It includes a LangGraph agent, a FastAPI service to serve it, a client to interact with the service, and a Streamlit app that uses the client to provide a chat interface. Data structures and settings are built with Pydantic.

This project offers a template for you to easily build and run your own agents using the LangGraph framework. It demonstrates a complete setup from agent definition to user interface, making it easier to get started with LangGraph-based projects by providing a full, robust toolkit.

 

https://agent-service-toolkit.streamlit.app/

Key Features

1. LangGraph Agent and latest features: A customizable agent built using the LangGraph framework. Implements the latest LangGraph v0.3 features including human in the loop with interrupt(), and flow control with Command, and langgraph-supervisor.
2. FastAPI Service: Serves the agent with both streaming and non-streaming endpoints.
3. Advanced Streaming: A novel approach to support both token-based and message-based streaming.
4. Streamlit Interface: Provides a user-friendly chat interface for interacting with the agent.
5. Multiple Agent Support: Run multiple agents in the service and call by URL path. Available agents and models are described in /info
6. Asynchronous Design: Utilizes async/await for efficient handling of concurrent requests.
7. Content Moderation: Implements LlamaGuard for content moderation (requires Groq API key).
8. Feedback Mechanism: Includes a star-based feedback system integrated with LangSmith.
9. Docker Support: Includes Dockerfiles and a docker compose file for easy development and deployment.
10. Testing: Includes robust unit and integration tests for the full repo.

Key Files

The repository is structured as follows:

• src/agents/: Defines several agents with different capabilities
• src/schema/: Defines the protocol schema
• src/core/: Core modules including LLM definition and settings
• src/service/service.py: FastAPI service to serve the agents
• src/client/client.py: Client to interact with the agent service
• src/streamlit_app.py: Streamlit app providing a chat interface
• tests/: Unit and integration tests