PromptMage

Skip to content Welcome to PromptMage PromptMage is a python framework to simplify the development of complex, multi-step applications based on LLMs. Get Started Learn More WARNING This application is currently in alpha state and under active development. Please be aware that the API and features may change at any time. Set up in 5 minutes Get PromptMage up and running quickly with simple installation steps. Deploy locally or on your server with ease. Getting started Version Control Built-in Track prompt development with integrated version control, making collaboration and iteration seamless. Learn more Prompt Playground Test, compare, and refine prompts in an intuitive interface designed for rapid iteration. Playground Auto-generated API Leverage a FastAPI-powered, automatically created API for easy integration and deployment. API Documentation Evaluation Mode Assess prompt performance through manual and automatic testing, ensuring reliability before deployment. Evaluation Guide More to Come Stay tuned for upcoming features and enhancements as we continue to evolve PromptMage. Roadmap About the Project PromptMage is a python framework to simplify the development of complex, multi-step applications based on LLMs. It is designed to offer an intuitive interface that simplifies the process of creating and managing LLM workflows as a self-hosted solution. PromptMage facilitates prompt testing and comparison, and incorporates version control features to help users track the development of their prompts. Suitable for both small teams and large enterprises, PromptMage seeks to improve productivity and foster the practical use of LLM technology. The approach with PromptMage is to provide a pragmatic solution that bridges the current gap in LLM workflow management. We aim to empower developers, researchers, and organizations by making LLM technology more accessible and manageable, thereby supporting the next wave of AI innovations. Take the walkthrough to see what you can do with PromptMage. Philosophy Integrate the prompt playground into your workflow for fast iteration Prompts as first-class citizens with version control and collaboration features Manual and automatic testing and validation of prompts Easy sharing of results with domain experts and stakeholders build-in, automatically created API with fastAPI for easy integration and deployment Type-hint everything for automatic inference and validation magic Projects using PromptMage product-review-research: An AI webapp build with PromptMage to provide in-depth analysis for products by researching trustworthy online reviews. Development To develop PromptMage, check out the DEVELOPMENT.md file. Contributing We welcome contributions from the community! If you're interested in improving PromptMage, you can contribute in the following ways: * Reporting Bugs: Submit an issue in our repository, providing a detailed description of the problem and steps to reproduce it. * Improve documentation: If you find any errors or have suggestions for improving the documentation, please submit an issue or a pull request. * Fixing Bugs: Check out our list of open issues and submit a pull request to fix any bugs you find. * Feature Requests: Have ideas on how to make PromptMage better? We'd love to hear from you! Please submit an issue, detailing your suggestions. * Pull Requests: Contributions via pull requests are highly appreciated. Please ensure your code adheres to the coding standards of the project, and submit a pull request with a clear description of your changes. To ensure a smooth contribution process, please follow these guidelines: * create an issue before submitting a pull request to discuss the changes you'd like to make. This helps us ensure that your contribution aligns with the project's goals and prevents duplicate work. * follow the coding standards of the project. Check the DEVELOPMENT.md file for more information. Make sure to check if your issue or PR has already been fixed or implemented before opening a new one! Contact For any inquiries or further information, feel free to reach out at promptmage@tobiassterbak.com. ❤️ Acknowledgements This project was supported by Back to top --- Skip to content Getting Started Installation To install promptmage, run the following command: pip install promptmage Annotated Code Example Here is an example of how to use promptmage in your application: from promptmage import PromptMage, Prompt, MageResult # Create a new promptmage instance mage = PromptMage( name="example", ) Steps are the building blocks of a flow. They are used to define the different parts of the flow and to connect them together. A step is just a python function with the @mage.step() decorator which returns a MageResult. Here is an example of how to create a step: @mage.step( name="step1", prompt_name="prompt1", initial=True ) def step1(question: str, prompt: Prompt) -> MageResult: response = client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": prompt.system}, { "role": "user", "content": prompt.user.format(question=question), }, ], ) answer = response.choices[0].message.content return MageResult( next_step=None, result=answer ) Usage Put the above code in a file called flow.py and setup the OpenAI client. To run the flow with promptmage, run the following command: promptmage run flow.py This will start the promptmage server and run the flow at the given path. You can now access the promptmage interface at http://localhost:8000/gui/. Usage with a remote backend server For a production setup and collaborative usage with teams you can run the promptmage server with a remote backend. To run the remote backend on a remote server, run the following command: promptmage serve --port 8021 To connect your promptmage script to the remote backend, you need to add the remote url to the PromptMage instance of your script: mage = PromptMage( name="example", remote="http://localhost:8021" ) Now you can run your script and the promptmage server will use the remote backend to run the flow and store the results. GUI walkthrough The promptmage interface is divided into four main sections: the flow playground, the run history, the prompt repository, and the evaluation section. Flow playground Initial flow playground for the example flow. Edit the step prompt of step 1. After the run you can see the execution graph and the results. Run history Here you can see all your runs and the results. By clicking on a run, you can look at the details. Prompt repository You can see all your prompts and versions in the prompts repository. More examples Have a look at the examples in the examples folder to see how to use promptmage in your application or workflow. Use with Docker You can find an usage example with docker here: Docker example. Back to top --- Skip to content Roadmap 2024 August Implement a dynamic execution graph for flows Implement an evaluation mode for applications September Implement a remote backend for PromptMage Improve error handling and reporting October More complex use-case examples Implement a robust task queue for LLM calls November Implement automatic evaluation with llm-as-a-judge December more to come! 2025 Back to top --- Skip to content Tutorial Welcome to the PromptMage tutorial! This tutorial will guide you through the basics of PromptMage, and show you how integrate it into your own LLM project. Use case For this tutorial, we want to build a simple multi-step LLM application. It contains multiple dependent steps, where the output of one step is used as the input for the next step. The application will be used to summarize an input text with extracting facts to summarize from. The application will have the following steps: Step 1: Extract facts from a given text Step 2: Summarize the text using the extracted facts We assume all the steps are implemented as separate Python functions that take input and return output in one python file summarizer.py. Step 1: Install PromptMage First, we need to install PromptMage. You can install PromptMage using pip: pip install promptmage It is recommended to install PromptMage in a virtual environment to avoid conflicts with other packages. Step 2: Add PromptMage to your project First, you need to add PromptMage to your project. You do that by adding the following to your summarizer.py file: # Create a new PromptMage instance mage = PromptMage(name="fact-summarizer") Next, you need to define the prompts and dependencies between the steps. You can do that by adding the following code to the functions in the summarizer.py file: @mage.step(name="extract", prompt_name="extract_facts", initial=True) def extract_facts(article: str, prompt: Prompt) -> str: # return MageResult(facts=facts, next_step="summarize") As a first step, this needs to be the initial step, so we set the initial parameter to True. This will be the first step that is executed when the application is run. Every step needs to return a MageResult object, which contains the output of the step and the name of the next step to be executed. In this case, the next step is the summarize step. Note, that you can also return a list of MageResult objects if you want to execute multiple steps in parallel. @mage.step(name="summarize", prompt_name="summarize_facts") def summarize_facts(facts: str, prompt: Prompt) -> str: # return MageResult(summary=summary) If the next_step is not specified, the step will be considered a terminal step and the application will stop after executing this step. Now you can access the prompts within the step functions using the prompt argument. The prompt argument is an instance of the Prompt class, which provides methods to interact with the prompt. By default we have a system and a user prompt available by prompt.system and prompt.user respectively. The prompts are later created in the web UI. You don't need to worry about saving the prompts and data, PromptMage will take care of that for you. Step 3: Run the application Now you can run the application by promptmage run summarizer.py This will start the PromptMage web UI, where you can interact with the prompts and run and see the output of the steps. You can access the web UI at http://localhost:8000/gui/. More examples can be found in the examples folder. Back to top