diff --git a/.gitignore b/.gitignore index 68bc17f9ff2104a9d7b6777058bb4c343ca72609..0bcd49f250986aa59a1f874ff0d99cbce2160259 100644 --- a/.gitignore +++ b/.gitignore @@ -158,3 +158,40 @@ cython_debug/ # and can be added to the global gitignore or merged into this file. For a more nuclear # option (not recommended) you can uncomment the following to ignore the entire idea folder. #.idea/ + +# Others +*.DS_Store + +# log +result/ +logs/ +lib/ + +# VSCODE +vscode/ +.vscode/ + +#.pytest_cache +.pytest_cache/ + +# others +.coverage +.coveragec + +# vis +result/example.html +example.html +datasets/gaia/test_files/ +datasets/gaia/val_files +datasets/gaia/*.json +datasets/gaia/utils/*.jsonl + +/runs/ + +.$*.drawio.bkp +/diagrams/*.pdf +/diagrams/*.png +/figures/*.pdf +/figures/*.png + +.pypirc \ No newline at end of file diff --git a/README.md b/README.md index 4b84550ab1237fa05a2127c64862463907ea07cc..e51486ff3696f5d0da939d90ad63d2a95d1e8b8d 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,139 @@ -# GPTSwarm -๐ GPTSwarm: LLM agents as Graphs +[](https://gptswarm.org) +[](https://arxiv.org/pdf/2305.17066.pdf) +[](https://github.com/mczhuge/GPTSwarm/blob/main/LICENSE) +[](https://github.com/metauto-ai/gptswarm/issues) +[](https://twitter.com/AI_KAUST) +[](https://metauto.ai/images/wechat.jpeg) + +<p align="left"> +<a href=""><img src="swarm/utils/assets/logo.png" alt="GPTSwarm" width="430px"></a> +</p> + +๐ **GPTSwarm is a graph-based framework for LLM-based agents, providing two high-level features:** + +* It lets you build LLM-based agents from graphs. +* It enables the customized and automatic self-organization of agent swarms with self-improvement capabilities. + + +## Edge optimization example + +Here is the edge optimization process that updates edge probabilities toward improvement of the benchmark score. Notice that within an agent, the edges are fixed, whereas the inter-agent connections are getting optimized towards either edge pruning (value 0, blue) or creation (value 1, red). + +<img src="assets/edge_opt.gif" alt="Edge optimization" width="300"> + +## About GPTSwarm + +<img src="assets/gpt_swarm.png" alt="Framework" width="800"> + +At a granular level, GPTSwarm is a library that includes the following components: + + +| Module | Description | +| ---- | --- | +| [**swarm.environment**](swarm/environment) | Domain-specific operations, agents, tools, and tasks | +| [**swarm.graph**](swarm/graph) | Graph-related functions for creating and executing agent graphs and swarm composite graphs | +| [**swarm.llm**](swarm/llm) | Interface for selecting LLM backends and calculating their operational costs | +| [**swarm.memory**](swarm/memory) | Index-based memory | +| [**swarm.optimizer**](swarm/optimizer) | Optimization algorithms designed to enhance agent performance and overall swarm efficiency | + + +## Quickstart + +**Clone the repo** + +```bash +git clone --recurse-submodules https://github.com/mczhuge/GPTSwarm.git +cd GPTSwarm/ +``` + +**Install packages** +``` +conda create -n swarm python=3.10 +conda activate swarm +pip install -r requirements_py310_<linux|macos>.txt +``` + +**You should add API keys in `.env.template` and change its name to `.env`** + +```python +OPENAI_API_KEY="" # for OpenAI LLM backend +SEARCHAPI_API_KEY="" # for Web Search +``` + +**Getting started with GPTSwarm is easy. Quickly run a predefined swarm** + +```python +from swarm.graph.swarm import Swarm + +swarm = Swarm(["IO", "IO", "IO"], "gaia") +task = "What is the capital of Jordan?" +inputs = {"task": task} +answer = await swarm.arun(inputs) +``` + +**or make use of tools, such as the file analyzer** + +```python +from swarm.graph.swarm import Swarm +swarm = Swarm(["IO", "TOT"], "gaia") +task = "Tell me more about this image and summarize it in 3 sentences." +files = ["./datasets/demos/js.png"] +inputs = {"task": task, "files": files} +danswer = swarm.run(inputs) +``` + +Check out the minimal Swarm example in Colab here: [](https://colab.research.google.com/github/mczhuge/GPTSwarm/blob/main/notebooks/demo_swarm.ipynb). + +See how to create a custom Agent and run a Swarm with it here: [](https://colab.research.google.com/github/mczhuge/GPTSwarm/blob/main/notebooks/demo_custom_agent.ipynb). + +Watch the following for an example of how to run the notebooks. + +https://github.com/mczhuge/GPTSwarm/assets/100462607/a05f78b7-5f9d-4762-8337-973702ce4493 + + +**๐ See [our experiments](https://github.com/mczhuge/GPTSwarm/tree/main/experiments) for more advanced use of our framework.** + +## Class diagram + +<img src="assets/class_diagram.png" alt="Edge optimization" width="700"> + +## Example of the Swarm + +<img src="assets/swarm_v3.png" alt="Edge optimization" width="500"> + +## More Visualizations + +<img src="assets/swarm_vis.png" alt="Edge optimization" width="800"> + +## Running with a local LLM + +We support local LM inference via [LM Studio](https://lmstudio.ai). Download their desktop app for Mac or Windows, choose a model from the Huggingface repository and start the server. Use `model_name='lmstudio'` in GPTSwarm code to run with the local LLM. + +<img src="assets/lm_studio.png" alt="Edge optimization" width="800"> + +## Initial Contributors + +* [Mingchen Zhuge](http://metauto.ai) (PhD Student, KAUST; Project Initiator) +* [Wenyi Wang](https://scholar.google.com/citations?user=79ODhuQAAAAJ&hl=en&oi=ao) (PhD Student, KAUST; Initial Participant) +* [Dmitrii Khizbullin](http://www.khizbullin.tech) (Research Engineer Lead, KAUST; Project Engineer Lead) +* [Louis Kirsch](http://louiskirsch.com) (PhD Student, IDSIA) +* [Francesco Faccio](https://scholar.google.com/citations?user=0z3DkrkAAAAJ&hl=en&oi=ao) (PostDoc, IDSIA; Visiting Researcher, KAUST) +* [Jรผrgen Schmidhuber](http://www.idsia.ch/~juergen/) (Director, KAUST AI Initiative; Scientific Director, IDSIA) + +Please read our [developer document](DEVELOPMENT.md) if you are interested in contributing. + + +## Citation +Please cite our paper if you find the library useful or interesting. +``` +@article{zhuge2024gptswarm, + title={Language Agents as Optimizable Graphs}, + author={Zhuge, Mingchen and Wang, Wenyi and Kirsch, Louis and Faccio, Francesco and Khizbulin, Dmitrii and Schmidhuber, Juergen}, + journal={Technical Report}, + year={2024} +} +``` + + + +