diff --git a/.github/workflows/python-package.yml b/.github/workflows/python-package.yml index bd850fb9d..01fa08c6f 100644 --- a/.github/workflows/python-package.yml +++ b/.github/workflows/python-package.yml @@ -336,4 +336,3 @@ jobs: echo "检测到Java文件被更改(根据环境变量APPBUILDER_JAVA_TESTS),准备启动Run Java tests with Jacoco部分..." cd cicd/app-builder/java bash print_coverage.sh - diff --git a/README.md b/README.md index 1b2605f06..52c5818b6 100644 --- a/README.md +++ b/README.md @@ -26,12 +26,12 @@ - 调用AI原生应用，通过[AppBuilderClient](/docs/BasisModule/Platform/Application/appbuilder_client.md)可访问并管理在百度智能云千帆AppBuilder[网页端](https://console.bce.baidu.com/ai_apaas/app)发布的AI原生应用，并可注册本地函数联动端云组件 - **编排** - 配置知识库，通过[KnowledgeBase](/docs/BasisModule/Platform/KnowledgeBase/knowledgebase.md)管理知识库，进行文档及知识切片的增删改查，配合[网页端](https://console.bce.baidu.com/ai_apaas/app)开发产业级的`RAG`应用 - - 编排工作流，提供了`Message`、`Component`、`AgentRuntime`多级工作流抽象，实现工作流编排，并可与LangChain、OpenAI等业界生态能力打通 + - 编排工作流，提供了`Message`、`Component`、`FlaskRuntime`、`ChainlitRuntime`多级工作流抽象，实现工作流编排，并可与LangChain、OpenAI等业界生态能力打通 - **监控** - 提供了可视化Tracing、详细DebugLog等监控工具，助力开发者在生产环境应用 - **部署** - - `AgentRuntime`支持部署为基于`Flask`与`gunicorn`的API服务 - - `AgentRuntime`支持部署为基于`Chainlit`的对话框交互前端 + - `FlaskRuntime`支持部署为基于`Flask`与`gunicorn`的API服务 + - `ChainlitRuntime`支持部署为基于`Chainlit`的对话框交互前端 - 提供了`appbuilder_bce_deploy`工具，可快速部署程序到百度云，提供公网API服务，联动AppBuilder工作流 ### 使用 AppBuilder-SDK 可以构建什么应用？ @@ -296,10 +296,9 @@ Hook: - [TRACE基础功能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/basic.md) - [TRACE拓展功能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/phoenix_method.md) - 部署: - - [交互式前端部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/AgentChainlit.md) + - [交互式前端部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/chainlit_deploy.md) - [公有云部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/cloud.md) - - [API 访问](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask.md) - - [AgentRuntime](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/agentruntime.md) + - [API 访问](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask_deploy.md) - [UserSession](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/usersession.md) - 平台: - 应用: diff --git a/cookbooks/components/chainlit_runtime.ipynb b/cookbooks/components/chainlit_runtime.ipynb new file mode 100644 index 000000000..ece31d389 --- /dev/null +++ b/cookbooks/components/chainlit_runtime.ipynb @@ -0,0 +1,297 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "id": "f802e64d-4eaa-445d-a48a-1042a91bc394", + "metadata": { + "tags": [] + }, + "source": [ + "# 基于ChainlitRuntime服务化组件\n", + "\n", + "## 目标\n", + "使用 ChainlitRuntime 对组件进行服务化。\n", + "\n", + "ChainlitRuntime 是对组件（Component）和应用的服务化封装，具体有如下几个功能：\n", + "- 一键服务化组件: 使得组件能够以服务的形式运行，支持对话框交互。\n", + "- Session 数据管理: 提供 Session 数据的管理功能，允许跟踪和存储用户会话数据。\n", + "- 请求时鉴权: 支持在请求时进行认证，确保安全性。\n", + "\n", + "\n", + "## 准备工作\n", + "### 安装Python SDK\n", + "\n", + "appbuilder 支持使用 pip 安装（要求Python >= 3.8），并且 ChainlitRuntime 服务化组件依赖 `appbuilder-sdk[serve]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2939356f-61c2-42e9-9e0c-fc6729c193f6", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "pip install appbuilder-sdk 'appbuilder-sdk[serve]'" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "aeb2fa55-075f-48df-a9fb-8b40d9900684", + "metadata": {}, + "source": [ + "## 基本用法\n", + "\n", + "### 1. 快速开始\n", + "\n", + "下面的示例会基于 Playground 组件，在 8091 端口部署 chainlit 服务: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41559341-fd7a-478c-a08b-1477d79e9d41", + "metadata": { + "ExecuteTime": { + "end_time": "2023-12-18T06:24:26.982459Z", + "start_time": "2023-12-18T06:23:53.771345Z" + } + }, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.chainlit_deploy import ChainlitRuntime\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "component = appbuilder.Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\"\n", + ")\n", + "\n", + "agent = ChainlitRuntime(component=component)\n", + "agent.chainlit_component(port=8091)" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "5fc5bc38-6bc5-4187-a8fd-f802d77d89fa", + "metadata": {}, + "source": [ + "### 2. ChainlitRuntime 参数说明" + ] + }, + { + "cell_type": "markdown", + "id": "c2364c35", + "metadata": {}, + "source": [ + "#### 2.1 类初始化参数说明\n", + "\n", + "ChainlitRuntime 初始化接受4个参数。\n", + "\n", + "| 参数名称 | 参数类型 | 是否必须 | 描述 | 示例值 |\n", + "|--|--|--|--|--|\n", + "| component | Component | 是 | 可运行的 Component, 该 Component 需要实现 run(message, stream, **args) 方法。 | Playground(prompt_template=\"{query}\", model=\"ERNIE-Bot\") |\n", + "| user_session_config | sqlalchemy.engine.URL\\|Str\\|None | 否 | 会话 Session 数据存储的数据库配置，遵循 sqlalchemy 后端定义，可参考[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)。默认使用 sqlite:///user_session.db，即本地的 SQLite 存储 | \"sqlite:///user_session.db\" |\n", + "| user_session| UserSession | 否 | 用户会话管理器，如果不指定则自动生成一个默认的 UserSession | UserSession(user_session_config) |\n", + "| tool_choice| ToolChoice| 否 | 可用于Agent强制执行的组件工具 | |" + ] + }, + { + "cell_type": "markdown", + "id": "61923b00", + "metadata": {}, + "source": [ + "## 高级用法" + ] + }, + { + "cell_type": "markdown", + "id": "a2303c76", + "metadata": {}, + "source": [ + "### 1. 一键服务化组件 \n", + "ChainlitRuntime 可以快速组件以服务的形式运行，支持对话框交互。" + ] + }, + { + "cell_type": "markdown", + "id": "673c7565", + "metadata": {}, + "source": [ + "**1.1 对话框交互组件-文本格式**\n", + "\n", + "基于 chainlit 的对话框交互对被服务化的组件的 message 参数更加严格，要求能够接受 Str 的基础类型。\n", + "\n", + "执行下面的代码，会启动一个 chainlit 页面，页面地址：0.0.0.0:8091" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e485544f", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.chainlit_deploy import ChainlitRuntime\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "component = appbuilder.Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\"\n", + ")\n", + "\n", + "agent = ChainlitRuntime(component=component)\n", + "agent.chainlit_component(port=8091)" + ] + }, + { + "cell_type": "markdown", + "id": "4d63bece", + "metadata": {}, + "source": [ + "Chainlit Demo页面示意图如下所示，\n", + "\n", + "![chainlit demo](image/agent_runtime_with_chainlit_demo.png)" + ] + }, + { + "cell_type": "markdown", + "id": "e6a93195", + "metadata": {}, + "source": [ + "**1.2 对话框交互组件-其他格式**\n", + "\n", + "对输出其他类型的组件，如图片、音频、代码等组件，提供渲染效果。\n", + "\n", + "注意：该接口目前只支持新协议组件，即appbuilder.core.component.v2实现下的组件。\n", + "\n", + "执行下面的代码，会启动一个 chainlit 页面，页面地址：0.0.0.0:8092" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "07348a19", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.chainlit_deploy import ChainlitRuntime\n", + "from appbuilder.core.components.v2 import SimilarQuestion\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "component = SimilarQuestion()\n", + "agent = ChainlitRuntime(component=component)\n", + "agent.chainlit_component_debug(\n", + " port=8092, \n", + " tool_eval_args={},\n", + " query_name = \"query\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "a92b9075", + "metadata": {}, + "source": [ + "Chainlit Demo页面示意图如下所示，\n", + "\n", + "![chainlit demo](image/agent_runtime_with_chainlit_component_debug.jpg)" + ] + }, + { + "cell_type": "markdown", + "id": "a074f367", + "metadata": {}, + "source": [ + "**1.2 对话框交互AppBuilderClient**\n", + "\n", + "基于 chainlit 的对话框交互AppBuilderClient，可实现对话交互。支持工作流Agent、自主规划Agent应用。\n", + "\n", + "执行下面的代码，会启动一个 chainlit 页面，页面地址：0.0.0.0:8091。可在页面上上传文档（可选）、执行对话。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0a75e035", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.chainlit_deploy import ChainlitRuntime\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = \"...\"\n", + "# 使用之前，在官网个人空间获取应用ID，如下图\n", + "app_id= \"...\"\n", + "client = appbuilder.AppBuilderClient(app_id)\n", + "agent = ChainlitRuntime(client)\n", + "agent.chainlit_agent(port=8091)" + ] + }, + { + "cell_type": "markdown", + "id": "d5162e29", + "metadata": {}, + "source": [ + "在官网个人空间获取应用ID\n", + "![get app_id](../app_builder_resources/app_id.png)\n", + "\n", + "使用服务上传文件并对话示例图如下所示\n", + "![chainlit demo](./image/agent_runtime_with_chainlit_agent.png)\n", + "\n", + "使用工作流Agent应用对话示例如下图所示\n", + "![chainlit demo](./image/agent_runtime_with_chainlit_chatflow.png)" + ] + }, + { + "cell_type": "markdown", + "id": "2392f46d", + "metadata": {}, + "source": [ + "### 2. Session 数据管理\n", + "请参考：[user_session.ipynb](./user_session.ipynb)\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/cookbooks/components/flask_runtime.ipynb b/cookbooks/components/flask_runtime.ipynb new file mode 100644 index 000000000..b901ee4bf --- /dev/null +++ b/cookbooks/components/flask_runtime.ipynb @@ -0,0 +1,400 @@ +{ + "cells": [ + { + "attachments": {}, + "cell_type": "markdown", + "id": "f802e64d-4eaa-445d-a48a-1042a91bc394", + "metadata": { + "tags": [] + }, + "source": [ + "# 基于FlaskRuntime服务化组件\n", + "\n", + "## 目标\n", + "使用 FlaskRuntime 对组件进行服务化。\n", + "\n", + "FlaskRuntime 是对组件（Component）的服务化封装，具体有如下几个功能：\n", + "- 一键服务化组件: 使得组件能够以服务的形式运行，支持 API 调用。\n", + "- Session 数据管理: 提供 Session 数据的管理功能，允许跟踪和存储用户会话数据。\n", + "- 请求时鉴权: 支持在请求时进行认证，确保安全性。\n", + "\n", + "\n", + "## 准备工作\n", + "### 安装Python SDK\n", + "\n", + "appbuilder 支持使用 pip 安装（要求Python >= 3.8），并且 FlaskRuntime 服务化组件依赖 `appbuilder-sdk[serve]`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2939356f-61c2-42e9-9e0c-fc6729c193f6", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "pip install appbuilder-sdk 'appbuilder-sdk[serve]'" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "aeb2fa55-075f-48df-a9fb-8b40d9900684", + "metadata": {}, + "source": [ + "## 基本用法\n", + "\n", + "### 快速开始\n", + "\n", + "下面的示例会基于 Playground 组件，在 8091 端口部署 Web 服务: " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "41559341-fd7a-478c-a08b-1477d79e9d41", + "metadata": { + "ExecuteTime": { + "end_time": "2023-12-18T06:24:26.982459Z", + "start_time": "2023-12-18T06:23:53.771345Z" + } + }, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.flask_deploy import FlaskRuntime\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "component = appbuilder.Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\"\n", + ")\n", + "\n", + "agent = FlaskRuntime(component=component)\n", + "agent.serve(port=8091)" + ] + }, + { + "cell_type": "markdown", + "id": "b71e24eb", + "metadata": {}, + "source": [ + "通过 Shell 命令测试启动的服务, 请求 Body 为组件 run 方法的入参：\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "12c71fa5", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "curl --location 'http://0.0.0.0:8091/chat' \\\n", + "--header 'Content-Type: application/json' \\\n", + "--data '{\n", + " \"message\": \"海淀区的面积是多少\",\n", + " \"stream\": false\n", + "}'" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "5fc5bc38-6bc5-4187-a8fd-f802d77d89fa", + "metadata": {}, + "source": [ + "## FlaskRuntime 参数说明" + ] + }, + { + "cell_type": "markdown", + "id": "c2364c35", + "metadata": {}, + "source": [ + "### 1. 类初始化参数说明\n", + "\n", + "FlaskRuntime 初始化接受两个参数。\n", + "\n", + "| 参数名称 | 参数类型 | 是否必须 | 描述 | 示例值 |\n", + "|--|--|--|--|--|\n", + "| component | Component | 是 | 可运行的 Component, 该 Component 需要实现 run(message, stream, **args) 方法。 | Playground(prompt_template=\"{query}\", model=\"ERNIE-Bot\") |\n", + "| user_session_config | sqlalchemy.engine.URL\\|Str\\|None | 否 | 会话 Session 数据存储的数据库配置，遵循 sqlalchemy 后端定义，可参考[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)。默认使用 sqlite:///user_session.db，即本地的 SQLite 存储 | \"sqlite:///user_session.db\" |\n", + "| user_session| UserSession | 否 | 用户会话管理器，如果不指定则自动生成一个默认的 UserSession | UserSession(user_session_config) |" + ] + }, + { + "cell_type": "markdown", + "id": "62e1af06", + "metadata": {}, + "source": [ + "### 2. API 服务参数\n", + "\n", + "#### 2.1 请求参数\n", + "\n", + "**接口定义**\n", + "\n", + "| URL | Method |\n", + "|--|--|\n", + "| /chat | POST |\n", + "\n", + "**Header 参数**\n", + "\n", + "| 参数名称 | 是否必须 | 描述 | 示例值 |\n", + "|--|--|--|--|\n", + "| Content-Type | 是 | 必须设置为\"application/json\" | \"application/json\" |\n", + "| X-Appbuilder-Token | 否 | 开启请求时认证能力时需要带入 APPBUILDER_TOKEN 进行鉴权 | 前往千帆AppBuilder官网创建密钥，流程详见[文档](https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥) |\n", + "\n", + "**Body 参数**\n", + "\n", + "| 参数名称 | 参数类型 | 是否必须 | 描述 | 示例值 |\n", + "|--|--|--|--|--|\n", + "| message | Any | 是 | 透传到 component 的 run 方法的 message 参数 | \"海淀区的面积是多少\" |\n", + "| stream | Bool | 否 | 是否流式调用。透传到 component 的 run 方法的 stream 参数。默认为 false | false |\n", + "| session_id | Str | 否 | 用于标示同一个会话（Session）。如果不传该值，后端会自动生成 session_id，在响应参数中返回 | \"99680089-5acb-4298-9ade-a1a3f6c28102\" |\n", + "| 其他参数 | Any | 否 | 透传到 component 的 run 方法 | - |\n", + "\n", + "\n", + "#### 2.2 响应参数\n", + "分为非流式响应和流式响应。\n", + "\n", + "**非流式响应**\n", + "\n", + "| 参数名称 | 参数类型 | 描述 | 示例值 |\n", + "|--|--|--|--|\n", + "| code | Int | 错误码。值为0表示成功，否则为失败。非0错误详见错误码部分描述 | 0 |\n", + "| message | Str | 错误信息描述。 | \"Missing input variable query in message ['海淀区的面积是多少']\" |\n", + "| result | Object | 请求结果 | - |\n", + "| + answer_message | Object | 组件返回值，由返回的 Message 序列化得到 | {\"content\":\"海淀区是北京市的一个区，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，西与石景山区、门头沟区交界，北与昌平区接壤。海淀区的面积为**431平方千米**，约占北京市总面积的2.6%。\",\"extra\":{},\"id\":\"6b4e5019-a708-4bc5-a6ec-595fb4285677\",\"mtype\":\"dict\",\"name\":\"msg\"} |\n", + "| + session_id | Str | 用于标示同一个会话（Session） | \"99680089-5acb-4298-9ade-a1a3f6c28102\" |\n", + "\n", + "**流式响应**\n", + "\n", + "流式数据以追加的形式返回。流式和非流式的数据结构一致，不再描述。\n", + "\n", + "#### 2.3 响应示例\n", + "\n", + "分为非流式响应和流式响应。\n", + "\n", + "**非流式响应**\n", + "\n", + "```shell\n", + "{\n", + " \"code\": 0,\n", + " \"message\": \"\",\n", + " \"result\": {\n", + " \"answer_message\": {\n", + " \"content\": \"海淀区是北京市的一个区，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，西与石景山区、门头沟区交界，北与昌平区接壤。海淀区的面积为**431平方千米**，约占北京市总面积的2.6%。\",\n", + " \"extra\": {},\n", + " \"id\": \"6b4e5019-a708-4bc5-a6ec-595fb4285677\",\n", + " \"mtype\": \"dict\",\n", + " \"name\": \"msg\"\n", + " },\n", + " \"session_id\": \"99680089-5acb-4298-9ade-a1a3f6c28102\"\n", + " }\n", + "}\n", + "```\n", + "\n", + "**流式响应**\n", + "\n", + "```shell\n", + "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"海淀区\", \"extra\": {}}}}\n", + "\n", + "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"，隶属于北京市，位于北京市主城区西部和西北部，东与西城区、朝阳区相邻，南与丰台区毗连，\", \"extra\": {}}}}\n", + "\n", + "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"西与石景山区、门头沟区交界，北与昌平区接壤，总面积**431平方千米**。\", \"extra\": {}}}}\n", + "\n", + "data: {\"code\": 0, \"message\": \"\", \"result\": {\"session_id\": \"663303a9-d83d-481f-a084-872ece87989c\", \"answer_message\": {\"content\": \"\", \"extra\": {}}}}\n", + "```\n", + "\n", + "#### 2.4 错误码\n", + "| 错误码 | 描述 |\n", + "|--|--|\n", + "| 400 | 客户端请求参数错误 |\n", + "| 1000 | 服务端执行错误 |" + ] + }, + { + "cell_type": "markdown", + "id": "61923b00", + "metadata": {}, + "source": [ + "## 高级用法" + ] + }, + { + "cell_type": "markdown", + "id": "a2303c76", + "metadata": {}, + "source": [ + "### 1. 一键服务化组件 \n", + "FlaskRuntime 可以快速组件以服务的形式运行，支持 API 调用和对话框交互。\n", + "\n", + "**1.1 API调用**\n", + "\n", + "API 调用的基础用法在快速开始小结已经给出，这里不再赘述。\n", + "\n", + "下面介绍使用 `gunicorn` 启动生产级 Web 服务的方法，`gunicorn` 是一个适用于 UNIX 的 Python WSGI HTTP 服务器，详见[项目链接](https://github.com/benoitc/gunicorn)。\n", + "\n", + "首先创建 `app.py` 文件，暴露 Flask App：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fde5cc94", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import appbuilder\n", + "from appbuilder.utils.flask_deploy import FlaskRuntime\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "def get_flask_app():\n", + " component = appbuilder.Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\"\n", + " )\n", + " agent = FlaskRuntime(component=component)\n", + " return agent.create_flask_app()" + ] + }, + { + "cell_type": "markdown", + "id": "a41ef57b", + "metadata": {}, + "source": [ + "基于 `gunicorn` 启动生产级服务：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2077833f", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "# 服务工作进程数\n", + "SERVER_WORKER_AMOUNT=8\n", + "# 服务工作进程启动方式\n", + "SERVER_WORKER_CLASS=gevent\n", + "# 服务超时时间\n", + "GUNICORN_TIMEOUT=60\n", + "\n", + "gunicorn \\\n", + " --bind \"0.0.0.0:8091\" \\\n", + " --workers ${SERVER_WORKER_AMOUNT} \\\n", + " --worker-class ${SERVER_WORKER_CLASS} \\\n", + " --timeout ${GUNICORN_TIMEOUT} \\\n", + " \"app:get_flask_app()\"" + ] + }, + { + "cell_type": "markdown", + "id": "2392f46d", + "metadata": {}, + "source": [ + "### 2. Session 数据管理\n", + "请参考：[user_session.ipynb](./user_session.ipynb)" + ] + }, + { + "cell_type": "markdown", + "id": "1c352077", + "metadata": {}, + "source": [ + "### 3. 请求时鉴权\n", + "FlaskRuntime 支持在请求时进行认证，确保安全性。\n", + "\n", + "使用该能力，在初始化组件时需要设置 lazy 鉴权：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5d505a1c", + "metadata": {}, + "outputs": [], + "source": [ + "import appbuilder\n", + "from appbuilder.utils.flask_deploy import FlaskRuntime\n", + "\n", + "# 无需配置 APPBUILDER_TOKEN 环境变量\n", + "\n", + "component = appbuilder.Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\",\n", + " lazy_certification=True, # 设置 lazy 鉴权，在创建时不进行认证\n", + ")\n", + "\n", + "agent = FlaskRuntime(component=component)\n", + "agent.serve(port=8091)" + ] + }, + { + "cell_type": "markdown", + "id": "6a1add96", + "metadata": {}, + "source": [ + "当初始化组件时进行了 lazy 鉴权，请求时请求头必须带上 `X-Appbuilder-Token` （即Appbuilder密钥，获取流程详见[千帆AppBuilder官网创建密钥](https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥)）进行鉴权：" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "53c5a1d2", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "curl --location 'http://0.0.0.0:8091/chat' \\\n", + " --header 'Content-Type: application/json' \\\n", + " --header 'X-Appbuilder-Token: ...' \\\n", + " --data '{\n", + " \"message\": \"海淀区的面积是多少\",\n", + " \"stream\": false\n", + " }'" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/cookbooks/components/image/agent_runtime_with_chainlit_component_debug.jpg b/cookbooks/components/image/agent_runtime_with_chainlit_component_debug.jpg new file mode 100644 index 000000000..335e6f849 Binary files /dev/null and b/cookbooks/components/image/agent_runtime_with_chainlit_component_debug.jpg differ diff --git a/cookbooks/components/user_session.ipynb b/cookbooks/components/user_session.ipynb new file mode 100644 index 000000000..d1fd03e20 --- /dev/null +++ b/cookbooks/components/user_session.ipynb @@ -0,0 +1,362 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "2392f46d", + "metadata": {}, + "source": [ + "## Session 数据管理\n", + "FlaskRuntime和ChainlitRuntime 提供 Session 数据的管理功能，允许跟踪和存储用户会话数据。一般只有在二次开发的组件需要使用该能力。\n", + "\n", + "**1、 二次开发组件**\n", + "\n", + "二次开发的组件需要重写组件的 run(message, stream, **args)方法，并且至少需要有 message 和 stream 两个参数。\n", + "\n", + "下面基于 QueryRewrite 和 Playground 两个组件，开发 PlaygroundWithHistory 组件，该组件需要对会话数据进行操作。\n", + "\n", + "当使用 Component 独立运行时，会话数据会被存储于内存。\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "079048e3", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import logging\n", + "from appbuilder.core.component import Component\n", + "from appbuilder import (\n", + " UserSession, Message, QueryRewrite, Playground,\n", + ")\n", + "\n", + "# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥\n", + "os.environ[\"APPBUILDER_TOKEN\"] = '...'\n", + "\n", + "class PlaygroundWithHistory(Component):\n", + " def __init__(self):\n", + " super().__init__()\n", + " self.query_rewrite = QueryRewrite(model=\"Qianfan-Agent-Speed-8K\")\n", + " self.playground = Playground(\n", + " prompt_template=\"{query}\",\n", + " model=\"ERNIE-Bot\"\n", + " )\n", + "\n", + " def run(self, message: Message, stream: bool=False):\n", + " user_session = UserSession()\n", + " # 获取 Session 历史数据\n", + " history_queries = user_session.get_history(\"query\", limit=1)\n", + " history_answers = user_session.get_history(\"answer\", limit=1)\n", + "\n", + " # query 改写\n", + " if history_queries and history_answers:\n", + " history = []\n", + " for query, answer in zip(history_queries, history_answers):\n", + " history.extend([query.content, answer.content])\n", + " logging.info(f\"history: {history}\")\n", + " message = self.query_rewrite(\n", + " Message(history + [message.content]), rewrite_type=\"带机器人回复\")\n", + " logging.info(f\"message: {message}\") \n", + "\n", + " # 执行 playground\n", + " answer = self.playground.run(message, stream)\n", + "\n", + " # 保存本轮数据\n", + " user_session.append({\n", + " \"query\": message,\n", + " \"answer\": answer,\n", + " }) \n", + " return answer\n", + "\n", + "# component 可以独立运行，session数据会被保存于内存\n", + "playground_with_history_component = PlaygroundWithHistory()\n", + "print(playground_with_history_component.run(Message(\"海淀区的面积是多少\"), stream=False))" + ] + }, + { + "cell_type": "markdown", + "id": "480fd56d", + "metadata": {}, + "source": [ + "**2、 会话数据存储数据库**\n", + "\n", + "使用 FlaskRuntime和ChainlitRuntime 对 Component 服务化，会话数据会被存储于数据库。\n", + "下面的代码以 SQLite 为例展示该能力，更多数据库配置详见[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "111b11de", + "metadata": {}, + "outputs": [], + "source": [ + "import appbuilder\n", + "from appbuilder.utils.flask_deploy import FlaskRuntime\n", + "user_session_config = \"sqlite:///foo.db\"\n", + "agent = FlaskRuntime(\n", + " component=playground_with_history_component, \n", + " user_session_config=user_session_config)\n", + "agent.serve(port=8091)" + ] + }, + { + "cell_type": "markdown", + "id": "f8584416", + "metadata": {}, + "source": [ + "**3、 查看user_session.db储存信息**\n", + "\n", + "\n", + "使用该能力，查看用户对话信息。" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "id": "9bbeb156", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [], + "source": [ + "import sqlite3 \n", + " \n", + "# 连接到 SQLite 数据库 \n", + "# 如果文件不存在，会自动在当前目录创建: \n", + "user_session_path = '本地user_session.db地址' \n", + "conn = sqlite3.connect(user_session_path) \n", + "cursor = conn.cursor() " + ] + }, + { + "cell_type": "markdown", + "id": "54cce03b", + "metadata": {}, + "source": [ + "执行 SQL 语句，列出SQLite数据库中的所有表" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "id": "f4ca2963", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[('appbuilder_session_messages',)]\n" + ] + } + ], + "source": [ + "# 执行一条 SQL 语句，列出所有表 \n", + "cursor.execute(\"SELECT name FROM sqlite_master WHERE type='table';\") \n", + "print(cursor.fetchall()) " + ] + }, + { + "cell_type": "markdown", + "id": "e5b838f4", + "metadata": {}, + "source": [ + "查询appbuilder_session_messages表的列信息,appbuilder_session_messages表的列信息为:\n", + "- id\n", + "- session_id\n", + "- request_id\n", + "- message_key\n", + "- message_value\n", + "- created_at\n", + "- updated_at\n", + "- deleted" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "id": "1ac388d7", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "id\n", + "session_id\n", + "request_id\n", + "message_key\n", + "message_value\n", + "created_at\n", + "updated_at\n", + "deleted\n" + ] + } + ], + "source": [ + "cursor.execute(\"PRAGMA table_info(appbuilder_session_messages);\") \n", + "columns_info = cursor.fetchall() \n", + "\n", + "column_names = [info[1] for info in columns_info] # info[1]是列名的位置 \n", + "for column_name in column_names: \n", + " print(column_name) " + ] + }, + { + "cell_type": "markdown", + "id": "3965454f", + "metadata": {}, + "source": [ + "查询表中的特定数据【以message_value信息为例】\n", + "输出content、id、token_usage等信息" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "id": "2e6d9e45", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "{'content': '你好', 'name': 'msg', 'mtype': 'str', 'id': '90c8d150-c7b9-44a4-ac77-50dd61ee329a'}\n", + "{'content': '你好，我是百度研发的知识增强大语言模型，中文名是文心一言，英文名是ERNIE Bot。我能够与人对话互动，回答问题，协助创作，高效便捷地帮助人们获取信息、知识和灵感。', 'name': 'msg', 'mtype': 'dict', 'id': '0fc6c5f7-23bf-4f25-8555-69b4155908b7', 'extra': {}, 'token_usage': {'prompt_tokens': 2, 'completion_tokens': 42, 'total_tokens': 44}}\n", + "{'content': '请介绍一下你自己', 'name': 'msg', 'mtype': 'dict', 'id': '4f5cf6d1-976e-456e-a632-93c4b2550523', 'extra': {'search_db': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？\\\\n答案：\\\\n打篮球好玩吗？', 'dataset_id': '1f777fa3-26db-4237-98d5-075abc07a84f', 'dataset_name': '多轮改写Prompt', 'document_id': '92e0f30d-1f30-46f0-8377-59ab0fb6eb93', 'document_name': '多轮改写prompt_带机器人回复.txt', 'id': '61657591-f610-44f0-84ec-67cedcb66447', 'mock_id': '1', 'position': 0, 'score': 0.511053, 'sentences': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？', 'id': '1f777fa3-26db-4237-98d5-075abc07a84f-5992356ea8c0c05485388623218e134e-975ec17bbf8206762df7cfb24a5cd85e-97468bdaf0accef718118a2b742d5cc0', 'score': 0.628708, 'source': None}], 'title': '多轮改写prompt_带机器人回复.txt', 'tokens': 0, 'type': 'engine', 'word_count': 188}]}, 'token_usage': {'prompt_tokens': 321, 'completion_tokens': 3, 'total_tokens': 324}}\n", + "{'content': '您好，我是文心一言，英文名是ERNIE Bot。我能够与人对话互动，回答问题，协助创作，高效便捷地帮助人们获取信息、知识和灵感。', 'name': 'msg', 'mtype': 'dict', 'id': 'f769c850-f4e5-4d02-a62a-766fd8c8132e', 'extra': {}, 'token_usage': {'prompt_tokens': 4, 'completion_tokens': 33, 'total_tokens': 37}}\n", + "{'content': '介绍一下baidu公司', 'name': 'msg', 'mtype': 'dict', 'id': '0ff3b298-7f01-4036-927d-722e00afaf33', 'extra': {'search_db': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？\\\\n答案：\\\\n打篮球好玩吗？', 'dataset_id': '1f777fa3-26db-4237-98d5-075abc07a84f', 'dataset_name': '多轮改写Prompt', 'document_id': '92e0f30d-1f30-46f0-8377-59ab0fb6eb93', 'document_name': '多轮改写prompt_带机器人回复.txt', 'id': '61657591-f610-44f0-84ec-67cedcb66447', 'mock_id': '1', 'position': 0, 'score': 0.511053, 'sentences': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？', 'id': '1f777fa3-26db-4237-98d5-075abc07a84f-5992356ea8c0c05485388623218e134e-975ec17bbf8206762df7cfb24a5cd85e-97468bdaf0accef718118a2b742d5cc0', 'score': 0.628708, 'source': None}], 'title': '多轮改写prompt_带机器人回复.txt', 'tokens': 0, 'type': 'engine', 'word_count': 188}]}, 'token_usage': {'prompt_tokens': 910, 'completion_tokens': 3, 'total_tokens': 913}}\n", + "{'content': '百度公司是中国最大的互联网搜索引擎提供商之一，也是全球领先的AI公司。以下是对百度的详细介绍：\\n\\n一、公司概况\\n\\n百度（BIDU）是一家提供中文搜索引擎的公司，由李彦宏于2000年1月在北京中关村创立。经过多年的发展，百度已经从单一的搜索引擎服务商成功转型为内容生态与人工智能（AI）融合的互联网公司。百度的使命是“用科技让复杂的世界更简单”，其核心价值观包括“简单可依赖”。\\n\\n二、主要业务与产品\\n\\n1. 搜索业务：百度搜索是中国最大的中文搜索引擎，每天响应来自全球各地的搜索请求。除了传统的网页、图片、视频等搜索结果外，还提供了知识图谱、语音搜索等创新功能。\\n2. 移动生态业务：百度APP是百度移动生态的核心产品，围绕看、搜、听、问、购等核心场景为用户提供服务。此外，百度还拥有信息流、智能小程序、百度知道、百度文库等一系列移动生态产品。\\n3. 智能云业务：百度智能云是百度提供的云计算服务平台，为企业提供计算、存储、网络等基础设施服务以及人工智能、大数据等中间件服务。\\n4. 智能驾驶业务：百度在自动驾驶领域具有领先地位，其Apollo自动驾驶平台已经发展出多种商业模式，包括自动驾驶技术解决方案、百度造车以及共享无人车等。\\n\\n三、财务状况与业绩表现\\n\\n根据百度发布的财报显示，其业绩表现亮眼。例如，在2023年，百度总营收达到了显著的水平，其中百度核心收入同比增长。这主要得益于公司在广告市场的持续领先地位以及非在线营销收入的增长。同时，百度在研发投入方面也保持了较高水平，以推动其AI技术的持续创新和发展。\\n\\n四、AI技术与创新\\n\\n百度在AI技术方面取得了显著成果。其文心大模型已经升级到4.0版本，用户规模超过了1亿。这一技术使得百度具备了与全球领先的AI产品相媲美的能力。此外，百度还在文字生成视频等领域进行了战略投资，以进一步拓展其业务范围并加强在AI领域的领先地位。\\n\\n综上所述，百度公司作为中国最大的互联网搜索引擎提供商和全球领先的AI公司之一，凭借其强大的技术实力和创新能力，在搜索、移动生态、智能云和智能驾驶等领域取得了显著的成果。', 'name': 'msg', 'mtype': 'dict', 'id': 'c4f02188-1206-4c02-be8e-1c5481428003', 'extra': {}, 'token_usage': {'prompt_tokens': 4, 'completion_tokens': 456, 'total_tokens': 460}}\n" + ] + } + ], + "source": [ + "import json\n", + "cursor.execute(\"SELECT message_value FROM appbuilder_session_messages;\") \n", + "for row in cursor.fetchall(): \n", + " print(json.loads(row[0]))" + ] + }, + { + "cell_type": "markdown", + "id": "5ff82880", + "metadata": {}, + "source": [ + "以id查询相关的agent_runtime信息" + ] + }, + { + "cell_type": "code", + "execution_count": 87, + "id": "e79e2554", + "metadata": { + "vscode": { + "languageId": "shellscript" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "16010b88-d766-4524-81ee-37f96ceadb4d\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "0bdafb3d-f7e3-4187-bc1d-63cf51fbda29\n", + "query\n", + "{'content': '你好', 'name': 'msg', 'mtype': 'str', 'id': '90c8d150-c7b9-44a4-ac77-50dd61ee329a'}\n", + "2024-07-30 15:01:10.949475\n", + "2024-07-30 15:01:10.949485\n", + "0\n", + "4bd85fd0-2e5c-4de1-a47e-ea122f7e928c\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "0bdafb3d-f7e3-4187-bc1d-63cf51fbda29\n", + "answer\n", + "{'content': '你好，我是百度研发的知识增强大语言模型，中文名是文心一言，英文名是ERNIE Bot。我能够与人对话互动，回答问题，协助创作，高效便捷地帮助人们获取信息、知识和灵感。', 'name': 'msg', 'mtype': 'dict', 'id': '0fc6c5f7-23bf-4f25-8555-69b4155908b7', 'extra': {}, 'token_usage': {'prompt_tokens': 2, 'completion_tokens': 42, 'total_tokens': 44}}\n", + "2024-07-30 15:01:10.955875\n", + "2024-07-30 15:01:10.955884\n", + "0\n", + "d6e433fc-ef4d-4ade-9a63-fec600e95481\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "6300b76b-2307-4ed3-9f6d-e61dfc621ff8\n", + "query\n", + "{'content': '请介绍一下你自己', 'name': 'msg', 'mtype': 'dict', 'id': '4f5cf6d1-976e-456e-a632-93c4b2550523', 'extra': {'search_db': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？\\\\n答案：\\\\n打篮球好玩吗？', 'dataset_id': '1f777fa3-26db-4237-98d5-075abc07a84f', 'dataset_name': '多轮改写Prompt', 'document_id': '92e0f30d-1f30-46f0-8377-59ab0fb6eb93', 'document_name': '多轮改写prompt_带机器人回复.txt', 'id': '61657591-f610-44f0-84ec-67cedcb66447', 'mock_id': '1', 'position': 0, 'score': 0.511053, 'sentences': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？', 'id': '1f777fa3-26db-4237-98d5-075abc07a84f-5992356ea8c0c05485388623218e134e-975ec17bbf8206762df7cfb24a5cd85e-97468bdaf0accef718118a2b742d5cc0', 'score': 0.628708, 'source': None}], 'title': '多轮改写prompt_带机器人回复.txt', 'tokens': 0, 'type': 'engine', 'word_count': 188}]}, 'token_usage': {'prompt_tokens': 321, 'completion_tokens': 3, 'total_tokens': 324}}\n", + "2024-07-30 15:01:19.161269\n", + "2024-07-30 15:01:19.161272\n", + "0\n", + "f0d76f00-93be-4d7c-a6b3-a9583fd677ad\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "6300b76b-2307-4ed3-9f6d-e61dfc621ff8\n", + "answer\n", + "{'content': '您好，我是文心一言，英文名是ERNIE Bot。我能够与人对话互动，回答问题，协助创作，高效便捷地帮助人们获取信息、知识和灵感。', 'name': 'msg', 'mtype': 'dict', 'id': 'f769c850-f4e5-4d02-a62a-766fd8c8132e', 'extra': {}, 'token_usage': {'prompt_tokens': 4, 'completion_tokens': 33, 'total_tokens': 37}}\n", + "2024-07-30 15:01:19.163405\n", + "2024-07-30 15:01:19.163411\n", + "0\n", + "936c394a-a3fa-48fd-a72f-b1778ebc35e4\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "5eb09e91-6496-44d9-9303-a4790e97f61c\n", + "query\n", + "{'content': '介绍一下baidu公司', 'name': 'msg', 'mtype': 'dict', 'id': '0ff3b298-7f01-4036-927d-722e00afaf33', 'extra': {'search_db': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？\\\\n答案：\\\\n打篮球好玩吗？', 'dataset_id': '1f777fa3-26db-4237-98d5-075abc07a84f', 'dataset_name': '多轮改写Prompt', 'document_id': '92e0f30d-1f30-46f0-8377-59ab0fb6eb93', 'document_name': '多轮改写prompt_带机器人回复.txt', 'id': '61657591-f610-44f0-84ec-67cedcb66447', 'mock_id': '1', 'position': 0, 'score': 0.511053, 'sentences': [{'content': '带机器人回复：请你扮演一个智能搜索改写补全机器人，请根据User的搜索历史以及对应的搜索结果，对最后一句话先进行主语继承改写，然后进行上下文信息补全，注意：不要改变原文的意思，答案要尽可能简洁，不要直接回答该问题，不要输出多于的内容。\\\\n\\\\n例子：\\\\n搜索历史：\\\\nUser：今天上午你干嘛了\\\\nAssistant：去打篮球啦\\\\nUser：好玩吗？', 'id': '1f777fa3-26db-4237-98d5-075abc07a84f-5992356ea8c0c05485388623218e134e-975ec17bbf8206762df7cfb24a5cd85e-97468bdaf0accef718118a2b742d5cc0', 'score': 0.628708, 'source': None}], 'title': '多轮改写prompt_带机器人回复.txt', 'tokens': 0, 'type': 'engine', 'word_count': 188}]}, 'token_usage': {'prompt_tokens': 910, 'completion_tokens': 3, 'total_tokens': 913}}\n", + "2024-07-30 15:02:21.551493\n", + "2024-07-30 15:02:21.551529\n", + "0\n", + "5cd9d7ef-e26f-48ca-bd55-de2861cd4e84\n", + "b2c9d058-4475-4258-ad90-4334f3d024d5\n", + "5eb09e91-6496-44d9-9303-a4790e97f61c\n", + "answer\n", + "{'content': '百度公司是中国最大的互联网搜索引擎提供商之一，也是全球领先的AI公司。以下是对百度的详细介绍：\\n\\n一、公司概况\\n\\n百度（BIDU）是一家提供中文搜索引擎的公司，由李彦宏于2000年1月在北京中关村创立。经过多年的发展，百度已经从单一的搜索引擎服务商成功转型为内容生态与人工智能（AI）融合的互联网公司。百度的使命是“用科技让复杂的世界更简单”，其核心价值观包括“简单可依赖”。\\n\\n二、主要业务与产品\\n\\n1. 搜索业务：百度搜索是中国最大的中文搜索引擎，每天响应来自全球各地的搜索请求。除了传统的网页、图片、视频等搜索结果外，还提供了知识图谱、语音搜索等创新功能。\\n2. 移动生态业务：百度APP是百度移动生态的核心产品，围绕看、搜、听、问、购等核心场景为用户提供服务。此外，百度还拥有信息流、智能小程序、百度知道、百度文库等一系列移动生态产品。\\n3. 智能云业务：百度智能云是百度提供的云计算服务平台，为企业提供计算、存储、网络等基础设施服务以及人工智能、大数据等中间件服务。\\n4. 智能驾驶业务：百度在自动驾驶领域具有领先地位，其Apollo自动驾驶平台已经发展出多种商业模式，包括自动驾驶技术解决方案、百度造车以及共享无人车等。\\n\\n三、财务状况与业绩表现\\n\\n根据百度发布的财报显示，其业绩表现亮眼。例如，在2023年，百度总营收达到了显著的水平，其中百度核心收入同比增长。这主要得益于公司在广告市场的持续领先地位以及非在线营销收入的增长。同时，百度在研发投入方面也保持了较高水平，以推动其AI技术的持续创新和发展。\\n\\n四、AI技术与创新\\n\\n百度在AI技术方面取得了显著成果。其文心大模型已经升级到4.0版本，用户规模超过了1亿。这一技术使得百度具备了与全球领先的AI产品相媲美的能力。此外，百度还在文字生成视频等领域进行了战略投资，以进一步拓展其业务范围并加强在AI领域的领先地位。\\n\\n综上所述，百度公司作为中国最大的互联网搜索引擎提供商和全球领先的AI公司之一，凭借其强大的技术实力和创新能力，在搜索、移动生态、智能云和智能驾驶等领域取得了显著的成果。', 'name': 'msg', 'mtype': 'dict', 'id': 'c4f02188-1206-4c02-be8e-1c5481428003', 'extra': {}, 'token_usage': {'prompt_tokens': 4, 'completion_tokens': 456, 'total_tokens': 460}}\n", + "2024-07-30 15:02:21.557667\n", + "2024-07-30 15:02:21.557671\n", + "0\n" + ] + } + ], + "source": [ + "cursor.execute(\"SELECT * FROM appbuilder_session_messages WHERE session_id = 'b2c9d058-4475-4258-ad90-4334f3d024d5';\") \n", + "for tuple in cursor.fetchall():\n", + " for message in tuple:\n", + " try: \n", + " message = json.loads(message)\n", + " print(message)\n", + " except:\n", + " print(message)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/docs/BasisModule/Deployment/AgentChainlit.md b/docs/BasisModule/Deployment/AgentChainlit.md deleted file mode 100644 index b42173fc8..000000000 --- a/docs/BasisModule/Deployment/AgentChainlit.md +++ /dev/null @@ -1,5 +0,0 @@ -# 交互式前端部署 - -## 基础组件基于Chainlit提供交互式前端页面 - -请参考 [组件服务化部署](https://github.com/baidubce/app-builder/blob/master/cookbooks/components/agent_runtime.ipynb) \ No newline at end of file diff --git a/docs/BasisModule/Deployment/agentruntime.md b/docs/BasisModule/Deployment/chainlit_deploy.md similarity index 65% rename from docs/BasisModule/Deployment/agentruntime.md rename to docs/BasisModule/Deployment/chainlit_deploy.md index decf02965..9c2286ce9 100644 --- a/docs/BasisModule/Deployment/agentruntime.md +++ b/docs/BasisModule/Deployment/chainlit_deploy.md @@ -1,13 +1,13 @@ -# `AgentRuntime`类 +# `ChainlitRuntime`类 ## 简介 -AgentRuntime 是对组件调用的服务化封装，开发者不是必须要用 AgentRuntime 才能运行自己的组件服务。但 AgentRuntime 可以快速帮助开发者服务化组件服务，并且提供API、对话框等部署方式。此外，结合 Component 和 Message 自带的运行和调试接口，可以方便开发者快速获得一个调试 Agent 的服务。 +ChainlitRuntime 是对组件和应用调用的服务化封装，开发者不是必须要用 ChainlitRuntime 才能运行自己的组件服务。但 ChainlitRuntime 可以快速帮助开发者服务化组件服务，并且提供API、对话框等部署方式。此外，结合 Component 和 Message 自带的运行和调试接口，可以方便开发者快速获得一个调试 Agent 的服务。 ## Python基本用法 -### 1、实例化`AgentRuntime() -> AgentRuntime` +### 1、实例化`ChainlitRuntime() -> ChainlitRuntime` #### 方法参数 @@ -15,6 +15,8 @@ AgentRuntime 是对组件调用的服务化封装，开发者不是必须要用 |--------|--------|------------|-----------| | component | Component | 可运行的 Component,需要实现 run(message, stream, **args) 方法 | "正确的component组件或client" | | user_session_config | sqlalchemy.engine.URL、str、None | Session 输出存储配置字符串。默认使用 sqlite:///user_session.db | "正确的存储配置字符串" | +| user_session| UserSession | 否 | 用户会话管理器，如果不指定则自动生成一个默认的 UserSession | UserSession(user_session_config) | +| tool_choice| ToolChoice| 否 | 可用于Agent强制执行的组件工具 | | #### 方法功能 @@ -25,15 +27,17 @@ AgentRuntime 是对组件调用的服务化封装，开发者不是必须要用 ```python import os import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime + os.environ["APPBUILDER_TOKEN"] = '...' component = appbuilder.Playground( prompt_template="{query}", model="eb-4" ) -agent = appbuilder.AgentRuntime(component=component) +agent = ChainlitRuntime(component=component) ``` -### 2、运行Agent服务`AgentRuntime.chat(message: Message, stream: bool = False, **args) -> Message` +### 2、运行Agent服务`ChainlitRuntime.chat(message: Message, stream: bool = False, **args) -> Message` #### 方法参数 @@ -51,49 +55,19 @@ agent = appbuilder.AgentRuntime(component=component) ```python import os import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime + os.environ["APPBUILDER_TOKEN"] = '...' component = appbuilder.Playground( prompt_template="{query}", model="eb-4" ) -agent = appbuilder.AgentRuntime(component=component) +agent = ChainlitRuntime(component=component) message = appbuilder.Message({"query": "你好"}) print(agent.chat(message, stream=False)) ``` -### 3、提供 Flask http API 接口`AgentRuntime.serve(self, host='0.0.0.0', debug=True, port=8092, url_rule="/chat"` - -#### 方法参数 - -| 参数名称 | 参数类型 | 描述 | 示例值 | -|--------|--------|------------|-----------| -| host | String | 服务主机地址，默认为 '0.0.0.0' | '0.0.0.0' | -| debug | bool | 是否是调试模式，默认为 True | False | -| port | int | 服务端口号，默认为 8092 | 8092 | -| url_rule | String | Flask 路由规则，默认为 '/chat' | '/chat' | - -#### 方法功能 - -将 component 服务化，提供 Flask http API 接口 - -#### 示例代码 - -```python -import os -import appbuilder -os.environ["APPBUILDER_TOKEN"] = '...' -component = appbuilder.Playground( - prompt_template="{query}", - model="eb-4" -) -user_session_config = "sqlite:///foo.db" -agent = appbuilder.AgentRuntime( - component=component, user_session_config=user_session_config) -agent.serve(debug=False, port=8091) -``` - - -### 4、提供 chainlit demo 页面`AgentRuntime.chainlit_demo(host='0.0.0.0', port=8091)` +### 3、提供 chainlit component 页面`ChainlitRuntime.chainlit_component(host='0.0.0.0', port=8091)` #### 方法参数 @@ -114,8 +88,10 @@ import os import logging from appbuilder.core.component import Component from appbuilder import ( - AgentRuntime, UserSession, Message, QueryRewrite, Playground, + UserSession, Message, QueryRewrite, Playground, ) +from appbuilder.utils.chainlit_deploy import ChainlitRuntime + os.environ["APPBUILDER_TOKEN"] = 'YOUR_APPBUILDER_TOKEN' class PlaygroundWithHistory(Component): def __init__(self): @@ -146,11 +122,11 @@ class PlaygroundWithHistory(Component): }) return answer -agent = AgentRuntime(component=PlaygroundWithHistory()) -agent.chainlit_demo(port=8091) +agent = ChainlitRuntime(component=PlaygroundWithHistory()) +agent.chainlit_component(port=8091) ``` -### 5、将 appbuilder client 服务化，提供 chainlit demo 页面`AgentRuntime.chainlit_agent(host='0.0.0.0', port=8091)` +### 5、将 appbuilder client 服务化，提供 chainlit demo 页面`ChainlitRuntime.chainlit_agent(host='0.0.0.0', port=8091)` 目前支持工作流Agent、自主规划Agent应用。 #### 方法参数 @@ -168,11 +144,12 @@ agent.chainlit_demo(port=8091) ```python import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime import os os.environ["APPBUILDER_TOKEN"] = '...' app_id = '...' # 已发布AppBuilder应用ID，可在console端查看 client = appbuilder.AppBuilderClient(app_id) -agent = appbuilder.AgentRuntime(component=client) +agent = ChainlitRuntime(component=client) agent.chainlit_agent(port=8091) ``` \ No newline at end of file diff --git a/docs/BasisModule/Deployment/cloud.md b/docs/BasisModule/Deployment/cloud.md index 40980f3ce..e16499cfa 100644 --- a/docs/BasisModule/Deployment/cloud.md +++ b/docs/BasisModule/Deployment/cloud.md @@ -55,9 +55,10 @@ touch sample/component_sample.py ```python # ./smaple目录下component_sample.py import appbuilder +from appbuilder.utils.flask_deploy import FlaskRuntime component = appbuilder.Playground(prompt_template="{query}", model="ERNIE-Bot") -agent = appbuilder.AgentRuntime(component=component) +agent = FlaskRuntime(component=component) agent.serve(port=8091) ``` diff --git a/docs/BasisModule/Deployment/flask.md b/docs/BasisModule/Deployment/flask.md deleted file mode 100644 index ef0e3c9ba..000000000 --- a/docs/BasisModule/Deployment/flask.md +++ /dev/null @@ -1,5 +0,0 @@ -# API 访问 - -## 基础组件基于Flask提供 API访问服务 - -请参考 [组件服务化部署](https://github.com/baidubce/app-builder/blob/master/cookbooks/components/agent_runtime.ipynb) \ No newline at end of file diff --git a/docs/BasisModule/Deployment/flask_deploy.md b/docs/BasisModule/Deployment/flask_deploy.md new file mode 100644 index 000000000..2b5f845ae --- /dev/null +++ b/docs/BasisModule/Deployment/flask_deploy.md @@ -0,0 +1,101 @@ +# `FlaskRuntime`类 + +## 简介 + +FlaskRuntime 是对组件调用的服务化封装，开发者不是必须要用 FlaskRuntime 才能运行自己的组件服务。但 FlaskRuntime 可以快速帮助开发者服务化组件服务，并且提供API、对话框等部署方式。此外，结合 Component 和 Message 自带的运行和调试接口，可以方便开发者快速获得一个调试 Agent 的服务。 + + +## Python基本用法 + +### 1、实例化`FlaskRuntime() -> FlaskRuntime` + +#### 方法参数 + +| 参数名称 | 参数类型 | 描述 | 示例值 | +|--------|--------|------------|-----------| +| component | Component | 可运行的 Component,需要实现 run(message, stream, **args) 方法 | "正确的component组件或client" | +| user_session_config | sqlalchemy.engine.URL、str、None | Session 输出存储配置字符串。默认使用 sqlite:///user_session.db | "正确的存储配置字符串" | +| user_session_config | sqlalchemy.engine.URL\|Str\|None | 否 | 会话 Session 数据存储的数据库配置，遵循 sqlalchemy 后端定义，可参考[文档](https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls)。默认使用 sqlite:///user_session.db，即本地的 SQLite 存储 | "sqlite:///user_session.db" | +｜user_session| UserSession | 否 | 用户会话管理器，如果不指定则自动生成一个默认的 UserSession | UserSession(user_session_config) | + +#### 方法功能 + +返回一个调试 Agent 的服务 + +#### 示例代码 + +```python +import os +import appbuilder +from appbuilder.utils.flask_deploy import FlaskRuntime + +os.environ["APPBUILDER_TOKEN"] = '...' +component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" +) +agent = FlaskRuntime(component=component) +``` + +### 2、运行Agent服务`FlaskRuntime.chat(message: Message, stream: bool = False, **args) -> Message` + +#### 方法参数 + +| 参数名称 | 参数类型 | 描述 | 示例值 | +|--------|--------|------------|-----------| +| message | Message | 该次对话用户输入的 Message | "正确的Message" | +| stream | bool | 是否使用流式请求。默认为 False | False | + +#### 方法功能 + +运行一个 Agent 服务，执行一次对话 + +#### 示例代码 + +```python +import os +import appbuilder +from appbuilder.utils.flask_deploy import FlaskRuntime + +os.environ["APPBUILDER_TOKEN"] = '...' +component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" +) +agent = FlaskRuntime(component=component) +message = appbuilder.Message({"query": "你好"}) +print(agent.chat(message, stream=False)) +``` + +### 3、提供 Flask http API 接口`FlaskRuntime.serve(self, host='0.0.0.0', debug=True, port=8092, url_rule="/chat"` + +#### 方法参数 + +| 参数名称 | 参数类型 | 描述 | 示例值 | +|--------|--------|------------|-----------| +| host | String | 服务主机地址，默认为 '0.0.0.0' | '0.0.0.0' | +| debug | bool | 是否是调试模式，默认为 True | False | +| port | int | 服务端口号，默认为 8092 | 8092 | +| url_rule | String | Flask 路由规则，默认为 '/chat' | '/chat' | + +#### 方法功能 + +将 component 服务化，提供 Flask http API 接口 + +#### 示例代码 + +```python +import os +import appbuilder +from appbuilder.utils.flask_deploy import FlaskRuntime + +os.environ["APPBUILDER_TOKEN"] = '...' +component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" +) +user_session_config = "sqlite:///foo.db" +agent = FlaskRuntime( + component=component, user_session_config=user_session_config) +agent.serve(debug=False, port=8091) +``` \ No newline at end of file diff --git a/docs/BasisModule/Deployment/usersession.md b/docs/BasisModule/Deployment/usersession.md index e0af798d1..c22035087 100644 --- a/docs/BasisModule/Deployment/usersession.md +++ b/docs/BasisModule/Deployment/usersession.md @@ -6,7 +6,7 @@ ## 应用场景 -**必须**在 AgentRuntime 启动的服务中使用。 +**必须**在 FlaskRuntime和ChainlitRuntime 启动的服务中使用。 ## Python基本用法 @@ -82,9 +82,9 @@ user_session.append({ }) ``` -### 4、UserSession结合AgentRuntime使用以及user_session.db文件读取 +### 4、UserSession结合FlaskRuntime/ChainlitRuntime使用以及user_session.db文件读取 -- [UserSession结合AgentRuntime使用以及user_session.db文件读取](https://github.com/baidubce/app-builder/blob/master/cookbooks/components/agent_runtime.ipynb) +- [UserSession结合FlaskRuntime/ChainlitRuntime使用以及user_session.db文件读取](https://github.com/baidubce/app-builder/blob/master/cookbooks/components/user_session.ipynb) diff --git a/docs/DevelopGuide/HowToContributeCode/README.md b/docs/DevelopGuide/HowToContributeCode/README.md index 893046647..2b76f2e0a 100644 --- a/docs/DevelopGuide/HowToContributeCode/README.md +++ b/docs/DevelopGuide/HowToContributeCode/README.md @@ -289,7 +289,7 @@ class BaiduSearchWithModel(Component): #### Test文件目录 * test文件需要为『test_』开头 -* 测试类需要形如『class TestAgentRuntime(unittest.TestCase)::』的定义方式 +* 测试类需要形如『class TestFlaskRuntime(unittest.TestCase)::』的定义方式 * test文件需要置于appbuilder-sdk-ext/appbuilder_sdk_ext/tests路径下 #### UnitTest提供三种标签实现两种运行模式 diff --git a/docs/QuickStart/ExamplesOfIndustrialPracticeApplications/README.md b/docs/QuickStart/ExamplesOfIndustrialPracticeApplications/README.md index 6174398af..604f84930 100644 --- a/docs/QuickStart/ExamplesOfIndustrialPracticeApplications/README.md +++ b/docs/QuickStart/ExamplesOfIndustrialPracticeApplications/README.md @@ -110,6 +110,7 @@ pip install chainlit ```python import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime # 空模版组件 playground = appbuilder.Playground( @@ -117,9 +118,9 @@ playground = appbuilder.Playground( model="Qianfan-Agent-Speed-8K" ) -# 使用 AgentRuntime 来服务化playground组件 -agent = appbuilder.AgentRuntime(component=playground) +# 使用 ChainlitRuntime 来服务化playground组件 +agent = ChainlitRuntime(component=playground) # 启动chainlit demo，会自动在浏览器打开体验对话框页面 -agent.chainlit_demo(port=8091) +agent.chainlit_component(port=8091) ``` \ No newline at end of file diff --git a/docs/QuickStart/StartFirstAINativeApplication/README.md b/docs/QuickStart/StartFirstAINativeApplication/README.md index 7d75e55b5..6b6e0a955 100644 --- a/docs/QuickStart/StartFirstAINativeApplication/README.md +++ b/docs/QuickStart/StartFirstAINativeApplication/README.md @@ -111,9 +111,11 @@ AppBuilder-SDK提供对组件的服务化能力。通过定义Agent，开发者 pip install chainlit ``` 而后，使用AppBuilder的Agent服务化功能，即可快速部署服务 +对于实现了run方法，返回str类的组件，可使用以下方法进行部署： ```python import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime # 空模版组件 playground = appbuilder.Playground( @@ -121,22 +123,43 @@ playground = appbuilder.Playground( model="Qianfan-Agent-Speed-8K" ) -# 使用 AgentRuntime 来服务化playground组件 -agent = appbuilder.AgentRuntime(component=playground) +# 使用 ChainlitRuntime 来服务化playground组件 +agent = ChainlitRuntime(component=playground) # 启动chainlit demo，会自动在浏览器打开体验对话框页面 -agent.chainlit_demo(port=8091) +agent.chainlit_component(port=8091) ``` +对可能输出其他类型的组件，如图片、音频、代码等组件，可使用以下接口查看各种输出格式的渲染效果。 +注意：该接口目前只支持新协议组件，即appbuilder.core.component.v2实现下的组件。 + +```python +import os +import appbuilder +from appbuilder.utils.chainlit_deploy import ChainlitRuntime +from appbuilder.core.components.v2 import SimilarQuestion + +# 使用组件之前，请前往千帆AppBuilder官网创建密钥，流程详见：https://cloud.baidu.com/doc/AppBuilder/s/Olq6grrt6#1、创建密钥 +os.environ["APPBUILDER_TOKEN"] = '...' + +component = SimilarQuestion() +agent = ChainlitRuntime(component=component) +agent.chainlit_component_debug( + port=8092, + tool_eval_args={}, + query_name = "query" +) +``` + + + 也可以对AppBuilderClient进行服务化，快速部署 ```python import os from appbuilder.core.component import Component -from appbuilder import ( - AgentRuntime, - AppBuilderClient, -) +from appbuilder.utils.chainlit_deploy import ChainlitRuntime +from appbuilder import AppBuilderClient if __name__ == "__main__": @@ -148,6 +171,6 @@ if __name__ == "__main__": # 网页已部署的应用链接为「地理小达人」：https://appbuilder.baidu.com/s/x1tSF app_id = "42eb211a-14b9-43d2-9fae-193c8760ef26" agent_builder = AppBuilderClient(app_id) - agent = AgentRuntime(component=agent_builder) + agent = ChainlitRuntime(component=agent_builder) agent.chainlit_agent(port=8091) ``` \ No newline at end of file diff --git a/docs/README.md b/docs/README.md index 2168c1a04..27e5103ac 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,10 +17,9 @@ - [TRACE基础功能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/basic.md) - [TRACE拓展功能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/phoenix_method.md) - 部署: - - [交互式前端部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/AgentChainlit.md) + - [交互式前端部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/chainlit_deploy.md) - [公有云部署](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/cloud.md) - - [API 访问](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask.md) - - [AgentRuntime](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/agentruntime.md) + - [API 访问](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask_deploy.md) - [UserSession](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/usersession.md) - 平台: - 应用: diff --git a/docs/README_en.md b/docs/README_en.md index eead7d289..b99b5a5bf 100644 --- a/docs/README_en.md +++ b/docs/README_en.md @@ -33,15 +33,15 @@ Baidu AI Cloud Qianfan AppBuilder-SDK offers the following essential features fo - Orchestrate and manage knowledge flows, managing knowledge bases through KnowledgeBase, performing create, read, update, and delete (CRUD) operations on documents and knowledge slices, and developing industry-grade RAG applications with Baidu AI Cloud Qianfan AppBuilder Platform. - - Orchestrate and automate workflows, providing multi-level workflow abstractions such as `Message`, `Component`, and `AgentRuntime` to achieve workflow orchestration and integration with industry ecosystem capabilities like LangChain and OpenAI. + - Orchestrate and automate workflows, providing multi-level workflow abstractions such as `Message`, `Component`, `FlaskRuntime`, and `ChainlitRuntime` to achieve workflow orchestration and integration with industry ecosystem capabilities like LangChain and OpenAI. 3. Monitoring - Providing monitoring tools such as visual Tracing and detailed DebugLog to assist developers in production environments. 4. Deployment - - AgentRuntime supports deployment as an API service based on Flask and Gunicorn. - - AgentRuntime supports deployment as a Chainlit-based dialog interaction front-end application. + - FlaskRuntime supports deployment as an API service based on Flask and Gunicorn. + - ChainlitRuntime supports deployment as a Chainlit-based dialog interaction front-end application. - Offering the appbuilder_bce_deploy tool for rapid deployment of programs to Baidu Cloud, providing public network API services, and enabling integration with AppBuilder's workflow functionalities. @@ -263,10 +263,9 @@ Hook: - [TRACE extended functions](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/phoenix_method.md) - [Debug functionality](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/Debug.md) - Deployment: - - [Interactive front-end deployment](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/AgentChainlit.md) + - [Interactive front-end deployment](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/chainlit_deploy.md) - [Public cloud deployment](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/cloud.md) - - [API access](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask.md) - - [AgentRuntime](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/agentruntime.md) + - [API access](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask_deploy.md) - [UserSession](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/usersession.md) - Platform: - Applications: diff --git a/docs/README_ja.md b/docs/README_ja.md index 5f00dddd8..760d2c9ec 100644 --- a/docs/README_ja.md +++ b/docs/README_ja.md @@ -30,15 +30,15 @@ Baidu AI Cloud Qianfan AppBuilder-SDKは、AIアプリケーション開発者 - 知識フローをオーケストレーションおよび管理し、KnowledgeBaseを通じて知識ベースを管理し、文書および知識スライスの作成、読み取り、更新、削除（CRUD）操作を実行し、Baidu AI Cloud Qianfan AppBuilderプラットフォームと共に業界グレードのRAGアプリケーションを開発します。 - - ワークフローをオーケストレーションおよび自動化し、`Message`、`Component`、`AgentRuntime`などの多層ワークフロー抽象を提供し、ワークフローのオーケストレーションを実現し、LangChainやOpenAIなどの業界エコシステム機能と統合します。 + - ワークフローをオーケストレーションおよび自動化し、`Message`、`Component`、`FlaskRuntime`、`ChainlitRuntime`などの多層ワークフロー抽象を提供し、ワークフローのオーケストレーションを実現し、LangChainやOpenAIなどの業界エコシステム機能と統合します。 3. 監視 - 開発者が生産環境で使用するための可視化トレースおよび詳細なデバッグログなどの監視ツールを提供します。 4. デプロイメント - - AgentRuntimeは、FlaskおよびGunicornに基づくAPIサービスとしてデプロイメントをサポートします。 - - AgentRuntimeは、Chainlitに基づく対話型フロントエンドアプリケーションとしてデプロイメントをサポートします。 + - FlaskRuntimeは、FlaskおよびGunicornに基づくAPIサービスとしてデプロイメントをサポートします。 + - ChainlitRuntimeは、Chainlitに基づく対話型フロントエンドアプリケーションとしてデプロイメントをサポートします。 - プログラムをBaidu Cloudに迅速にデプロイし、パブリックネットワークAPIサービスを提供し、AppBuilderのワークフロー機能と統合するためのappbuilder_bce_deployツールを提供します。 @@ -259,10 +259,9 @@ print(answer.content.answer) - [TRACE拡張機能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/phoenix_method.md) - [Debug機能](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Trace/Debug.md) - デプロイ: - - [インタラクティブなフロントエンドデプロイ](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/AgentChainlit.md) + - [インタラクティブなフロントエンドデプロイ](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/chainlit_deploy.md) - [パブリッククラウドデプロイ](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/cloud.md) - - [API アクセス](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask.md) - - [AgentRuntime](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/agentruntime.md) + - [API アクセス](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/flask_deploy.md) - [UserSession](https://github.com/baidubce/app-builder/blob/master/docs/BasisModule/Deployment/usersession.md) - プラットフォーム: - アプリケーション: diff --git a/mkdocs.yml b/mkdocs.yml index 7311aa4c1..a67b7cb85 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -68,10 +68,9 @@ nav: - TRACE拓展功能: BasisModule/Trace/phoenix_method.md - Debug功能: BasisModule/Trace/Debug.md - 部署: - - 交互式前端部署: BasisModule/Deployment/AgentChainlit.md + - 交互式前端部署: BasisModule/Deployment/chainlit_runtime.md - 公有云部署: BasisModule/Deployment/cloud.md - - API 访问: BasisModule/Deployment/flask.md - - AgentRuntime: BasisModule/Deployment/agentruntime.md + - API 访问: BasisModule/Deployment/flask_deploy.md - UserSession: BasisModule/Deployment/usersession.md - 平台: - 应用: diff --git a/python/__init__.py b/python/__init__.py index c3bc9f92a..00af065c3 100644 --- a/python/__init__.py +++ b/python/__init__.py @@ -171,6 +171,8 @@ def get_default_header(): from appbuilder.core.user_session import UserSession from appbuilder.utils.logger_util import logger +from appbuilder.utils.flask_deploy import FlaskRuntime +from appbuilder.utils.chainlit_deploy import ChainlitRuntime from appbuilder.core.manifest.manifest_decorator import manifest, manifest_parameter from appbuilder.core.manifest.models import Manifest diff --git a/python/core/agent.py b/python/core/agent.py index fbc0ad412..dcd46fa5c 100644 --- a/python/core/agent.py +++ b/python/core/agent.py @@ -19,6 +19,7 @@ import json import shutil import inspect +from deprecated import deprecated from pydantic import BaseModel, model_validator, Extra from typing import Optional, Dict, Any, Union import appbuilder @@ -26,12 +27,14 @@ from appbuilder.core.component import Component from appbuilder.core.message import Message from appbuilder.utils.logger_util import logger +from appbuilder.utils.flask_deploy import FlaskRuntime +from appbuilder.utils.chainlit_deploy import ChainlitRuntime from appbuilder.core.console.appbuilder_client.data_class import ToolChoiceFunction, ToolChoice, Action # 流式场景首包超时时，最大重试次数 MAX_RETRY_COUNT = 3 - +@deprecated(reason="deplecated. Use FlaskRuntime or ChainlitRuntime instead") class AgentRuntime(BaseModel): r""" AgentRuntime 是对组件调用的服务化封装，开发者不是必须要用 AgentRuntime 才能运行自己的组件服务。 @@ -202,6 +205,8 @@ def run(self, message: Message, stream: bool=False): user_session_config: Optional[Union[Any, str]] = None user_session: Optional[Any] = None tool_choice: ToolChoice = None + flask_deploy: FlaskRuntime = None + chainlit_deploy: ChainlitRuntime = None class Config: """ @@ -231,8 +236,24 @@ def init(cls, values: Dict) -> Dict: values.update({ "user_session": UserSession(values.get("user_session_config")) }) + flask_deploy = appbuilder.FlaskRuntime( + component=values.get("component"), + user_session_config=values.get("user_session_config"), + user_session=values.get("user_session") + ) + chainlit_deploy = appbuilder.ChainlitRuntime( + component=values.get("component"), + user_session_config=values.get("user_session_config"), + user_session=values.get("user_session"), + tool_choice=values.get("tool_choice") + ) + values.update({ + "flask_deploy": flask_deploy, + "chainlit_deploy": chainlit_deploy + }) return values + @deprecated(reason="deplecated. Use FlaskRuntime.chat or ChainlitRuntime.chat() instead") def chat(self, message: Message, stream: bool = False, **args) -> Message: """ 执行一次对话 @@ -247,6 +268,7 @@ def chat(self, message: Message, stream: bool = False, **args) -> Message: """ return self.component.run(message=message, stream=stream, **args) + @deprecated(reason="deplecated. Use FlaskRuntime.create_flask_app() instead") def create_flask_app(self, url_rule="/chat"): """ 创建 Flask 应用，主要用于 Gunicorn 这样的 WSGI 服务器来运行服务。 @@ -257,181 +279,10 @@ def create_flask_app(self, url_rule="/chat"): Returns: Flask """ - # lazy import flask - try: - from flask import Flask, current_app, request, Response - from flask_restful import reqparse, Resource - from werkzeug.exceptions import BadRequest - from flask import stream_with_context - - except ImportError: - raise ImportError("Flask module is not installed. Please install it using 'pip install " - "flask~=2.3.2 flask-restful==0.3.9'.") - app = Flask(__name__) - app.json.ensure_ascii = False - - @app.errorhandler(BadRequest) - def handle_bad_request(e): - return {"code": 400, "message": f'{e}', "result": None}, 400 - - @app.errorhandler(Exception) - def handle_bad_request(e): - if hasattr(e, "code"): - return {"code": e.code, "message": str(e), "result": None}, 200 - else: - return {"code": 500, "message": "Internal Server Error", "result": None}, 200 - - def warp(): - """ - 根据component的lazy_certification属性处理请求。 - - Args: - 无参数。 - - Returns: - 如果stream为True，则返回流式响应（Content-Type为text/event-stream）。 - 如果stream为False，则返回包含处理结果的字典。 - - Raises: - BadRequest: 当请求头中缺少必要的X-Appbuilder-Authorization时抛出。 - BadRequest: 当请求体中缺少必要的message字段时抛出。 - BadRequest: 当请求体中session_id字段不是字符串类型时抛出。 - BadRequest: 当请求体中stream字段不是布尔类型时抛出。 - - """ - # 根据component是否lazy_certification，分成两种情况： - # 1. lazy_certification为True，初始化时未被认证，每次请求都需要带入AppbuilderToken - # 2. lazy_certification为False，初始化时已经认证，请求时不需要带入AppbuilderToken，并且带入也无效 - if self.component.lazy_certification: - app_builder_token = None - for key in ["X-Appbuilder-Token", "X-Appbuilder-Authorization"]: - if key in request.headers: - app_builder_token = request.headers[key] - break - if not app_builder_token: - raise BadRequest( - "X-Appbuilder-Authorization is required in Headers") - try: - self.component.set_secret_key_and_gateway( - secret_key=app_builder_token) - except appbuilder.core._exception.BaseRPCException as e: - logging.error(f"failed to verify. err={e}", exc_info=True) - raise BadRequest("X-Appbuilder-Authorization invalid") - except Exception as e: - logging.error(f"failed to verify. err={e}", exc_info=True) - raise e - else: - pass - - data = request.get_json() - if "message" not in data: - raise BadRequest("message is required") - message = Message(data.pop('message')) - if "session_id" not in data: - session_id = str(uuid.uuid4()) - else: - session_id = data.pop("session_id") - if not isinstance(session_id, str): - raise BadRequest("session_id must be str type") - if "stream" not in data: - stream = False - else: - stream = data.pop("stream") - if not isinstance(stream, bool): - raise BadRequest("stream must be bool type") - request_id = request.headers.get("X-Appbuilder-Request-Id", str(uuid.uuid4())) - user_id = request.headers.get("X-Appbuilder-User-Id", None) - - init_context(session_id=session_id, request_id=request_id, user_id=user_id) - logging.info( - f"request_id={request_id}, session_id={session_id}] message={message}," - f" stream={stream}, data={data}, start run...") - - def gen_sse_resp(): - with app.app_context(): - received_first_packet = False - retry_count = 0 - while retry_count < MAX_RETRY_COUNT: - try: - answer = self.chat(message, stream, **data) - except Exception as e: # 调用chat方法报错，直接返回 - code = 500 if not hasattr(e, "code") else e.code - err_resp = {"code": code, "message": "InternalServerError", "result": None} - logging.error( - f"request_id={request_id}, session_id={session_id}, err={e}, execute self.chat failed", exc_info=True) - yield "data: " + json.dumps(err_resp, ensure_ascii=False) + "\n\n" - return - else: # 调用chat方法成功，开始生成流式事件 - content_iterator = iter(answer.content) - answer.content = None - result = None - try: - for sub_content in content_iterator: - result = copy.deepcopy(answer) - result.content = sub_content - yield "data: " + json.dumps({ - "code": 0, "message": "", - "result": { - "session_id": session_id, - "is_completion": False, - "answer_message": json.loads(result.json(exclude_none=True)) - } - }, ensure_ascii=False) + "\n\n" - received_first_packet = True - except Exception as e: - retry_count += 1 - logging.error( - f"[request_id={request_id}, session_id={session_id}] err={e}, " - f"retry_count={retry_count}", exc_info=True) - # 如果未收到首包且重试次数小于最大重试次数，则尝试重新执行一次chat方法 - if not received_first_packet and retry_count < MAX_RETRY_COUNT: - continue - else: # 其它情况返回 - logging.error( - f"[request_id={request_id}, session_id={session_id}] err={e}, " - f"retry_count={retry_count}, received_first_packet={received_first_packet}" - , exc_info=True) - code = 500 if not hasattr(e, "code") else e.code - err_resp = {"code": code, "message": "InternalServerError", "result": None} - yield "data: " + json.dumps(err_resp, ensure_ascii=False) + "\n\n" - return - result.content = "" - yield "data: " + json.dumps({ - "code": 0, "message": "", - "result": { - "session_id": session_id, - "is_completion": True, - "answer_message": json.loads(result.json(exclude_none=True)) - } - }, ensure_ascii=False) + "\n\n" - logging.info( - f"request_id={request_id}, session_id={session_id}]" - f"retry_count={retry_count}, success response", exc_info=True) - self.user_session._post_append() - return # 正常返回 - - if stream: # 流式 - return Response(stream_with_context(gen_sse_resp()), 200, - {'Content-Type': 'text/event-stream; charset=utf-8'}) - if not stream: # 非流式 - try: - answer = self.chat(message, stream, **data) - blocking_result = json.loads(copy.deepcopy(answer).json(exclude_none=True)) - logging.debug(f"[request_id={request_id}, session_id={session_id}] blocking_result={blocking_result}") - self.user_session._post_append() - return { - "code": 0, "message": "", - "result": {"session_id": session_id, "answer_message": blocking_result} - } - except Exception as e: - logging.error( - f"[request_id={request_id}, session_id={session_id}] err={e}", exc_info=True) - code = 500 if not hasattr(e, "code") else e.code - return {"code": code, "message": "InternalServerError", "result": None} - - app.add_url_rule(url_rule, 'chat', warp, methods=['POST']) + app = self.flask_deploy.create_flask_app(url_rule=url_rule) return app + @deprecated(reason="deplecated. Use FlaskRuntime.serve() instead") def serve(self, host='0.0.0.0', debug=True, port=8092, url_rule="/chat"): """ 将 component 服务化，提供 Flask http API 接口 @@ -448,39 +299,7 @@ def serve(self, host='0.0.0.0', debug=True, port=8092, url_rule="/chat"): app = self.create_flask_app(url_rule=url_rule) app.run(host=host, debug=debug, port=port) - def prepare_chainlit_readme(self): - """ - 准备 Chainlit 的 README 文件 - - Args: - 无 - - Returns: - 无 - - Raises: - 无 - - 说明: - 从 utils 文件夹中拷贝 chainlit.md 文件到当前工作目录下，如果当前工作目录下已存在 chainlit.md 文件，则不拷贝。 - - """ - try: - # 获取当前python命令执行的路径，而不是文件的位置 - cwd_path = os.getcwd() - # 获取当前文件的路径所在文件夹 - current_file_path = os.path.dirname( - os.path.dirname(os.path.abspath(__file__))) - chainlit_readme_path = os.path.join( - current_file_path, "utils", "chainlit.md") - - # 拷贝chainlit_readme到cwd_path - # 如果cwd_path下已经存在chainlit_readme，则不拷贝 - if not os.path.exists(os.path.join(cwd_path, "chainlit.md")): - shutil.copy(chainlit_readme_path, cwd_path) - except: - logger.error("Failed to copy chainlit.md to current directory") - + @deprecated(reason="deplecated. Use ChainlitRuntime.chainlit_component() instead") def chainlit_demo(self, host='0.0.0.0', port=8091): """ 将 component 服务化，提供 chainlit demo 页面 @@ -492,43 +311,9 @@ def chainlit_demo(self, host='0.0.0.0', port=8091): Returns: None """ - # lazy import chainlit - try: - import chainlit as cl - import chainlit.cli - except ImportError: - raise ImportError("chainlit module is not installed. Please install it using 'pip install " - "chainlit~=1.0.200'.") - import click - from click.testing import CliRunner - - self.prepare_chainlit_readme() - - @cl.on_message # this function will be called every time a user inputs a message in the UI - async def main(message: cl.Message): - session_id = cl.user_session.get("id") - request_id = str(uuid.uuid4()) - init_context(session_id=session_id, request_id=request_id) - msg = cl.Message(content="") - await msg.send() - stream_message = self.chat(Message(message.content), stream=True) - - for part in stream_message.content: - if token := part or "": - await msg.stream_token(token) - await msg.update() - self.user_session._post_append() - - # start chainlit service - if os.getenv('APPBUILDER_RUN_CHAINLIT') == '1': - pass - else: - os.environ['APPBUILDER_RUN_CHAINLIT'] = '1' - target = sys.argv[0] - runner = CliRunner() - runner.invoke( - chainlit.cli.chainlit_run, [target, '--watch', "--port", port, "--host", host]) + self.chainlit_deploy.chainlit_component(host=host, port=port) + @deprecated(reason="deplecated. Use ChainlitRuntime.chainlit_agent() instead") def chainlit_agent(self, host='0.0.0.0', port=8091): """ 将 appbuilder client 服务化，提供 chainlit demo 页面 @@ -540,100 +325,4 @@ def chainlit_agent(self, host='0.0.0.0', port=8091): Returns: None """ - # lazy import chainlit - try: - import chainlit as cl - import chainlit.cli - except ImportError: - raise ImportError("chainlit module is not installed. Please install it using 'pip install " - "chainlit~=1.0.200'.") - import click - from click.testing import CliRunner - - if not isinstance(self.component, appbuilder.AppBuilderClient): - raise ValueError( - "chainlit_agent require component must be an instance of AppBuilderClient") - self.prepare_chainlit_readme() - - conversation_ids = [] - interrupt_dict = {} - - def _chat(message: cl.Message): - if len(conversation_ids) == 0: - raise ValueError("create new conversation failed!") - conversation_id = conversation_ids[-1] - file_ids = [] - if len(message.elements) > 0: - file_id = self.component.upload_local_file( - conversation_id, message.elements[0].path) - file_ids.append(file_id) - - interrupt_ids = interrupt_dict.get(conversation_id, []) - interrupt_event_id = interrupt_ids.pop() if len(interrupt_ids) > 0 else None - action = None - if interrupt_event_id is not None: - action = Action.create_resume_action(interrupt_event_id) - - tmp_message = self.component.run(conversation_id=conversation_id, query=message.content, file_ids=file_ids, - stream=True, tool_choice=self.tool_choice, action=action) - res_message=list(tmp_message.content) - - interrupt_event_id = None - for ans in res_message: - for event in ans.events: - if event.content_type == "chatflow_interrupt": - interrupt_event_id = event.detail.get("interrupt_event_id") - if event.content_type == "publish_message" and event.event_type == "chatflow": - answer = event.detail.get("message") - ans.answer += answer - - if interrupt_event_id is not None: - interrupt_ids.append(interrupt_event_id) - interrupt_dict[conversation_id] = interrupt_ids - tmp_message.content = res_message - return tmp_message - - @cl.on_chat_start - async def start(): - session_id = cl.user_session.get("id") - request_id = str(uuid.uuid4()) - init_context(session_id=session_id, request_id=request_id) - conversation_ids.append(self.component.create_conversation()) - interrupt_dict[conversation_ids[-1]] = [] - - @cl.on_message # this function will be called every time a user inputs a message in the UI - async def main(message: cl.Message): - msg = cl.Message(content="") - await msg.send() - await msg.update() - - stream_message = _chat(message) - detail_json_list = [] - for part in stream_message.content: - if token := part.answer or "": - await msg.stream_token(token) - for event in part.events: - detail = event.detail - detail_json = json.dumps( - detail, indent=4, ensure_ascii=False) - detail_json_list.append(detail_json) - await msg.update() - - @cl.step(name="详细信息") - def show_json(detail_json): - return "```json\n" + detail_json + "\n```" - for detail_json in detail_json_list: - if len(detail_json) > 2: - show_json(detail_json) - await msg.update() - self.user_session._post_append() - - # start chainlit service - if os.getenv('APPBUILDER_RUN_CHAINLIT') == '1': - pass - else: - os.environ['APPBUILDER_RUN_CHAINLIT'] = '1' - target = sys.argv[0] - runner = CliRunner() - runner.invoke( - chainlit.cli.chainlit_run, [target, '--watch', "--port", port, "--host", host]) + self.chainlit_deploy.chainlit_component(host=host, port=port) \ No newline at end of file diff --git a/python/core/context.py b/python/core/context.py index 15c97f9bc..ebb294d17 100644 --- a/python/core/context.py +++ b/python/core/context.py @@ -68,8 +68,8 @@ def get_context() -> SessionContext: session_id = _LOCAL_KEY + str(uuid.uuid4())[:-len(_LOCAL_KEY)] request_id = _LOCAL_KEY + str(uuid.uuid4())[:-len(_LOCAL_KEY)] logging.debug( - "Unable to find the AgentRuntime context. You need to use UserSession " - "in AgentRuntime.serve or AgentRuntime.chainlit_demo. " + "Unable to find the context. You need to use UserSession " + "in FlaskRuntime.serve or ChainlitRuntime.chainlit_component. " f"Generate session_id({session_id}) and request_id({request_id}) here for local debugging.") ctx = init_context(session_id=session_id, request_id=request_id) return ctx \ No newline at end of file diff --git a/python/core/user_session.py b/python/core/user_session.py index a8b938ada..5c06fdd9a 100644 --- a/python/core/user_session.py +++ b/python/core/user_session.py @@ -33,7 +33,7 @@ def lazy_import_sqlalchemy(): class UserSession(object): """ 会话数据管理工具，实例化后将是一个全局变量。 - 提供保存对话数据与获取历史数据的方法，**必须**在 AgentRuntime 启动的服务中使用。 + 提供保存对话数据与获取历史数据的方法，**必须**在 FlaskRuntime和ChainlitRuntime 启动的服务中使用。 """ _instance = None _initialized = False diff --git a/python/tests/component_tool_eval_cases.py b/python/tests/component_tool_eval_cases.py index ae7809d0e..92e529aa8 100644 --- a/python/tests/component_tool_eval_cases.py +++ b/python/tests/component_tool_eval_cases.py @@ -12,7 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. from appbuilder.core.component import Component -from appbuilder.tests.component_schemas import text_schema, url_schema, image_schema, code_schema, file_schema, oral_text_schema, references_schema, chart_schema, audio_schema, plan_schema, function_call_schema +from appbuilder.tests.component_schemas import text_schema, url_schema, image_schema, \ + code_schema, file_schema, oral_text_schema, references_schema, chart_schema, audio_schema, plan_schema, function_call_schema class Case(): def init_args(self): diff --git a/python/tests/test_agent.py b/python/tests/test_agent.py index b63e8ecc3..43176b33c 100644 --- a/python/tests/test_agent.py +++ b/python/tests/test_agent.py @@ -8,13 +8,14 @@ from appbuilder.core.component import Component from appbuilder.utils.sse_util import SSEClient from appbuilder import ( - AgentRuntime, Message, Playground, AppBuilderClient ) +from appbuilder.utils.flask_deploy import FlaskRuntime +from appbuilder.utils.chainlit_deploy import ChainlitRuntime -class TestAgentRuntime(unittest.TestCase): +class TestRuntime(unittest.TestCase): def setUp(self): """ 设置环境变量。 @@ -34,7 +35,7 @@ def test_no_token_http(self): model="ERNIE-3.5-8K", lazy_certification=True, ) - agent = appbuilder.AgentRuntime(component=component) + agent = FlaskRuntime(component=component) app = agent.create_flask_app(url_rule="/chat") app.config['TESTING'] = True client = app.test_client() @@ -49,12 +50,13 @@ def test_no_token_http(self): def test_err_http(self): """ 测试http """ + print(os.getenv("APPBUILDER_TOKEN", "")) component = appbuilder.Playground( prompt_template="{query}", model="ERNIE-3.5-8K", lazy_certification=True, ) - agent = appbuilder.AgentRuntime(component=component) + agent = FlaskRuntime(component=component) app = agent.create_flask_app(url_rule="/chat") app.config['TESTING'] = True client = app.test_client() @@ -77,7 +79,7 @@ def test_stream_http(self): model="ERNIE-3.5-8K", lazy_certification=True, ) - agent = appbuilder.AgentRuntime(component=component) + agent = FlaskRuntime(component=component) app = agent.create_flask_app(url_rule="/chat") app.config['TESTING'] = True client = app.test_client() @@ -105,7 +107,7 @@ def test_http(self): model="ERNIE-3.5-8K", lazy_certification=True, ) - agent = appbuilder.AgentRuntime(component=component) + agent = FlaskRuntime(component=component) app = agent.create_flask_app(url_rule="/chat") app.config['TESTING'] = True client = app.test_client() @@ -128,13 +130,8 @@ def test_init_with_valid_component(self): prompt_template="{query}", model="ERNIE-3.5-8K" ) - agent = AgentRuntime(component=component) - - def test_init_with_invalid_component(self): - """ 测试在component非法时运行 """ - component = "invalid_component" - with self.assertRaises(pydantic.ValidationError): - agent = AgentRuntime(component=component) + agent = FlaskRuntime(component=component) + agent = ChainlitRuntime(component=component) def test_chat_with_valid_message_and_blocking(self): """ 测试在消息有效时处理 """ @@ -142,18 +139,22 @@ def test_chat_with_valid_message_and_blocking(self): prompt_template="{query}", model="ERNIE-3.5-8K" ) - agent = appbuilder.AgentRuntime(component=component) + agent = FlaskRuntime(component=component) message = appbuilder.Message({"query": "你好"}) answer = agent.chat(message, stream=False) + agent2 = ChainlitRuntime(component=component) + message = appbuilder.Message({"query": "你好"}) + answer2 = agent2.chat(message, stream=False) self.assertIs(type(answer.content), str) - + self.assertIs(type(answer2.content), str) + def test_chat_with_valid_message_and_streaming(self): """ 测试在消息有效时处理 """ component = Playground( prompt_template="{query}", model="ERNIE-3.5-8K" ) - agent = AgentRuntime(component=component) + agent = FlaskRuntime(component=component) message = Message({"query": "你好"}) answer = agent.chat(message, stream=True) for it in answer.content: diff --git a/python/tests/test_appbuilder_client.py b/python/tests/test_appbuilder_client.py index ee616cf10..b010f0a23 100644 --- a/python/tests/test_appbuilder_client.py +++ b/python/tests/test_appbuilder_client.py @@ -7,7 +7,7 @@ @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_run_with_handler.py b/python/tests/test_appbuilder_client_run_with_handler.py index 00a213593..61070f521 100644 --- a/python/tests/test_appbuilder_client_run_with_handler.py +++ b/python/tests/test_appbuilder_client_run_with_handler.py @@ -56,7 +56,7 @@ def tool_calls(self, event): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppbuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall.py b/python/tests/test_appbuilder_client_toolcall.py index 94c0c8924..7fb9dd30a 100644 --- a/python/tests/test_appbuilder_client_toolcall.py +++ b/python/tests/test_appbuilder_client_toolcall.py @@ -4,7 +4,7 @@ @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_event_handler.py b/python/tests/test_appbuilder_client_toolcall_event_handler.py index 044a5418c..2b3d6750e 100644 --- a/python/tests/test_appbuilder_client_toolcall_event_handler.py +++ b/python/tests/test_appbuilder_client_toolcall_event_handler.py @@ -34,7 +34,7 @@ def success(self, run_context, run_response): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_event_handler_error.py b/python/tests/test_appbuilder_client_toolcall_event_handler_error.py index afce553be..bb33ffae4 100644 --- a/python/tests/test_appbuilder_client_toolcall_event_handler_error.py +++ b/python/tests/test_appbuilder_client_toolcall_event_handler_error.py @@ -22,7 +22,7 @@ def success(self, run_context, run_response): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_event_handler_stream.py b/python/tests/test_appbuilder_client_toolcall_event_handler_stream.py index 6d4114242..e882aa31e 100644 --- a/python/tests/test_appbuilder_client_toolcall_event_handler_stream.py +++ b/python/tests/test_appbuilder_client_toolcall_event_handler_stream.py @@ -33,7 +33,7 @@ def running(self, run_context, run_response): print("\n\033[1;31m","-> Agent 流式回答: \n", run_response.answer, "\033[0m") @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_event_handler_v2.py b/python/tests/test_appbuilder_client_toolcall_event_handler_v2.py index edd454342..51c167b7c 100644 --- a/python/tests/test_appbuilder_client_toolcall_event_handler_v2.py +++ b/python/tests/test_appbuilder_client_toolcall_event_handler_v2.py @@ -34,7 +34,7 @@ def success(self, run_context, run_response): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_event_handler_v3.py b/python/tests/test_appbuilder_client_toolcall_event_handler_v3.py index 9a6e5abe5..3b7926882 100644 --- a/python/tests/test_appbuilder_client_toolcall_event_handler_v3.py +++ b/python/tests/test_appbuilder_client_toolcall_event_handler_v3.py @@ -34,7 +34,7 @@ def running(self, run_context, run_response): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_stream.py b/python/tests/test_appbuilder_client_toolcall_stream.py index f2971e4aa..468bb79a1 100644 --- a/python/tests/test_appbuilder_client_toolcall_stream.py +++ b/python/tests/test_appbuilder_client_toolcall_stream.py @@ -5,7 +5,7 @@ import os @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_v2.py b/python/tests/test_appbuilder_client_toolcall_v2.py index 45bbcc460..a9351f453 100644 --- a/python/tests/test_appbuilder_client_toolcall_v2.py +++ b/python/tests/test_appbuilder_client_toolcall_v2.py @@ -5,7 +5,7 @@ import os @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_appbuilder_client_toolcall_v3.py b/python/tests/test_appbuilder_client_toolcall_v3.py index 9fdc73486..78eca25ad 100644 --- a/python/tests/test_appbuilder_client_toolcall_v3.py +++ b/python/tests/test_appbuilder_client_toolcall_v3.py @@ -5,7 +5,7 @@ import os @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL","") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_async_appbuilder_client_toolcall.py b/python/tests/test_async_appbuilder_client_toolcall.py index 1e03f834b..b4c85917d 100644 --- a/python/tests/test_async_appbuilder_client_toolcall.py +++ b/python/tests/test_async_appbuilder_client_toolcall.py @@ -47,7 +47,7 @@ async def success(self, run_context, run_response): @unittest.skipUnless(os.getenv("TEST_CASE", "UNKNOWN") == "CPU_SERIAL", "") -class TestAgentRuntime(unittest.TestCase): +class TestAppBuilderClient(unittest.TestCase): def setUp(self): """ 设置环境变量。 diff --git a/python/tests/test_core_agent.py b/python/tests/test_core_agent.py index 23de1a4a7..ea0caa56e 100644 --- a/python/tests/test_core_agent.py +++ b/python/tests/test_core_agent.py @@ -17,7 +17,7 @@ import random from appbuilder.core.components.llms.style_writing import StyleWriting -from appbuilder.core.agent import AgentRuntime +from appbuilder.utils.flask_deploy import FlaskRuntime from appbuilder.core.component import Component from appbuilder.core.message import Message from appbuilder.utils.sse_util import SSEClient @@ -69,8 +69,8 @@ def setUp(self): def test_core_agent_create_flask1(self): component = FakeComponent1() - # agent = AgentRuntime(component=StyleWriting(model="eb")) - agent = AgentRuntime(component=component) + # agent = FlaskRuntime(component=StyleWriting(model="eb")) + agent = FlaskRuntime(component=component) app = agent.create_flask_app() client = app.test_client() @@ -122,7 +122,7 @@ def test_core_agent_create_flask1(self): def test_core_agent_create_flask2(self): component = FakeComponent2() - agent = AgentRuntime(component=component) + agent = FlaskRuntime(component=component) app = agent.create_flask_app() client = app.test_client() payload = { diff --git a/python/utils/chainlit.md b/python/utils/chainlit.md index d9ebfc4d8..32b94f6ca 100644 --- a/python/utils/chainlit.md +++ b/python/utils/chainlit.md @@ -11,8 +11,9 @@ - 提供 `API调用` & `交互式窗口` 两种服务化部署方式，支持快速上云，平滑嵌入到你的产品中 ## 如何使用AppBuilder-SDK & Chainlit可视化功能💻 -当前SDK中的`AgentRuntime`模块基于Chainlit实现了基础的可视化功能，支持AppBuilderClient + 能力组件实现可视化交互。 -- `chainlit_demo`接口支持基础组件的简单交互 +当前SDK中的`ChainlitRuntime`模块基于Chainlit实现了基础的可视化功能，支持AppBuilderClient + 能力组件实现可视化交互。 +- `chainlit_component`接口支持基础组件的简单文本交互 +- `chainlit_component_debug`接口支持基础组件的tool_eval方法交互，支持新协议输出渲染 - `chainlit_agent`接口支持AppBuilderClient的进阶交互，提供新建会话和上传文件的功能 如果对于可视化有更多需求，可以参考AppBuilder SDK中的代码，基于Chainlit进行二次开发。 diff --git a/python/utils/chainlit_deploy.py b/python/utils/chainlit_deploy.py new file mode 100644 index 000000000..fc9b2efc0 --- /dev/null +++ b/python/utils/chainlit_deploy.py @@ -0,0 +1,694 @@ +"""chainlit deploy""" +import os, sys +import shutil +import json +from typing import Any, Optional, Union +from click.testing import CliRunner +import uuid +import logging + +import appbuilder +from appbuilder.core.component import Component, Content +from appbuilder.core.message import Message +from appbuilder.utils.logger_util import logger +from appbuilder.core.context import init_context +from appbuilder.core.user_session import UserSession +from appbuilder.core.console.appbuilder_client.data_class import ToolChoice, Action + + +class ChainlitRuntime(object): + """ChainlitRuntime 是对组件和应用调用的chainlit服务化封装，开发者不是必须要用 ChainlitRuntime 才能运行自己的组件服务。 + ChainlitRuntime 可以快速帮助开发者服务化组件服务，并且提供对话框部署功能。 + 此外，结合 Component 和 Message 自带的运行和调试接口，可以方便开发者快速获得一个调试 Agent 的服务。 + + Examples: + + .. code-block:: python + + import os + import sys + import appbuilder + from appbuilder.utils.chainlit_deploy import ChainlitRuntime + os.environ["APPBUILDER_TOKEN"] = '...' + + component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" + ) + agent = ChainlitRuntime(component=component) + message = appbuilder.Message({"query": "你好"}) + print(agent.chat(message, stream=False)) + + .. code-block:: python + + import os + import sys + import appbuilder + from appbuilder.utils.chainlit_deploy import ChainlitRuntime + from + os.environ["APPBUILDER_TOKEN"] = '...' + + component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" + ) + agent = ChainlitRuntime(component=component) + message = appbuilder.Message({"query": "你好"}) + print(agent.chat(message, stream=False)) + + .. code-block:: python + + import os + import sys + import appbuilder + from appbuilder.utils.chainlit_deploy import ChainlitRuntime + from appbuilder.core.components.v2 import SimilarQuestion + os.environ["APPBUILDER_TOKEN"] = '...' + component = SimilarQuestion() + agent = ChainlitRuntime(component=component) + agent.chainlit_component_debug( + port=8092, + tool_eval_args={}, + query_name = "query" + ) + + Session 数据管理 : 除去上述简单应用外，还支持 Session 数据管理，下面是一个例子 + + .. code-block:: python + + import os + import sys + from appbuilder.core.component import Component + from appbuilder import ( + ChainlitRuntime, UserSession, Message, QueryRewrite, Playground, + ) + + os.environ["APPBUILDER_TOKEN"] = '...' + + class PlaygroundWithHistory(Component): + def __init__(self): + super().__init__() + self.query_rewrite = QueryRewrite(model="Qianfan-Agent-Speed-8k") + self.play = Playground( + prompt_template="{query}", + model="eb-4" + ) + + def run(self, message: Message, stream: bool=False): + user_session = UserSession() + # 获取 Session 历史数据 + history_queries = user_session.get_history("query", limit=1) + history_answers = user_session.get_history("answer", limit=1) + + if history_queries and history_answers: + history = [] + for query, answer in zip(history_queries, history_answers): + history.extend([query.content, answer.content]) + logging.info(f"history: {history}") + message = self.query_rewrite( + Message(history + [message.content]), rewrite_type="带机器人回复") + logging.info(f"message: {message}") + answer = self.play.run(message, stream) + # 保存本轮数据 + user_session.append({ + "query": message, + "answer": answer, + }) + return answer + + agent = ChainlitRuntime(component=PlaygroundWithHistory()) + agent.chainlit_component(port=8091) + + Session 信息查看 : 查看本地user_session.db数据库内部信息，下面是一个例子 + + .. code-block:: python + + import sqlite3 + import json + + # 连接到 SQLite 数据库 + # 如果文件不存在，会自动在当前目录创建: + user_session_path = 'your_user_session.db地址' + conn = sqlite3.connect(user_session_path) + cursor = conn.cursor() + + # 执行一条 SQL 语句，列出所有表 + cursor.execute("SELECT name FROM sqlite_master WHERE type='table';") + print(cursor.fetchall()) + + # 查询appbuilder_session_messages表的列信息 + cursor.execute("PRAGMA table_info(appbuilder_session_messages);") + columns_info = cursor.fetchall() + + column_names = [info[1] for info in columns_info] # info[1]是列名的位置 + for column_name in column_names: + print(column_name) + + # 查询特定表中的数据 + cursor.execute("SELECT message_value FROM appbuilder_session_messages;") + for row in cursor.fetchall(): + print(json.loads(row[0])) + + # 关闭 Connection: + conn.close() + + """ + def __init__(self, + component: Component, + user_session_config: Optional[Union[Any, str]] = None, + user_session: Optional[UserSession] = None, + tool_choice: ToolChoice = None + ): + """init + + Args: + component (Component): 需要服务化的组件实例 + user_session_config (sqlalchemy.engine.URL|str|None): Session 输出存储配置字符串。默认使用 sqlite:///user_session.db + 遵循 sqlalchemy 后端定义，参考文档：https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls + user_session (UserSession): 用户会话管理器，如果不指定则自动生成一个默认的 UserSession + tool_choice (ToolChoice): 可用于Agent强制执行的组件工具 + + """ + self.component = component + if user_session is None: + if user_session_config is None: + self.user_session = UserSession() + logger.info("init user_session with default UserSession") + else: + self.user_session = UserSession(user_session_config) + logger.info("init user_session with user_session_config") + else: + self.user_session = user_session + self.tool_choice = tool_choice + self._prepare_chainlit_readme() + + def chat(self, message: Message, stream: bool = False, **kwargs) -> Message: + """ + 执行一次对话 + + Args: + message (Message): 该次对话用户输入的 Message + stream (bool): 是否流式请求 + **args: 其他参数，会被透传到 component + + Returns: + Message(Message): 返回的 Message + """ + return self.component.run(message=message, stream=stream, **kwargs) + + def _prepare_chainlit_readme(self): + """ + 准备 Chainlit 的 README 文件 + 从 utils 文件夹中拷贝 chainlit.md 文件到当前工作目录下，如果当前工作目录下已存在 chainlit.md 文件，则不拷贝。 + + Args: + None + + Returns: + None + + Raises: + None + """ + try: + # 获取当前python命令执行的路径，而不是文件的位置 + cwd_path = os.getcwd() + # 获取当前文件的路径所在文件夹 + current_file_path = os.path.dirname( + os.path.dirname(os.path.abspath(__file__))) + chainlit_readme_path = os.path.join( + current_file_path, "utils/chainlit.md") + logger.debug(f"chainlit_readme_path:{chainlit_readme_path}") + if not os.path.exists(chainlit_readme_path): + raise FileNotFoundError(f"Chainlit readme file not found at {chainlit_readme_path}") + + # 拷贝chainlit_readme到cwd_path + # 如果cwd_path下已经存在chainlit_readme，则不拷贝 + if not os.path.exists(os.path.join(cwd_path, "chainlit.md")): + shutil.copy(chainlit_readme_path, cwd_path) + logger.info("chainlit readme file copied successfully") + except: + logger.error("Failed to copy chainlit.md to current directory") + + def chainlit_component(self, host='0.0.0.0', port=8091): + """ + 将 component 服务化，提供 chainlit demo 页面 + + Args: + host (str): 服务 host + port (int): 服务 port + + Returns: + None + """ + try: + import chainlit as cl + import chainlit.cli + except ImportError: + raise ImportError("chainlit module is not installed. Please install it using 'pip install " + "chainlit'.") + @cl.on_message # this function will be called every time a user inputs a message in the UI + async def main(message: cl.Message): + session_id = cl.user_session.get("id") + request_id = str(uuid.uuid4()) + init_context(session_id=session_id, request_id=request_id) + msg = cl.Message(content="") + await msg.send() + stream_message = self.chat(Message(message.content), stream=True) + + for part in stream_message.content: + if token := part or "": + await msg.stream_token(token) + await msg.update() + self.user_session._post_append() + + if os.getenv('APPBUILDER_RUN_CHAINLIT') == '1': + pass + else: + os.environ['APPBUILDER_RUN_CHAINLIT'] = '1' + target = sys.argv[0] + runner = CliRunner() + runner.invoke( + chainlit.cli.chainlit_run, [target, '--watch', "--port", port, "--host", host]) + + def chainlit_component_debug( + self, + host='0.0.0.0', + port=8092, + query_name: str = "query", + tool_eval_args: dict = {}): + """componnet tool_eval""" + """ + 将 appbuilder component 服务化，内部调用组件tool_eval方法，提供 chainlit demo 页面和各类型输出的渲染效果。 + 注意：目前只支持新协议组件 + + Args: + host (str): 服务 host + port (int): 服务 port + query_name(str): 组件tool_eval运行时接收自然语言输入的变量名 + tool_eval_args(dict): 组件tool_eval运行需要的其他参数 + + Returns: + None + """ + try: + import chainlit as cl + import chainlit.cli + except ImportError: + raise ImportError("chainlit module is not installed. Please install it using 'pip install " + "chainlit'.") + + def _chat(message: cl.Message): + query = message.content + tool_eval_args.update({query_name: query}) + tool_result = self.component.tool_eval(**tool_eval_args) + try: + message_result = _convert_componentoutput_to_msg(tool_result) + yield from message_result + except Exception as e: + text = "组件输出报错：" + str(e) + elements = [ + cl.Text( + name="error", + content=text + ) + ] + message_result = cl.Message( + content="组件输出报错", + elements=elements + ) + yield message_result + + def _echart_to_plotly(echarts_data): + """把echart图表数据转换为plotly.graph_objects.Figure图片""" + try: + import plotly.graph_objects as go + except ImportError: + raise ImportError("plotly module is not installed. Please install it using 'pip install plotly'") + + fig = go.Figure() + + # 处理不同的图表类型 + for series in echarts_data.get('series', []): + if "dataset" in echarts_data: + data = echarts_data['dataset'][0]["source"] + else: + data = series['data'] + + if series['type'] == 'bar': + fig.add_trace(go.Bar( + x=echarts_data['xAxis']['data'], + y=data, + name=series.get('name', '') + )) + elif series['type'] == 'line': + if 'xAxis' in echarts_data and 'data' in echarts_data['xAxis']: + x = echarts_data['xAxis']['data'] + y = data + else: + x = [point[0] for point in data] + y = [point[1] for point in data] + fig.add_trace(go.Scatter( + x=x, + y=y, + mode='lines+markers', + name=series.get('name', '') + )) + elif series['type'] == 'scatter': + fig.add_trace(go.Scatter( + x=[point[0] for point in data], + y=[point[1] for point in data], + mode='markers', + name=series.get('name', '') + )) + elif series['type'] == 'pie': + fig.add_trace(go.Pie( + labels=[item['name'] for item in data], + values=[item['value'] for item in data], + name=series.get('name', '') + )) + elif series['type'] in ['tree', 'treemap']: + data = data[0]['children'] + def process_data(data, parent=""): + labels = [] + parents = [] + for item in data: + labels.append(item['name']) + parents.append(parent) + if item['children']: + child_labels, child_parents = process_data(item['children'], item['name']) + labels.extend(child_labels) + parents.extend(child_parents) + return labels, parents + + labels, parents = process_data(data) + + # 创建Treemap图 + fig.add_trace(go.Treemap( + labels=labels, + parents=parents + )) + elif series['type'] == 'funnel': + fig.add_trace(go.Funnel( + y=[item['name'] for item in data], + x=[item['value'] for item in data] + )) + elif series['type'] == 'radar': + indicator_data = echarts_data["radar"]["indicator"] + series_data = data[0]["value"] + for indicator in indicator_data: + fig.add_trace(go.Scatterpolar( + r=series_data, + theta=[indicator["name"]] * len(series_data), + fill="toself", + name=indicator["name"], + subplot="polar" + )) + + # 更新布局 + layout_update = { + 'title': echarts_data.get('title', {}).get('text', ''), + 'legend': dict(orientation='h', yanchor='bottom', y=-0.2), + 'margin': dict(t=70, r=100, b=50, l=20) + } + + if 'radar' in echarts_data: + layout_update['polar'] = { + 'radialaxis': {'visible': True}, + 'angularaxis': {'visible': True} + } + else: + layout_update['xaxis_title'] = echarts_data.get('xAxis', {}).get('name', '') + if 'yAxis' in echarts_data: + yaxis = echarts_data['yAxis'] + if isinstance(yaxis, list): + yaxis = yaxis[0] + else: + yaxis = {} + layout_update['yaxis_title'] = yaxis.get('name', '') + + fig.update_layout(**layout_update) + return fig + + def _convert_code(name: str, code: str): + """把组件code输出转换为code格式的Text输出""" + from pygments.lexers import guess_lexer + try: + lexer = guess_lexer(code) + language = lexer.name + except: + language = "" + logging.info(f"langauge: {language}") + return cl.Text( + name=name, + content=code, + display="inline", + language=language + ) + + def _convert_text(text): + """解析组件Text输出，主要提取echart图表数据""" + elements = [] + import re + pattern = r'\n\n```echarts-option\n(.*?)\n```' + echarts_match = re.search(pattern, text.info, re.DOTALL) + if echarts_match: + echarts_data = echarts_match.group(1) + logging.info(f"chart data: {echarts_data}") + echarts_data_list = json.loads(echarts_data) + for echarts_data in echarts_data_list: + fig = _echart_to_plotly(echarts_data) + elements.append(cl.Plotly( + name="chart", + figure=fig, + display="inline" + )) + + rest = text.info[echarts_match.end():].strip() + if rest: + elements.append(cl.Text( + name="text", + content=rest, + display = "inline" + )) + if not echarts_match: + elements.append(cl.Text( + name="text", + content=text.info.strip(), + display = "inline" + )) + return elements + + def _convert_content_to_elements(content: Content) -> list: + """把AB ComponentOutput.Content转换为chainlit的elements""" + elements = [] + type = content.type + text = content.text + if type == "text" or type == "oral_text": + elements.extend(_convert_text(text)) + elif type == "image": + elements.append(cl.Image( + name="image", + display="inline", + size="large", + url=text.url, + )) + elif type == "files": + if text.url: + elements.append(cl.File( + name=text.filename, + url=text.url, + display="inline", + )) + elif type == "audio": + elements.append(cl.Audio( + name="audio", + display="inline", + size="medium", + url=text.url + )) + elif type == "chart": + json_data = json.loads(text.data) + fig = _echart_to_plotly(json_data) + elements.append(cl.Plotly( + name="chart", + figure=fig, + display="inline" + )) + elif type == "code": + elements.append(_convert_code( + "code", text.code + )) + elif type == "references": + json_data = text.model_dump_json(indent=4) + elements.append(cl.Text( + name="reference", + content=json_data, + display="inline", + language="json" + )) + elif type == "json": + data = json.loads(text.data) + data_str = json.dumps( + data, + ensure_ascii=False, + indent=4 + ) + elements.append(cl.Text( + name="json", + content=data_str, + display="inline", + )) + elif type == "urls": + elements.append(cl.Text( + name="url", + content="下载链接："+text.url + )) + else: + logging.info(f"invalid type: {type}") + raise ValueError("invalid type") + + return elements + + def _convert_componentoutput_to_msg(result): + """把ComponentOutput转换为cl.Message""" + elements = [] + for iter in result: + content = iter.content[0] + elements = _convert_content_to_elements(content) + tool_name = component.manifests[0]["name"] + msg = cl.Message(content=tool_name, elements=elements) + yield msg + + @cl.on_message # this function will be called every time a user inputs a message in the UI + async def main(message: cl.Message): + logging.info(f"message: {message.content}") + stream_message = _chat(message) + for msg in stream_message: + logging.info(f"msg {msg}") + await msg.send() + + await msg.update() + + if os.getenv('APPBUILDER_RUN_CHAINLIT') == '1': + pass + else: + os.environ['APPBUILDER_RUN_CHAINLIT'] = '1' + target = sys.argv[0] + runner = CliRunner() + runner.invoke( + chainlit.cli.chainlit_run, [target, '--watch', "--port", port, "--host", host]) + + + def chainlit_agent(self, host='0.0.0.0', port=8091): + """ + 将 appbuilder client 服务化，提供 chainlit demo 页面 + + Args: + host (str): 服务 host + port (int): 服务 port + + Returns: + None + """ + try: + import chainlit as cl + import chainlit.cli + except ImportError: + raise ImportError("chainlit module is not installed. Please install it using 'pip install " + "chainlit'.") + if not isinstance(self.component, appbuilder.AppBuilderClient): + raise ValueError( + "chainlit_agent require component must be an instance of AppBuilderClient") + + conversation_ids = [] + interrupt_dict = {} + + def _chat(message: cl.Message): + if len(conversation_ids) == 0: + raise ValueError("create new conversation failed!") + conversation_id = conversation_ids[-1] + file_ids = [] + if len(message.elements) > 0: + file_id = self.component.upload_local_file( + conversation_id, message.elements[0].path) + file_ids.append(file_id) + + interrupt_ids = interrupt_dict.get(conversation_id, []) + interrupt_event_id = interrupt_ids.pop() if len(interrupt_ids) > 0 else None + action = None + if interrupt_event_id is not None: + action = Action.create_resume_action(interrupt_event_id) + + tmp_message = self.component.run(conversation_id=conversation_id, query=message.content, file_ids=file_ids, + stream=True, tool_choice=self.tool_choice, action=action) + res_message=list(tmp_message.content) + + interrupt_event_id = None + for ans in res_message: + for event in ans.events: + if event.content_type == "chatflow_interrupt": + interrupt_event_id = event.detail.get("interrupt_event_id") + if event.content_type == "publish_message" and event.event_type == "chatflow": + answer = event.detail.get("message") + ans.answer += answer + + if interrupt_event_id is not None: + interrupt_ids.append(interrupt_event_id) + interrupt_dict[conversation_id] = interrupt_ids + tmp_message.content = res_message + return tmp_message + + @cl.on_chat_start + async def start(): + session_id = cl.user_session.get("id") + request_id = str(uuid.uuid4()) + init_context(session_id=session_id, request_id=request_id) + conversation_ids.append(self.component.create_conversation()) + interrupt_dict[conversation_ids[-1]] = [] + + @cl.on_message # this function will be called every time a user inputs a message in the UI + async def main(message: cl.Message): + msg = cl.Message(content="") + await msg.send() + await msg.update() + + stream_message = _chat(message) + detail_json_list = [] + for part in stream_message.content: + if token := part.answer or "": + await msg.stream_token(token) + for event in part.events: + detail = event.detail + detail_json = json.dumps( + detail, indent=4, ensure_ascii=False) + detail_json_list.append(detail_json) + await msg.update() + + @cl.step(name="详细信息") + def show_json(detail_json): + return "```json\n" + detail_json + "\n```" + for detail_json in detail_json_list: + if len(detail_json) > 2: + show_json(detail_json) + await msg.update() + self.user_session._post_append() + + if os.getenv('APPBUILDER_RUN_CHAINLIT') == '1': + pass + else: + os.environ['APPBUILDER_RUN_CHAINLIT'] = '1' + target = sys.argv[0] + runner = CliRunner() + runner.invoke( + chainlit.cli.chainlit_run, [target, '--watch', "--port", port, "--host", host]) + + +if __name__ == "__main__": + from appbuilder.core.components.v2 import Text2Image + component = Text2Image() + agent = ChainlitRuntime( + component=component, + ) + agent.chainlit_component_debug(port=8092, tool_eval_args={}, + query_name = "query") diff --git a/python/utils/flask_deploy.py b/python/utils/flask_deploy.py new file mode 100644 index 000000000..f7c12cc65 --- /dev/null +++ b/python/utils/flask_deploy.py @@ -0,0 +1,389 @@ +import uuid +import json +import copy +from typing import Optional, Union, Any + +import appbuilder +from appbuilder.core.component import Component +from appbuilder.core.message import Message +from appbuilder.core.user_session import UserSession +from appbuilder.utils.logger_util import logger +from appbuilder.core.context import init_context + + +# 流式场景首包超时时，最大重试次数 +MAX_RETRY_COUNT = 3 + +class FlaskRuntime(object): + r""" + FlaskRuntime 是对应用调用的服务化封装, 开发者不是必须要用 FlaskRuntime 才能运行自己的组件服务。 + FlaskRuntime提供chat直接对话接口; 也支持使用 Gunicorn 这样的 WSGI 服务器来运行服务, 提供API部署方式。 + + + Examples: + # 直接调用chat接口进行对话 + .. code-block:: python + + import os + import sys + import appbuilder + from appbuilder.utils.flask_deploy import FlaskRuntime + os.environ["APPBUILDER_TOKEN"] = '...' + + component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" + ) + agent = FlaskRuntime(component=component) + message = appbuilder.Message({"query": "你好"}) + print(agent.chat(message, stream=False)) + + # 启动一个flask服务, 使用curl调用flask服务 + .. code-block:: python + + import os + import sys + import appbuilder + from appbuilder.utils.flask_deploy import FlaskRuntime + os.environ["APPBUILDER_TOKEN"] = '...' + + component = appbuilder.Playground( + prompt_template="{query}", + model="eb-4" + ) + user_session_config = "sqlite:///foo.db" + agent = FlaskRuntime( + component=component, user_session_config=user_session_config) + agent.serve(debug=False, port=8091) + + .. code-block:: shell + + curl --location 'http://0.0.0.0:8091/chat' \ + --header 'Content-Type: application/json' \ + --header 'X-Appbuilder-Token: ...' \ + --data '{ + "message": "你是谁", + "stream": false + }' + + Session 数据管理 : 除去上述简单应用外，还支持 Session 数据管理，下面是一个例子 + + .. code-block:: python + + import os + import sys + from appbuilder.core.component import Component + from appbuilder import ( + FlaskRuntime, UserSession, Message, QueryRewrite, Playground, + ) + + os.environ["APPBUILDER_TOKEN"] = '...' + + class PlaygroundWithHistory(Component): + def __init__(self): + super().__init__() + self.query_rewrite = QueryRewrite(model="Qianfan-Agent-Speed-8k") + self.play = Playground( + prompt_template="{query}", + model="eb-4" + ) + + def run(self, message: Message, stream: bool=False): + user_session = UserSession() + # 获取 Session 历史数据 + history_queries = user_session.get_history("query", limit=1) + history_answers = user_session.get_history("answer", limit=1) + + if history_queries and history_answers: + history = [] + for query, answer in zip(history_queries, history_answers): + history.extend([query.content, answer.content]) + logging.info(f"history: {history}") + message = self.query_rewrite( + Message(history + [message.content]), rewrite_type="带机器人回复") + logging.info(f"message: {message}") + answer = self.play.run(message, stream) + # 保存本轮数据 + user_session.append({ + "query": message, + "answer": answer, + }) + return answer + + agent = FlaskRuntime(component=PlaygroundWithHistory()) + agent.server() + + Session 信息查看 : 查看本地user_session.db数据库内部信息，下面是一个例子 + + .. code-block:: python + + import sqlite3 + import json + + # 连接到 SQLite 数据库 + # 如果文件不存在，会自动在当前目录创建: + user_session_path = 'your_user_session.db地址' + conn = sqlite3.connect(user_session_path) + cursor = conn.cursor() + + # 执行一条 SQL 语句，列出所有表 + cursor.execute("SELECT name FROM sqlite_master WHERE type='table';") + print(cursor.fetchall()) + + # 查询appbuilder_session_messages表的列信息 + cursor.execute("PRAGMA table_info(appbuilder_session_messages);") + columns_info = cursor.fetchall() + + column_names = [info[1] for info in columns_info] # info[1]是列名的位置 + for column_name in column_names: + print(column_name) + + # 查询特定表中的数据 + cursor.execute("SELECT message_value FROM appbuilder_session_messages;") + for row in cursor.fetchall(): + print(json.loads(row[0])) + + # 关闭 Connection: + conn.close() + """ + def __init__(self, + component: Component, + user_session_config: Optional[Union[Any, str]] = None, + user_session: Optional[UserSession] = None + ): + """init + + Args: + component (Component): 可运行的 Component, 需要实现 run(message, stream, args) 方法 + user_session_config (sqlalchemy.engine.URL|str|None): Session 输出存储配置字符串。默认使用 sqlite:///user_session.db + 遵循 sqlalchemy 后端定义，参考文档：https://docs.sqlalchemy.org/en/20/core/engines.html#backend-specific-urls + user_session (UserSession): 用户会话管理器，如果不指定则自动生成一个默认的 UserSession + """ + try: + from flask import Flask + + except ImportError: + raise ImportError("Flask module is not installed. Please install it using 'pip install " + "flask~=2.3.2 flask-restful==0.3.9'.") + self.app = Flask(__name__) + self.app.json.ensure_ascii = False + self.component = component + if user_session is None: + if user_session_config is None: + self.user_session = UserSession() + logger.info("init user_session with default UserSession") + else: + self.user_session = UserSession(user_session_config) + logger.info("init user_session with user_session_config") + else: + self.user_session = user_session + + def chat(self, message: Message, stream: bool = False, **args) -> Message: + """ + 执行一次对话 + + Args: + message (Message): 该次对话用户输入的 Message + stream (bool): 是否流式请求 + **args: 其他参数，会被透传到 component + + Returns: + Message(Message): 返回的 Message + """ + return self.component.run(message=message, stream=stream, **args) + + def create_flask_app(self, url_rule="/chat"): + """ + 创建 Flask 应用，主要用于 Gunicorn 这样的 WSGI 服务器来运行服务。 + + Args: + None + + Returns: + Flask + """ + try: + from flask import request, Response + from werkzeug.exceptions import BadRequest + from flask import stream_with_context + + except ImportError: + raise ImportError("Flask module is not installed. Please install it using 'pip install " + "flask~=2.3.2 flask-restful==0.3.9'.") + @self.app.errorhandler(BadRequest) + def handle_bad_request(e): + return {"code": 400, "message": f'{e}', "result": None}, 400 + + @self.app.errorhandler(Exception) + def handle_bad_request(e): + if hasattr(e, "code"): + return {"code": e.code, "message": str(e), "result": None}, 200 + else: + return {"code": 500, "message": "Internal Server Error", "result": None}, 200 + + def warp(): + """ + 根据component的lazy_certification属性处理请求。 + + Args: + 无参数。 + + Returns: + 如果stream为True，则返回流式响应（Content-Type为text/event-stream）。 + 如果stream为False，则返回包含处理结果的字典。 + + Raises: + BadRequest: 当请求头中缺少必要的X-Appbuilder-Authorization时抛出。 + BadRequest: 当请求体中缺少必要的message字段时抛出。 + BadRequest: 当请求体中session_id字段不是字符串类型时抛出。 + BadRequest: 当请求体中stream字段不是布尔类型时抛出。 + + """ + # 根据component是否lazy_certification，分成两种情况： + # 1. lazy_certification为True，初始化时未被认证，每次请求都需要带入AppbuilderToken + # 2. lazy_certification为False，初始化时已经认证，请求时不需要带入AppbuilderToken，并且带入也无效 + if self.component.lazy_certification: + app_builder_token = None + for key in ["X-Appbuilder-Token", "X-Appbuilder-Authorization"]: + if key in request.headers: + app_builder_token = request.headers[key] + break + if not app_builder_token: + raise BadRequest( + "X-Appbuilder-Authorization is required in Headers") + try: + self.component.set_secret_key_and_gateway( + secret_key=app_builder_token) + except appbuilder.core._exception.BaseRPCException as e: + logger.error(f"failed to verify. err={e}", exc_info=True) + raise BadRequest("X-Appbuilder-Authorization invalid") + except Exception as e: + logger.error(f"failed to verify. err={e}", exc_info=True) + raise e + else: + pass + + data = request.get_json() + if "message" not in data: + raise BadRequest("message is required") + message = Message(data.pop('message')) + if "session_id" not in data: + session_id = str(uuid.uuid4()) + else: + session_id = data.pop("session_id") + if not isinstance(session_id, str): + raise BadRequest("session_id must be str type") + if "stream" not in data: + stream = False + else: + stream = data.pop("stream") + if not isinstance(stream, bool): + raise BadRequest("stream must be bool type") + request_id = request.headers.get("X-Appbuilder-Request-Id", str(uuid.uuid4())) + user_id = request.headers.get("X-Appbuilder-User-Id", None) + + init_context(session_id=session_id, request_id=request_id, user_id=user_id) + logger.info( + f"request_id={request_id}, session_id={session_id}] message={message}," + f" stream={stream}, data={data}, start run...") + + def gen_sse_resp(): + with self.app.app_context(): + received_first_packet = False + retry_count = 0 + while retry_count < MAX_RETRY_COUNT: + try: + answer = self.chat(message, stream, **data) + except Exception as e: # 调用chat方法报错，直接返回 + code = 500 if not hasattr(e, "code") else e.code + err_resp = {"code": code, "message": "InternalServerError", "result": None} + logger.error( + f"request_id={request_id}, session_id={session_id}, err={e}, execute self.chat failed", exc_info=True) + yield "data: " + json.dumps(err_resp, ensure_ascii=False) + "\n\n" + return + else: # 调用chat方法成功，开始生成流式事件 + content_iterator = iter(answer.content) + answer.content = None + result = None + try: + for sub_content in content_iterator: + result = copy.deepcopy(answer) + result.content = sub_content + yield "data: " + json.dumps({ + "code": 0, "message": "", + "result": { + "session_id": session_id, + "is_completion": False, + "answer_message": json.loads(result.json(exclude_none=True)) + } + }, ensure_ascii=False) + "\n\n" + received_first_packet = True + except Exception as e: + retry_count += 1 + logger.error( + f"[request_id={request_id}, session_id={session_id}] err={e}, " + f"retry_count={retry_count}", exc_info=True) + # 如果未收到首包且重试次数小于最大重试次数，则尝试重新执行一次chat方法 + if not received_first_packet and retry_count < MAX_RETRY_COUNT: + continue + else: # 其它情况返回 + logger.error( + f"[request_id={request_id}, session_id={session_id}] err={e}, " + f"retry_count={retry_count}, received_first_packet={received_first_packet}" + , exc_info=True) + code = 500 if not hasattr(e, "code") else e.code + err_resp = {"code": code, "message": "InternalServerError", "result": None} + yield "data: " + json.dumps(err_resp, ensure_ascii=False) + "\n\n" + return + result.content = "" + yield "data: " + json.dumps({ + "code": 0, "message": "", + "result": { + "session_id": session_id, + "is_completion": True, + "answer_message": json.loads(result.json(exclude_none=True)) + } + }, ensure_ascii=False) + "\n\n" + logger.info( + f"request_id={request_id}, session_id={session_id}]" + f"retry_count={retry_count}, success response", exc_info=True) + self.user_session._post_append() + return # 正常返回 + + if stream: # 流式 + return Response(stream_with_context(gen_sse_resp()), 200, + {'Content-Type': 'text/event-stream; charset=utf-8'}) + if not stream: # 非流式 + try: + answer = self.chat(message, stream, **data) + blocking_result = json.loads(copy.deepcopy(answer).json(exclude_none=True)) + logger.debug(f"[request_id={request_id}, session_id={session_id}] blocking_result={blocking_result}") + self.user_session._post_append() + return { + "code": 0, "message": "", + "result": {"session_id": session_id, "answer_message": blocking_result} + } + except Exception as e: + logger.error( + f"[request_id={request_id}, session_id={session_id}] err={e}", exc_info=True) + code = 500 if not hasattr(e, "code") else e.code + return {"code": code, "message": "InternalServerError", "result": None} + + self.app.add_url_rule(url_rule, 'chat', warp, methods=['POST']) + return self.app + + def serve(self, host='0.0.0.0', debug=True, port=8092, url_rule="/chat"): + """ + 将 component 服务化，提供 Flask http API 接口 + + Args: + host (str): 服务运行的host地址，默认为'0.0.0.0' + debug (bool): 是否开启debug模式，默认为True + port (int): 服务运行的端口号，默认为8092 + url_rule (str): 服务的URL规则，默认为"/chat" + + Returns: + None + """ + self.app = self.create_flask_app(url_rule=url_rule) + self.app.run(host=host, debug=debug, port=port) \ No newline at end of file