安装命令行工具

以下所有操作都是通过dbgpt命令来完成。 要使用dbgpt命令，首先需要安装DB-GPT项目, 可以通过如下命令安装

  1. $ pip install -e ".[default]"

同时也可以通过脚本模式来使用

  1. $ python dbgpt/cli/cli_scripts.py

启动Model Controller

  1. $ dbgpt start controller

默认情况下，Model Server会启动在8000端口

启动Model Worker

:::color2 启动chatglm2-6b模型Worker

:::

  1. dbgpt start worker --model_name chatglm2-6b \
  2. --model_path /app/models/chatglm2-6b \
  3. --port 8001 \
  4. --controller_addr http://127.0.0.1:8000

:::color2 启动vicuna-13b-v1.5模型Worker

:::

  1. dbgpt start worker --model_name vicuna-13b-v1.5 \
  2. --model_path /app/models/vicuna-13b-v1.5 \
  3. --port 8002 \
  4. --controller_addr http://127.0.0.1:8000

:::danger ⚠️ 注意：确保使用您自己的模型名称和模型路径。

:::

启动Embedding模型服务

  1. dbgpt start worker --model_name text2vec \
  2. --model_path /app/models/text2vec-large-chinese \
  3. --worker_type text2vec \
  4. --port 8003 \
  5. --controller_addr http://127.0.0.1:8000

:::danger

⚠️ 注意：确保使用您自己的模型名称和模型路径。

:::

:::success 查看并检查已部署模型

:::

  1. $ dbgpt model list
  3. +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
  4. |    Model Name   | Model Type |    Host    | Port | Healthy | Enabled | Prompt Template |       Last Heartbeat       |
  5. +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+
  6. |   chatglm2-6b   |    llm     | 172.17.0.2 | 8001 |   True  |   True  |                 | 2023-09-12T23:04:31.287654 |
  7. |  WorkerManager  |  service   | 172.17.0.2 | 8001 |   True  |   True  |                 | 2023-09-12T23:04:31.286668 |
  8. |  WorkerManager  |  service   | 172.17.0.2 | 8003 |   True  |   True  |                 | 2023-09-12T23:04:29.845617 |
  9. |  WorkerManager  |  service   | 172.17.0.2 | 8002 |   True  |   True  |                 | 2023-09-12T23:04:24.598439 |
  10. |     text2vec    |  text2vec  | 172.17.0.2 | 8003 |   True  |   True  |                 | 2023-09-12T23:04:29.844796 |
  11. | vicuna-13b-v1.5 |    llm     | 172.17.0.2 | 8002 |   True  |   True  |                 | 2023-09-12T23:04:24.597775 |
  12. +-----------------+------------+------------+------+---------+---------+-----------------+----------------------------+

使用模型服务

如上部署好的模型服务，可以通过dbgpt_server来使用。首先修改.env配置文件来更改连接模型地址。

  1. LLM_MODEL=vicuna-13b-v1.5
  2. # The current default MODEL_SERVER address is the address of the Model Controller
  3. MODEL_SERVER=http://127.0.0.1:8000

启动Webserver

  1. dbgpt start webserver --light

—light 表示不启动嵌入式模型服务。

或者直接可以通过命令制定模型的方式启动。 
  1. LLM_MODEL=chatglm2-6b dbgpt start webserver --light

使用示例

这是一个通过集群方式部署Embedding模型并使用的完整命令行样例程序。

  1. 第一步： 启动Controller Server
  1. dbgpt start controller
  1. 第二步: 启动向量模型Worker
  1. # 2. second start embedding model worker
  2. dbgpt start worker --model_name text2vec \
  3. --model_path /app/models/text2vec-large-chinese \
  4. --worker_type text2vec \
  5. --port 8003 \
  6. --controller_addr http://127.0.0.1:8000
  1. 第三步：启动apiserver
  1. dbgpt start apiserver --controller_addr http://127.0.0.1:8000 --api_keys EMPTY
  1. 第四步: 测试使用
  1. curl http://127.0.0.1:8100/api/v1/embeddings \
  2. -H "Authorization: Bearer EMPTY" \
  3. -H "Content-Type: application/json" \
  4. -d '{
  5.     "model": "text2vec",
  6.     "input": "Hello world!"
  7. }'
  1. 通过包的方式代码引入使用
  1.  from dbgpt.rag.embedding import OpenAPIEmbeddings
  3.   openai_embeddings = OpenAPIEmbeddings(
  4.       api_url="http://localhost:8100/api/v1/embeddings",
  5.       api_key="EMPTY",
  6.       model_name="text2vec",
  7.   )
  8.   texts = ["Hello, world!", "How are you?"]
  9.   openai_embeddings.embed_documents(texts)

命令行用法

更多关于命令行的使用，可以通过查看命令行帮助进行使用, 如下是一个参考样例

:::color1 查看dbgpt帮助 dbgpt —help

:::

  1. dbgpt --help
  3. Already connect 'dbgpt'
  4. Usage: dbgpt [OPTIONS] COMMAND [ARGS]...
  6. Options:
  7.   --log-level TEXT  Log level
  8.   --version         Show the version and exit.
  9.   --help            Show this message and exit.
  11. Commands:
  12.   install    Install dependencies, plugins, etc.
  13.   knowledge  Knowledge command line tool
  14.   model      Clients that manage model serving
  15.   start      Start specific server.
  16.   stop       Start specific server.
  17.   trace      Analyze and visualize trace spans.

:::color1 查看dbgpt 启动命令 dbgpt start —help

:::

  1. dbgpt start --help
  3. Already connect 'dbgpt'
  4. Usage: dbgpt start [OPTIONS] COMMAND [ARGS]...
  6.   Start specific server.
  8. Options:
  9.   --help  Show this message and exit.
  11. Commands:
  12.   apiserver   Start apiserver
  13.   controller  Start model controller
  14.   webserver   Start webserver(dbgpt_server.py)
  15.   worker      Start model worker
  16. (dbgpt_env) magic@B-4TMH9N3X-2120 ~ %

:::color1 查看dbgpt 启动模型服务帮助命令 dbgpt start worker —help

:::

  1. dbgpt start worker --help
  3. Already connect 'dbgpt'
  4. Usage: dbgpt start worker [OPTIONS]
  6.   Start model worker
  8. Options:
  9.   --model_name TEXT               Model name  [required]
  10.   --model_path TEXT               Model path  [required]
  11.   --worker_type TEXT              Worker type
  12.   --worker_class TEXT             Model worker class,
  13.                                   pilot.model.cluster.DefaultModelWorker
  14.   --model_type TEXT               Model type: huggingface, llama.cpp, proxy
  15.                                   and vllm  [default: huggingface]
  16.   --host TEXT                     Model worker deploy host  [default: 0.0.0.0]
  17.   --port INTEGER                  Model worker deploy port  [default: 8001]
  18.   --daemon                        Run Model Worker in background
  19.   --limit_model_concurrency INTEGER
  20.                                   Model concurrency limit  [default: 5]
  21.   --standalone                    Standalone mode. If True, embedded Run
  22.                                   ModelController
  23.   --register                      Register current worker to model controller
  24.                                   [default: True]
  25.   --worker_register_host TEXT     The ip address of current worker to register
  26.                                   to ModelController. If None, the address is
  27.                                   automatically determined
  28.   --controller_addr TEXT          The Model controller address to register
  29.   --send_heartbeat                Send heartbeat to model controller
  30.                                   [default: True]
  31.   --heartbeat_interval INTEGER    The interval for sending heartbeats
  32.                                   (seconds)  [default: 20]
  33.   --log_level TEXT                Logging level
  34.   --log_file TEXT                 The filename to store log  [default:
  35.                                   dbgpt_model_worker_manager.log]
  36.   --tracer_file TEXT              The filename to store tracer span records
  37.                                   [default:
  38.                                   dbgpt_model_worker_manager_tracer.jsonl]
  39.   --tracer_storage_cls TEXT       The storage class to storage tracer span
  40.                                   records
  41.   --device TEXT                   Device to run model. If None, the device is
  42.                                   automatically determined
  43.   --prompt_template TEXT          Prompt template. If None, the prompt
  44.                                   template is automatically determined from
  45.                                   model path, supported template: zero_shot,vi
  46.                                   cuna_v1.1,llama-2,codellama,alpaca,baichuan-
  47.                                   chat,internlm-chat
  48.   --max_context_size INTEGER      Maximum context size  [default: 4096]
  49.   --num_gpus INTEGER              The number of gpus you expect to use, if it
  50.                                   is empty, use all of them as much as
  51.                                   possible
  52.   --max_gpu_memory TEXT           The maximum memory limit of each GPU, only
  53.                                   valid in multi-GPU configuration
  54.   --cpu_offloading                CPU offloading
  55.   --load_8bit                     8-bit quantization
  56.   --load_4bit                     4-bit quantization
  57.   --quant_type TEXT               Quantization datatypes, `fp4` (four bit
  58.                                   float) and `nf4` (normal four bit float),
  59.                                   only valid when load_4bit=True  [default:
  60.                                   nf4]
  61.   --use_double_quant              Nested quantization, only valid when
  62.                                   load_4bit=True  [default: True]
  63.   --compute_dtype TEXT            Model compute type
  64.   --trust_remote_code             Trust remote code  [default: True]
  65.   --verbose                       Show verbose output.
  66.   --help                          Show this message and exit.

:::color1 查看dbgpt模型服务相关命令 dbgpt model —help

:::

  1. dbgpt model --help
  4. Already connect 'dbgpt'
  5. Usage: dbgpt model [OPTIONS] COMMAND [ARGS]...
  7.   Clients that manage model serving
  9. Options:
  10.   --address TEXT  Address of the Model Controller to connect to. Just support
  11.                   light deploy model, If the environment variable
  12.                   CONTROLLER_ADDRESS is configured, read from the environment
  13.                   variable
  14.   --help          Show this message and exit.
  16. Commands:
  17.   chat     Interact with your bot from the command line
  18.   list     List model instances
  19.   restart  Restart model instances
  20.   start    Start model instances
  21.   stop     Stop model instances