除了界面化的使用之外，本项目也提供了丰富的命令行工具。 可以实现模型部署，服务部署与启停，知识库操作(查看、删除、文档加载)，调试与问题定位等多种能力。

下面对相关的命令行工具进行一个系统的使用介绍。

前置准备

在使用dbgpt命令之前，首先需要完成项目的安装。 详细的安装教程可以参考: 源码安装

使用

命令行提供了多种能力，我们可以通过如下命令查看， 如图所示，我们即可看到dbgpt的命令列表，分别有install、knowledge、model、start、stop、trace

  1. ~ dbgpt --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt [OPTIONS] COMMAND [ARGS]...
  5. Options:
  6.   --log-level TEXT  Log level
  7.   --version         Show the version and exit.
  8.   --help            Show this message and exit.
  10. Commands:
  11.   install    Install dependencies, plugins, etc.
  12.   knowledge  Knowledge command line tool
  13.   model      Clients that manage model serving
  14.   start      Start specific server.
  15.   stop       Stop specific server.
  16.   trace      Analyze and visualize trace spans.

安装命令

install命令提供了各种依赖包、插件的安装使用。

:::danger ⚠️ 注: 当前插件重构中，相关功能下个版本开放。

:::

知识库命令

knowledge 命令主要提供了知识库相关的操作，当前主要的命令有delete、list、load

  1. ~ dbgpt knowledge --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt knowledge [OPTIONS] COMMAND [ARGS]...
  5.   Knowledge command line tool
  7. Options:
  8.   --address TEXT  Address of the Api server(If not set, try to read from
  9.                   environment variable: API_ADDRESS).  [default:
  10.                   http://127.0.0.1:5000]
  11.   --help          Show this message and exit.
  13. Commands:
  14.   delete  Delete your knowledge space or document in space
  15.   list    List knowledge space
  16.   load    Load your local documents to DB-GPT

load

load 指知识库文档加载，可以通过load命令批量加载知识库文档。

  1. ~ dbgpt knowledge load --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt knowledge load [OPTIONS]
  5.   Load your local documents to DB-GPT
  7. Options:
  8.   --space_name TEXT         Your knowledge space name  [default: default]
  9.   --vector_store_type TEXT  Vector store type.  [default: Chroma]
  10.   --local_doc_path TEXT     Your document directory or document file path.
  11.                             [default: /Users/magic/workspace/github/eosphoros-
  12.                             ai/DB-GPT/pilot/datasets]
  13.   --skip_wrong_doc          Skip wrong document.
  14.   --overwrite               Overwrite existing document(they has same name).
  15.   --max_workers INTEGER     The maximum number of threads that can be used to
  16.                             upload document.
  17.   --pre_separator TEXT      Preseparator, this separator is used for pre-
  18.                             splitting before the document is actually split by
  19.                             the text splitter. Preseparator are not included
  20.                             in the vectorized text.
  21.   --separator TEXT          This is the document separator. Currently, only
  22.                             one separator is supported.
  23.   --chunk_size INTEGER      Maximum size of chunks to split.
  24.   --chunk_overlap INTEGER   Overlap in characters between chunks.
  25.   --help                    Show this message and exit.

命令行使用 - 图1

list

list 命令主要是展示知识库相关的信息。 如展示知识空间、文档内容、Chunk内容等。

  1. ~ dbgpt knowledge list --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt knowledge list [OPTIONS]
  5.   List knowledge space
  7. Options:
  8.   --space_name TEXT               Your knowledge space name. If None, list all
  9.                                   spaces
  10.   --doc_id INTEGER                Your document id in knowledge space. If Not
  11.                                   None, list all chunks in current document
  12.   --page INTEGER                  The page for every query  [default: 1]
  13.   --page_size INTEGER             The page size for every query  [default: 20]
  14.   --show_content                  Query the document content of chunks
  15.   --output [text|html|csv|latex|json]
  16.                                   The output format
  17.   --help                          Show this message and exit.

delete

删除命令支持知识库与文档的删除。可以通过dbgpt knowledge delete --help 查看相关的命令细节

  1. ~ dbgpt knowledge delete --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt knowledge delete [OPTIONS]
  5.   Delete your knowledge space or document in space
  7. Options:
  8.   --space_name TEXT  Your knowledge space name  [default: default]
  9.   --doc_name TEXT    The document name you want to delete. If doc_name is
  10.                      None, this command will delete the whole space.
  11.   -y                 Confirm your choice
  12.   --help             Show this message and exit.

命令行使用 - 图2

模型命令

model相关命令主要在多模型部署时使用，关于模型集群部署可以查看集群部署模式

  1. ~ dbgpt model --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model [OPTIONS] COMMAND [ARGS]...
  5.   Clients that manage model serving
  7. Options:
  8.   --address TEXT  Address of the Model Controller to connect to. Just support
  9.                   light deploy model, If the environment variable
  10.                   CONTROLLER_ADDRESS is configured, read from the environment
  11.                   variable
  12.   --help          Show this message and exit.
  14. Commands:
  15.   chat     Interact with your bot from the command line
  16.   list     List model instances
  17.   restart  Restart model instances
  18.   start    Start model instances
  19.   stop     Stop model instances

chat

通过chat命令可以实现与模型在命令行窗口对话。 

  1. ~ dbgpt model chat --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model chat [OPTIONS]
  5.   Interact with your bot from the command line
  7. Options:
  8.   --model_name TEXT  The name of model  [required]
  9.   --system TEXT      System prompt
  10.   --help             Show this message and exit.

list

通过list命令可以查看当前模型列表

  1. ~ dbgpt model list --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model list [OPTIONS]
  5.   List model instances
  7. Options:
  8.   --model_name TEXT  The name of model
  9.   --model_type TEXT  The type of model
  10.   --help             Show this message and exit.

restart

通过restart命令可以重启模型

  1. ~ dbgpt model restart  --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model restart [OPTIONS]
  5.   Restart model instances
  7. Options:
  8.   --model_name TEXT  The name of model  [required]
  9.   --model_type TEXT  The type of model
  10.   --help             Show this message and exit.

start

start命令主要负责模型启动

  1. ~ dbgpt model start  --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model start [OPTIONS]
  5.   Start model instances
  7. Options:
  8.   --model_name TEXT           The model name to deploy  [required]
  9.   --model_path TEXT           The model path to deploy
  10.   --host TEXT                 The remote host to deploy model  [default:
  11.                               30.183.153.197]
  12.   --port INTEGER              The remote port to deploy model  [default: 5000]
  13.   --worker_type TEXT          Worker type  [default: llm]
  14.   --device TEXT               Device to run model. If None, the device is
  15.                               automatically determined
  16.   --model_type TEXT           Model type: huggingface, llama.cpp, proxy and
  17.                               vllm  [default: huggingface]
  18.   --prompt_template TEXT      Prompt template. If None, the prompt template is
  19.                               automatically determined from model path,
  20.                               supported template: zero_shot,vicuna_v1.1,llama-
  21.                               2,codellama,alpaca,baichuan-chat,internlm-chat
  22.   --max_context_size INTEGER  Maximum context size  [default: 4096]
  23.   --num_gpus INTEGER          The number of gpus you expect to use, if it is
  24.                               empty, use all of them as much as possible
  25.   --max_gpu_memory TEXT       The maximum memory limit of each GPU, only valid
  26.                               in multi-GPU configuration
  27.   --cpu_offloading            CPU offloading
  28.   --load_8bit                 8-bit quantization
  29.   --load_4bit                 4-bit quantization
  30.   --quant_type TEXT           Quantization datatypes, `fp4` (four bit float)
  31.                               and `nf4` (normal four bit float), only valid
  32.                               when load_4bit=True  [default: nf4]
  33.   --use_double_quant          Nested quantization, only valid when
  34.                               load_4bit=True  [default: True]
  35.   --compute_dtype TEXT        Model compute type
  36.   --trust_remote_code         Trust remote code  [default: True]
  37.   --verbose                   Show verbose output.
  38.   --help                      Show this message and exit.

stop

stop命令主要负责模型停止

  1. ~ dbgpt model stop  --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt model stop [OPTIONS]
  5.   Stop model instances
  7. Options:
  8.   --model_name TEXT  The name of model  [required]
  9.   --model_type TEXT  The type of model
  10.   --host TEXT        The remote host to stop model  [required]
  11.   --port INTEGER     The remote port to stop model  [required]
  12.   --help             Show this message and exit.

命令行使用 - 图3

启动/停止命令

dbgpt start 与 dbgpt stop相关命令是一组服务注册发现相关的接口。 分别有 apiserver、controller、worker 以及webserver

  1. ~ dbgpt start --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt start [OPTIONS] COMMAND [ARGS]...
  5.   Start specific server.
  7. Options:
  8.   --help  Show this message and exit.
  10. Commands:
  11.   apiserver   Start apiserver
  12.   controller  Start model controller
  13.   webserver   Start webserver(dbgpt_server.py)
  14.   worker      Start model worker

apiserver

通过 dbgpt start apiserver 可以启动模型的api服务，默认启动端口8100

  1. ~ dbgpt start apiserver --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt start apiserver [OPTIONS]
  5.   Start apiserver
  7. Options:
  8.   --host TEXT                Model API server deploy host  [default: 0.0.0.0]
  9.   --port INTEGER             Model API server deploy port  [default: 8100]
  10.   --daemon                   Run Model API server in background
  11.   --controller_addr TEXT     The Model controller address to connect
  12.                              [default: http://127.0.0.1:8000]
  13.   --api_keys TEXT            Optional list of comma separated API keys
  14.   --log_level TEXT           Logging level
  15.   --log_file TEXT            The filename to store log  [default:
  16.                              dbgpt_model_apiserver.log]
  17.   --tracer_file TEXT         The filename to store tracer span records
  18.                              [default: dbgpt_model_apiserver_tracer.jsonl]
  19.   --tracer_storage_cls TEXT  The storage class to storage tracer span records
  20.   --help                     Show this message and exit.
  23. # 启动apiserver
  24. ~ dbgpt start apiserver
  25. Already connect 'dbgpt'
  26. 2023-12-07 14:35:21 B-4TMH9N3X-2120.local pilot.component[95201] INFO Register component with name dbgpt_model_registry and instance: <pilot.model.cluster.controller.controller.ModelRegistryClient object at 0x28f4e0c70>
  27. 2023-12-07 14:35:21 B-4TMH9N3X-2120.local pilot.component[95201] INFO Register component with name dbgpt_worker_manager_factory and instance: <pilot.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x28f4e2110>
  28. 2023-12-07 14:35:21 B-4TMH9N3X-2120.local pilot.component[95201] INFO Register component with name dbgpt_model_api_server and instance: <pilot.model.cluster.apiserver.api.APIServer object at 0x28f4e2170>
  29. INFO:     Started server process [95201]
  30. INFO:     Waiting for application startup.
  31. INFO:     Application startup complete.
  32. INFO:     Uvicorn running on http://0.0.0.0:8100 (Press CTRL+C to quit)
  33. INFO:     127.0.0.1:56638 - "GET /docs HTTP/1.1" 200 OK
  34. INFO:     127.0.0.1:56665 - "GET /openapi.json HTTP/1.1" 200 OK
  35. ^CINFO:     Shutting down
  36. INFO:     Waiting for application shutdown.
  37. INFO:     Application shutdown complete.
  38. INFO:     Finished server process [95201]

controller

通过dbgpt start controller可以启动管控服务，默认启动端口8000

  1. ~ dbgpt start --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt start [OPTIONS] COMMAND [ARGS]...
  5.   Start specific server.
  7. Options:
  8.   --help  Show this message and exit.
  10. Commands:
  11.   apiserver   Start apiserver
  12.   controller  Start model controller
  13.   webserver   Start webserver(dbgpt_server.py)
  14.   worker      Start model worker
  15. (dbgpt_env) magic@B-4TMH9N3X-2120 ~ % dbgpt start controller
  16. Already connect 'dbgpt'
  17. INFO:     Started server process [96797]
  18. INFO:     Waiting for application startup.
  19. INFO:     Application startup complete.
  20. INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

webserver

通过dbgpt start webserver 可以启动前端服务，默认端口 5000, 通过 http://127.0.0.1:5000进行访问

  1. ~ dbgpt start webserver --help
  2. Already connect 'dbgpt'
  3. Usage: dbgpt start webserver [OPTIONS]
  5.   Start webserver(dbgpt_server.py)
  7. Options:
  8.   --host TEXT                Webserver deploy host  [default: 0.0.0.0]
  9.   --port INTEGER             Webserver deploy port  [default: 5000]
  10.   --daemon                   Run Webserver in background
  11.   --controller_addr TEXT     The Model controller address to connect. If None,
  12.                              read model controller address from environment
  13.                              key `MODEL_SERVER`.
  14.   --model_name TEXT          The default model name to use. If None, read
  15.                              model name from environment key `LLM_MODEL`.
  16.   --share                    Whether to create a publicly shareable link for
  17.                              the interface. Creates an SSH tunnel to make your
  18.                              UI accessible from anywhere.
  19.   --remote_embedding         Whether to enable remote embedding models. If it
  20.                              is True, you need to start a embedding model
  21.                              through `dbgpt start worker --worker_type
  22.                              text2vec --model_name xxx --model_path xxx`
  23.   --log_level TEXT           Logging level
  24.   --light                    enable light mode
  25.   --log_file TEXT            The filename to store log  [default:
  26.                              dbgpt_webserver.log]
  27.   --tracer_file TEXT         The filename to store tracer span records
  28.                              [default: dbgpt_webserver_tracer.jsonl]
  29.   --tracer_storage_cls TEXT  The storage class to storage tracer span records
  30.   --disable_alembic_upgrade  Whether to disable alembic to initialize and
  31.                              upgrade database metadata
  32.   --help                     Show this message and exit.

命令行使用 - 图4

worker

dbgpt start worker主要用于启动工作模型，详细的使用可见: 集群部署

调试(Debugging)

dbgpt 项目提供了丰富的debug命令，详细的使用参见: 调试(Debugging)