CLI Tool（bioctl）

Bio-OS provides the bioctl command-line tool for communication with the Bio-OS control plane.

Installation

Step1 - Download Bioctl

Download the Bio-OS release package from GitHub (https://github.com/Bio-OS/bioos/releases) and extract it locally.

Step2 - Bioctl Config

1. Move the bioctl binary to the executable directory, which is typically /usr/local/bin on macOS or Linux.

2. Create a .bioctl folder in the User directory, which on macOS is typically located at /Users/username.

# Create the ./bioctl directory
mkdir /Users/username/.bioctl

1. In the .bioctl folder, create a bioctl.yaml file and copy the following code into the YAML file.

client:
  serverAddr: xx.xx.xxx.xxx/apiserver
  serverName: localhost
  insecure: true
  username: admin
  password: admin
  method: http
  timeout: 10
```

1. In the bioctl.yaml file, change the serverAddr IP address (xx.xx.xxx.xxx) to the server address of your deployed Bio-OS.

2. After completing the above steps, you can run bioctl in the command line and view the help information.

Usage

Bioctl supports two usage modes: interactive and command-line.

• Interactive: When you enter certain command groups (e.g., bioctl, bioctl workspace, bioctl workflow, bioctl submission, or bioctl datamodel), Bioctl enters interactive mode and displays options to choose from.

• Command-line: You can also execute commands directly, for example:

  bioctl workspace create <workspace_name>

  to run the command immediately.

Workspace Commands

Usage: bioctl workspace [OPTIONS] COMMAND [ARGS]...
Options:
-h, --help            show this help message and exit
  
Commands:
  Create <Workspace_name>   Create workspace
    Options:
       -d, --description     
       -t, --mountType      
       -p，--mountPATH       
       
       
  import                     Import workspace
    Options:
       -y, --yaml PATH     
       -t, --mountType       
       -p，--mountPATH        

  delete  <Workspace_name>    Delete workspace
    Options:


  list                        List orkspace
    Options:
      -p, --page              
      -s, --size             
      --order-by           
      --search-word           
      --ids                    

  
  Update  <Workspace_name>    Update workspace
      Options:
       -n，--name         
       -d, --description  

Import Workspace

bioctl workspace import -y workspacexxx.yaml

workspacexxx.yaml demo

In the YAML file, the path values for datamodel, workflow, and notebook must be specified relative to the location of the YAML file. An example of folder organization is shown below:

|--workspacexxx.yaml

|--data

|--sample.csv

|--sample_sheet.csv

|--workspace_data.csv

|--workflow

|--Cram-to-Bam

|--CramToBam.wdl

|--notebook

|--__dashboard__.ipynb

|--First-Notebooks.ipynb

name: Workspacexxx
version: "1"
description: Demo
dataModels:
- name: sample
  type: entity
  path: data/sample.csv
- name: sample_set
  type: entitySet
  path: data/sample_set.csv
- name: workspace_data
  type: workspace
  path: data/workspace_data.csv
workflows:
- name: Cram-to-Bam
  language: WDL
  version: "1.0"
  mainWorkflowPath: CramToBam.wdl
  path: workflow/Cram-to-Bam
  metadata:
    scheme: https
    repo: gitee.com/joy_lee/seq-format-conversion01
    tag: v0.47
- name: GATK-HaplotypeCaller
  language: WDL
  version: "1.0"
  mainWorkflowPath: Hello-GATK.wdl
  path: workflow/GATK-HaplotypeCaller
  metadata:
    scheme: https
    repo: gitee.com/joy_lee/gatk-pipeline
    tag: v0.34
notebooks:
  image:
    name: paas-tob-qa-cn-shanghai.cr.volces.com/infcprelease/jupyterhub-datascience-notebook:miracle-1.4.1
    disPlayName: datascience
    packages: '{"R": {"DT": "0.20", "R6": "2.5.1", "fs": "1.5.0", "gh": "1.3.0", "ps":
      "1.6.0", "DBI": "1.1.1", "TTR": "0.24.2", "bit": "4.0.4", "cli": "3.1.0", "hms":
      "1.1.1", "ids": "1.0.1", "ini": "0.3.1", "lhs": "1.1.3", "rex": "1.2.0", "sys":
      "3.4", "xts": "0.12.1", "zip": "2.2.0", "zoo": "1.8-9", "MASS": "7.3-54", "Rcpp":
      "1.0.7", "base": "4.1.1", "blob": "1.2.2", "brew": "1.0-6", "brio": "1.1.2",
      "covr": "3.5.1", "curl": "4.3.2", "desc": "1.4.0", "gert": "1.4.3", "glue":
      "1.5.0", "grid": "4.1.1", "httr": "1.4.2"}}'
    basicEnv:
    - Python3.9.10
    - R 4.1.2
    - Julia 1.7.2
  artifacts:
  - name: __dashboard__
    path: notebook/__dashboard__.ipynb
  - name: First-Notebooks
    path: notebook/First-Notebooks.ipynb

Create Workspace

bioctl workspace create <workspace_name> -d descriptionxxxx  -t  nfs  -p /path

Delete Workspace

bioctl workspace delete <workspace_name>

List Workspace

$ [admin@bioos]bioctl workspace list --order-by time

WorkspaceName   WorkspaceID    CreateTime                        Mounttype   MountPath 
xxxxx           XXXXXX         2022-09-09 01:15:26.394 +0000


$ [admin@bioos]bioctl workspace list --search-word xxxxx

WorkspaceName   WorkspaceID    CreateTime                        Mounttype   MountPath 
xxxxx           XXXXXX         2022-09-09 01:15:26.394 +0000

Workflow Commands

Usage: bioctl workflow [OPTIONS] COMMAND [ARGS]...

Commands:
  create <workflow_name>     
      -w, --workspace        
      -d, --description    
      --language    
      --url          
      -d, --description 
      --tag               
      --token             
      -p, --main-workflow-path  
   
  import    以文件形式导入workspace的工作流
    Options:                
        -f, --file    
        -w, --workspace  

  delete <worflow_name>   
    Options:
        -w, --workspace  
        
  list       
    Options:
        -w, --workspace    
        -p, --page       
        -s, --size     
        --order-by    
        --search-words   
        --ids            
        
  update  <worflow_name> 
      Options
         -w, --workspace     
         -n, --name           
         -d, --description    

Create workflow

bioctl workflow create myworkflow -w myworkspace -url https://github.com/jixinchi/example-wdl.git --tag 0.1 -p main.wdl

Import workflow

bioctl workflow import -w workspace_name -i /home/usr/workflow_git.json

List workflow

#list Workflow metadata 
$ bioctl workflow list <workflow_name> 
--------------
workflow : workflow_name
--------------
WorkflowID:
description:
    xxxxxxxxxx
Create time: 2022-09-09 01:15:26.394 +0000
Git Address:

$ bioctl workflow list -w workspace_name
Worflow_name     WorkflowID      Create_date                        
WorkflowA                        2022-09-09 01:15:26.394 +0000
WorkflowB                        2022-09-09 01:15:26.394 +0000

Submission Commands

Usage: bioctl submission [OPTIONS] COMMAND [ARGS]...

Commands:
  submit <workflow_name>     
    Options:
        -w, --workspace          
        -t, --type               
        -f, --file               
        -m, --datamodel          
        --data-model-rows        
        --call-caching            
        -d，--description      
        
  list   
    Options:
        -w, --workspace   
        -p, --page         
        -s, --size         
        --order-by        
        --search-word     
        --status           
        --ids              
        
  query <submission_id>    
    Options:
        -w, --workspace          
        -r, --run-id string      
        -t, --task-name string

  stop   <submission_id>
    Options:
        -w, --workspace 
        -r, --run-id string  

  delete <submission_id>
    Options:
        -w, --workspace 
        
  log <submission_id> 
     Options: 
        -w, --workspace 
        -r, --run-id string   
        -t, --task-name string 
        
  output <submission_id> 
     Options: 
        -w, --workspace       
        -r, --run-id string 

Submit workflow

bioctl submission submit myworkflow -t  -f workflows/myworkflow/myworkflow.inputs.json -m sample -w demo

The following is the JSON format for input and output files

# input/Output json format
{   
    "inputsTemplate": {
        "test.name1": "this.id-1",
        "test.name2": "this.1_id",
        "test.request": "this.id-2"
    },
    "outputsTemplate": {
        
    }
}

Terminate workflow

## Terminate a submission 
bioctl submission stop --submission_id xxx -w workspaceidxxx
## Terminate a workflow run
bioctl submission stop --submission_id xxx --run_id xxx workspaceidxxx

Query submission status

bioctl submission query <submission_id> -w workspaceidxxx

+----------+--------------------------+--------------------------------------+------------+
| RUN ID                                   |       CREATE TIME        | RUN STATUS  |
+----------+--------------------------+--------------------------------------+------------+
| 87546541-f41d-4745-9d1e-4b9b0f27b033     | 2020-10-21T06:34:10.859Z |  Running    |
| cc195aee-41c7-44b0-89e5-4e94ba47f56b     | 2020-10-21T05:52:28.360Z |  Succeeded  |
| 7209675e-4277-4d68-8848-725abca6b143     | 2020-10-21T05:52:08.340Z |  Succeeded  |
| 05ed1979-39d4-41db-8535-446919088a2b     | 2020-10-21T03:50:01.010Z |  Succeeded  |
| b2394c34-9086-441f-af49-02735b5710ed     | 2020-10-21T03:45:40.783Z |  Succeeded  |
+----------+--------------------------+--------------------------------------+------------+

Query submission outputs

bioctl submission output <submission_id> 

The output follows this rule: if the value is a string, the content is displayed directly; if it is a file, the corresponding file path is provided.

[
        {
                "Path": "id",
                "Value": "50f65c11-acc3-4d8a-8f66-fab1f136f17c"
        },
        {
                "Path": "outputs.hello.out",
                "Value": "S3://....../...../"
        }
]

Datamodel Commands

Usage: bioctl datamodel [OPTIONS] COMMAND [ARGS]...

Commands:
  import                    import datamodels
    Options:
        -w, --workspace
        -i, --inputsFile 

  delete <datamodel_name>   delete datamodels
    Options:
        -w, --workspace 

  list                      list datamodels in the workspace
    Options:
        -w, --workspace
        -n, --name
        -t, --types