Containers - Handbook - Project configuration

This is a technical reference for Sphere Engine Containers project configuration. You will find here a detailed description covering:

• file structure,
• parameters (meaning, type, validation rules, description),
• examples.

The project configuration is specified by a single JSON file (config.json) of a well-defined structure.

General overview

Let's start by presenting the complete config.json configuration file based on an example. This is to demonstrate a general document structure.

{
    // configuration version
    "version": 2.1,

    // collection of run configurations
    "scenarios": {
        // first run configuration
        "scenario_1": {
            // whether to run this scenario by default
            "default": true,
            // whether to allow Internet connection
            "network": false,
            // list of services (e.g. database)
            "services": ["mysql"],
            // a sequence of consecutive Workspace execution stages
            "stages": {
                // configuration of an "init" stage
                "init": {
                    "command": null
                },
                // configuration of a "build" stage
                "build": {
                    "command": null
                },
                // configuration of a "run" stage
                "run": {
                    // a path to a script or shell command to run
                    "command": ".sphere-engine/run.sh",
                    // where should the data be redirected
                    "redirect_streams_to": "console",
                    // whether to skip this stage
                    "skip": false,
                    // how long should the stage be allowed to execute
                    "timeout": 10
                },
                // configuration of a "test" stage
                "test": {
                    "command": null
                },
                // configuration of a "post" stage
                "post": {
                    "command": null
                }
            },
            "output": {
                // type of the Workspace's output data
                "type": "file",
                // output data location
                "path": "my_output.csv",
                //  whether to present the output when the Workspace loads
                "show_at_startup": false
            },
            // configuration for tests required if you use WeightedScoreTestsGrader
            // if you don't use WeightedScoreTestsGrader the key is not used
            // please see WeightedScoreTestsGrader docs for more information
            // tests weights are specififed as selector. Each selector can match specific unit tests
            "tests_weights": [
                {
                    // optional name of the test case to match (or wildcard: "*")
                    "name": "test_add_operation",
                    // optional name of the test case classname to match (or wildcard: "*")
                    "classname": "TestClass",
                    // optional status of the test case to match (or wildcard: "*")
                    "status": "ok",
                    // score assigned by this selector
                    "weight": 20
                },
                {
                    "name": "other_test",
                    "weight": 0
                }
            ],
            // collection of directories that should be collected and archived among other scenario execution results
            "auxdata": {
                "custom_dir1": "d1",
                "custom_dir2": "d2",
                "custom_dir3": "/tmp/xyz"
            },
            // options for webhooks
            "webhooks": {
                "workspace.scenario.finished": {
                    "include_streams": ["auxdata", "tests_report", "stage_run_output"]
                }
            }
        },
        // second run configuration
        "scenario_2": {
            "default": false,
            ...
        }
    },

    // settings related to files
    "files": {
        // list of files that should be hidden in the file explorer
        "hidden": [".env", ".gitignore"],
        // list of files that should automatically open after launching the Workspace
        "open_at_startup": ["README.md"],
        // list of read-only files
        "read_only": ["config.yaml"]
    },

    // any custom data of any type
    "custom": {
        "internal_tag": "3317c1863f8dd227c2e438e6cabf466b",
        "categories": ["simple programming", "data processing", "algorithms"]
    }
}

Detailed specification

In the following section we present detailed information regarding the config.json configuration file structure.

config.json (root)

A configuration file of the Sphere Engine Containers project.

type: object
required: yes
parameters:

example:

{
  "version": 2.1,

  "scenarios": {
    "simple_run": {
      "stages": {
        "run": {
          "command": ".sphere-engine/run.sh"
        }
      }
    }
  }
}

root.version

Version of the config file.

type: number
required: yes
values: 2.1

example:

"version": 2.1

root.scenarios

A collection of scenarios that will be run by the user of the Workspace.

type: object
required: yes
parameters:

example:

{
  "my_custom_scenario_1": {
    "stages": {
      "run": {
        "command": ".sphere-engine/run1.sh"
      }
    }
  },
  "my_custom_scenario_2": {
    "stages": {
      "build": {
        "command": ".sphere-engine/build.sh"
      },
      "run": {
        "command": ".sphere-engine/run2.sh"
      }
    }
  }
}

root.scenarios._scenarioName_

A definition of a single execution scenario.

type: object
required: yes
parameters:

example:

{
  "stages": {
    "build": {
      "command": ".sphere-engine/build.sh"
    },
    "run": {
      "command": ".sphere-engine/run2.sh"
    }
  }
}

root.scenarios._scenarioName_.default

Whether this scenario should be run on Ctrl+Enter keyboard shortcut in the Workspace.

type: boolean
required: no
default: false

example:

"default": true

root.scenarios._scenarioName_.network

Should the Workspace have access to the Internet.

type: boolean
required: no
default: false

example:

"network": true

root.scenarios._scenarioName_.services

Services needed in a scenario; e.g. database

type: array[string]
required: no
default: []
available services:

example:

"services": ["mysql", "mongodb"]

root.scenarios._scenarioName_.output

Settings related to scenario output data

type: object
required: no
default: {}
parameters:

example:

{
  "type": "file",
  "path": "my_output.csv",
  "show_at_startup": true
}

root.scenarios._scenarioName_.output.type

Type of the Workspace's output data.

type: string
required: yes
values: console, file, none, tests, vnc, web

example:

"type": "web"

root.scenarios._scenarioName_.output.path

Path to the output file or url to the server.

type: string
required: no
default: _empty_

example:

"path": "my_output.csv"

root.scenarios._scenarioName_.output.show_at_startup

Whether to present the output when the Workspace loads

type: boolean
required: no
default: false

example:

"show_at_startup": true

root.scenarios._scenarioName_.stages

A sequence of consecutive Workspace execution stages

type: object
required: yes
parameters:

example:

{
  "build": {
    "command": ".sphere-engine/build.sh"
  },
  "run": {
    "command": ".sphere-engine/run.sh"
  }
}

root.scenarios._scenarioName_.stages._stageName_

Configuration of an individual stage.

stage_name: init, build, run, test, post
type: object
required: no
parameters:

example:

{
  "command": ".sphere-engine/stage_script.sh",
  "skip": false,
  "timeout": 10
}

root.scenarios._scenarioName_.stages._stageName_.command

Path to a a script or shell command to run

type: string
required: no

example:

"command": ".sphere-engine/stage_script.sh"

root.scenarios._scenarioName_.stages._stageName_.skip

Whether to skip the stage

type: string
required: no
default: false

example:

"skip": true

root.scenarios._scenarioName_.stages._stageName_.timeout

Timeout of the stage in seconds

type: string
required: no
default: 30

example:

"timeout": 90

root.scenarios._scenarioName_.stages._stageName_.redirect_streams_to

Target location for redirecting the stage's output and error streams. The default value is files for submissions created via API, and console for workspaces. To extract output or error streams from a workspace (e.g., via webhooks), use the both option.

type: string
required: no
default: console
values: console, files, both

example:

"redirect_streams_to": "files"

root.scenarios._scenarioName_.auxdata

A collection of directories that should be collected and archived among other scenario execution results. It is provided as key: value pairs, where the key is a name under which the directory pointed by the value should be stored.

For more information, please refer to a extended description of this parameter.

type: object
required: no
default: {}

example:

{
  "custom_dir1": "d1",
  "custom_dir2": "d2",
  "custom_dir3": "/tmp/xyz"
}

root.scenarios._scenarioName_.tests_weights

A collection of selectors for WeightedScoreTestsGrader that specify what scores assign to what unit tests.

You can find complete weighted unit tests example here

type: array
required: no
parameters:

example:

[
  {
    "name": "test_add_operation",
    "weight": 20
  },
  {
      "name": "test_add_operation",
      "status": "failure",
      "weight": -20
  },
  {
      "name": "test_mul_operation",
      "weight": 20
  }
]

root.scenarios._scenarioName_.tests_weights[i]

A definition of a single test case selector. You can learn more here WeightedScoreTestsGrader.

type: object
required: yes
parameters:

example:

{
    "name": "test_add_operation",
    "status": "failure",
    "classname": "*",
    "weight": -20
}

root.scenarios._scenarioName_.webhooks

The options that can modify the behavior of webhooks.

type: object
required: no
parameters:

example:

{
    "workspace.scenario.finished": {...}
}

root.scenarios._scenarioName_.webhooks.workspace.scenario.finished

The workspace.scenario.finished webhook is sent when a scenario is finished.

type: object
required: no
parameters:

available streams for include_streams parameter:

[
    "auxdata",
    "tests_report",
    "stage_init_output", "stage_init_error",
    "stage_build_output", "stage_build_error",
    "stage_run_output", "stage_run_error",
    "stage_test_output", "stage_test_error",
    "stage_post_output", "stage_post_error"
]

example:

{
    "include_streams": [
        "tests_report",
        "stage_run_output"
    ]
}

root.files

Settings related to Workspace files.

type: object
required: no
default: {}
parameters:

example:

{
  "hidden": [".env", ".gitignore"],
  "open_at_startup": ["README.md"],
  "read_only": ["config.yaml"]
}

root.files.hidden

Files that will be hidden from the file explorer.

type: array[string]
required: no
default: []

example:

"hidden": [".env", ".gitignore"]

root.files.open_at_startup

Files that will be open when the Workspace loads.

type: array[string]
required: no
default: []

example:

"open_at_startup": ["README.md"]

root.files.read_only

Files that are read only - cannot be modified in the Workspace

type: array[string]
required: no
default: []

example:

"read_only": ["config.yaml"]

root.custom

Any data of any structure.

type: object
required: no
default: {}
parameters: any

example:

{
  "internal_tag": "3317c1863f8dd227c2e438e6cabf466b",
  "categories": ["simple programming", "data processing", "algorithms"]
}