Containers
Handbook - Project configuration

This is a technical reference for Sphere Engine Containers project configuration. You will find here a detailed description covering:

  • file structure,
  • parameters (meaning, type, validation rules, description),
  • examples.

The project configuration is specified by a single JSON file (config.json) of a well-defined structure.

General overview

Let's start by presenting the complete config.json configuration file based on an example. This is to demonstrate a general document structure.

{
  // configuration version
  "version": 2.1,

  // collection of run configurations
  "scenarios": {
    // first run configuration
    "scenario_1": {
      // whether to run this scenario by default
      "default": true,
      // whether to allow Internet connection
      "network": false,
      // list of services (e.g., database)
      "services": ["mysql"],
      // a sequence of consecutive workspace execution stages
      "stages": {
        // configuration of an "init" stage
        "init": {
          "command": null
        },
        // configuration of a "build" stage
        "build": {
          "command": null
        },
        // configuration of a "run" stage
        "run": {
          // a path to a script or shell command to run
          "command": ".sphere-engine/run.sh",
          // where should the data be redirected
          "redirect_streams_to": "console",
          // whether to skip this stage
          "skip": false,
          // how long should the stage be allowed to execute
          "timeout": 10
        },
        // configuration of a "test" stage
        "test": {
          "command": null
        },
        // configuration of a "post" stage
        "post": {
          "command": null
        }
      },
      "output": {
        // type of the workspace's output data
        "type": "file",
        // output data location
        "path": "my_output.csv",
        //  whether to present the output when the workspace loads
        "show_at_startup": false
      },
      // configuration for tests required if you use WeightedScoreTestsGrader
      // if you don't use WeightedScoreTestsGrader the key is not used
      // please see WeightedScoreTestsGrader docs for more information
      // tests weights are specififed as selector. Each selector can match specific unit tests
      "tests_weights": [
        {
          // optional name of the test case to match (or wildcard: "*")
          "name": "test_add_operation",
          // optional name of the test case classname to match (or wildcard: "*")
          "classname": "TestClass",
          // optional status of the test case to match (or wildcard: "*")
          "status": "ok",
          // score assigned by this selector
          "weight": 20
        },
        {
          "name": "other_test",
          "weight": 0
        }
      ],
      // collection of directories that should be collected and archived among other scenario execution results
      "auxdata": {
        "custom_dir1": "d1",
        "custom_dir2": "d2",
        "custom_dir3": "/tmp/xyz"
      },
      // options for webhooks
      "webhooks": {
        "workspace.scenario.finished": {
          "include_streams": ["auxdata", "tests_report", "stage_run_output"]
        }
      }
    },
    // second run configuration
    "scenario_2": {
      "default": false,
      ...
    }
  },

  // settings related to files
  "files": {
    // list of files that should be hidden in the file explorer
    "hidden": [".env", ".gitignore"],
    // list of files that should automatically open after launching the workspace
    "open_at_startup": ["README.md"],
    // list of read-only files
    "read_only": ["config.yaml"]
  },

  // any custom data of any type
  "custom": {
    "internal_tag": "3317c1863f8dd227c2e438e6cabf466b",
    "categories": ["simple programming", "data processing", "algorithms"]
  }
}

Detailed specification

In the following section we present detailed information regarding the config.json configuration file structure.

config.json (root)

A configuration file of the Sphere Engine Containers project.

type: object
required: yes
parameters:

Name Type Description Required Default
version number Version of the config file Yes -
scenarios object A collection of scenarios that will be run by the user of the workspace Yes -
files object Settings related to workspace files No {}
custom object Any custom data No {}

example:

{
  "version": 2.1,

  "scenarios": {
    "simple_run": {
      "stages": {
        "run": {
          "command": ".sphere-engine/run.sh"
        }
      }
    }
  }
}

root.version

Version of the config file.

type: number
required: yes
values: 2.1

example:

"version": 2.1

root.scenarios

A collection of scenarios that will be run by the user of the workspace.

type: object
required: yes
parameters:

Name Type Description Required
_scenarioName1_ object(scenario) Definition of scenario #1 No
... ... ... ...
_scenarioName2_ object(scenario) Definition of scenario #N No

example:

{
  "my_custom_scenario_1": {
    "stages": {
      "run": {
        "command": ".sphere-engine/run1.sh"
      }
    }
  },
  "my_custom_scenario_2": {
    "stages": {
      "build": {
        "command": ".sphere-engine/build.sh"
      },
      "run": {
        "command": ".sphere-engine/run2.sh"
      }
    }
  }
}

Note: It is necessary to define at least one scenario.

Note: You can use any custom name for a key in the scenarios JSON structure.

root.scenarios._scenarioName_

A definition of a single execution scenario.

type: object
required: yes
parameters:

Name Type Description Required Default
default boolean If set to true, this scenario will be run on Ctrl+Enter keyboard shortcut in the workspace No false
network boolean Should the workspace have access to the Internet No true
services array[string] Services needed in a scenario; e.g., database No []
output object Settings related to scenario output data No {}
stages object Stages that will be run one after another in a given scenario Yes -
auxdata object Collection of directories that should be collected and archived among other scenario execution results No {}
tests_weights array[test_selector] List of test case selectors used to assign scores to test cases No []
webhooks object Options for webhooks No {}

example:

{
  "stages": {
    "build": {
      "command": ".sphere-engine/build.sh"
    },
    "run": {
      "command": ".sphere-engine/run2.sh"
    }
  }
}

Note: It is necessary to define at least one stage.

root.scenarios._scenarioName_.default

Whether this scenario should be run on Ctrl+Enter keyboard shortcut in the workspace. If no scenario has a default value set, then the first scenario found is executed.

type: boolean
required: no
default: false

example:

"default": true

root.scenarios._scenarioName_.network

Should the workspace have access to the Internet.

type: boolean
required: no
default: true

example:

"network": true

root.scenarios._scenarioName_.services

Services needed in a scenario; e.g., database.

type: array[string]
required: no
default: []
available services:

Service name Description Details
mysql MySQL database server the port is available using the environment variable $SE_NETWORK_PORT_MYSQL
mongodb MongoDB database server the port is available using the environment variable $SE_NETWORK_PORT_MONGODB
vnc VNC (Virtual Network Computing) server the port is available using the environment variable $SE_NETWORK_PORT_VNC

example:

"services": ["mysql", "mongodb"]

root.scenarios._scenarioName_.output

Settings related to scenario output data.

type: object
required: no
default: {}
parameters:

Name Type Description Required Default
type string Type of the workspace's output data Yes -
path string Path to the output file or url to the server No _empty_
show_at_startup boolean Whether to present the output when the workspace loads No false

example:

{
  "type": "file",
  "path": "my_output.csv",
  "show_at_startup": true
}

root.scenarios._scenarioName_.output.type

Type of the workspace's output data.

type: string
required: yes
values: console, file, tests, vnc, web

example:

"type": "web"

root.scenarios._scenarioName_.output.path

Path to the output file or url to the server.

type: string
required: no
default: _empty_

example:

"path": "my_output.csv"

Note: The value should contain relative or absolute path to the file or url on the server.

Note: The value can be omitted if the output type is set to tests.

root.scenarios._scenarioName_.output.show_at_startup

Whether to present the output when the workspace loads.

type: boolean
required: no
default: false

example:

"show_at_startup": true

root.scenarios._scenarioName_.stages

A sequence of consecutive workspace execution stages.

type: object
required: yes
parameters:

Name Type Description Required Default
init object(stage) Stage #1 - init No {}
build object(stage) Stage #2 - build No {}
run object(stage) Stage #3 - run No {}
test object(stage) Stage #4 - test No {}
post object(stage) Stage #5 - post No {}

example:

{
  "build": {
    "command": ".sphere-engine/build.sh"
  },
  "run": {
    "command": ".sphere-engine/run.sh"
  }
}

Note: It is necessary to define at least one stage.

root.scenarios._scenarioName_.stages._stageName_

Configuration of an individual stage.

stage_name: init, build, run, test, post
type: object
required: no
parameters:

Name Type Description Required Default
command string Path to a script or shell command to run Yes -
skip boolean Whether to skip the stage No false
timeout number Timeout of the stage in seconds No 30
redirect_streams_to string Target location for redirecting the stage's output and error streams No console

example:

{
  "command": ".sphere-engine/stage_script.sh",
  "skip": false,
  "timeout": 10
}

root.scenarios._scenarioName_.stages._stageName_.command

Path to a script or shell command to run.

type: string
required: no

example:

"command": ".sphere-engine/stage_script.sh"

Note: The value should contain relative or absolute path to the Bash script or a shell command to run.

root.scenarios._scenarioName_.stages._stageName_.skip

Whether to skip the stage.

type: string
required: no
default: false

example:

"skip": true

root.scenarios._scenarioName_.stages._stageName_.timeout

Timeout of the stage in seconds.

type: string
required: no
default: 30

example:

"timeout": 90

Note: For the value, provide an integer representing the number of seconds.

root.scenarios._scenarioName_.stages._stageName_.redirect_streams_to

Target location for redirecting the stage's output and error streams. The default value is files for submissions created via API, and console for workspaces. To extract output or error streams from a workspace (e.g., via webhooks), use the both option.

In case of redirecting streams to files (i.e., files and both values), the resulting file locations are pre-defined. For detailed information about these file locations, kindly consult the "Environment" section in the CLI toolkit document or the "Stage" section in the Python toolkit document.

type: string
required: no
default: console
values: console, files, both

example:

"redirect_streams_to": "files"

root.scenarios._scenarioName_.auxdata

A collection of directories that should be collected and archived among other scenario execution results. It is provided as key: value pairs, where the key is a name under which the directory pointed by the value should be stored.

For more information, please refer to an extended description of this parameter.

type: object
required: no
default: {}

example:

{
  "custom_dir1": "d1",
  "custom_dir2": "d2",
  "custom_dir3": "/tmp/xyz"
}

root.scenarios._scenarioName_.tests_weights

A collection of selectors for WeightedScoreTestsGrader that specify what scores assign to what unit tests.

You can find complete weighted unit tests example here.

type: array
required: no
parameters:

Name Type Description Required
[0] object(test_selector) Definition of unit test case selector #1 No
... ... ... ...
[N] object(test_selector) Definition of unit tests case selector #N No

example:

[
  {
    "name": "test_add_operation",
    "weight": 20
  },
  {
    "name": "test_add_operation",
    "status": "failure",
    "weight": -20
  },
  {
    "name": "test_mul_operation",
    "weight": 20
  }
]

root.scenarios._scenarioName_.tests_weights[i]

A definition of a single test case selector. You can learn more here WeightedScoreTestsGrader.

type: object
required: yes
parameters:

Name Type Description Required Default
name string Test case name to be matched No "*"
classname string Test case class name to be matched No "*"
status "ok" | "failure" | "error" Test case status to be matched No "ok"
weight number Score that will be assigned to a test matched by this test case selector Yes
example: 
{
  "name": "test_add_operation",
  "status": "failure",
  "classname": "*",
  "weight": -20
}

Note: It is necessary to define weight, all other properties are optional.

root.scenarios._scenarioName_.webhooks

The options that can modify the behavior of webhooks.

type: object
required: no
parameters:

Name Type Description Required
workspace.scenario.finished object sent when a scenario is finished No

example:

{
  "workspace.scenario.finished": {...}
}

root.scenarios._scenarioName_.webhooks.workspace.scenario.finished

The workspace.scenario.finished webhook is sent when a scenario is finished.

type: object
required: no
parameters:

Name Type Description Required Default
include_streams array[string] Streams to be included in the webhook No ["auxdata", "tests_report"]

available streams for include_streams parameter: 

[
  "auxdata",
  "tests_report",
  "stage_init_output", "stage_init_error",
  "stage_build_output", "stage_build_error",
  "stage_run_output", "stage_run_error",
  "stage_test_output", "stage_test_error",
  "stage_post_output", "stage_post_error"
]

Important: Adding too many streams to the include_streams can have a negative impact on webhook delivery time. It is advisable to export only the necessary files.

example:

{
  "include_streams": [
    "tests_report",
    "stage_run_output"
  ]
}

root.files

Settings related to workspace files.

type: object
required: no
default: {}
parameters:

Name Type Description Required Default
hidden array[string] Files that will be hidden from the file explorer No []
open_at_startup array[string] Files that will be open when the workspace loads No []
read_only array[string] Files that are read only - cannot be modified in the workspace No []

example:

{
  "hidden": [".env", ".gitignore"],
  "open_at_startup": ["README.md"],
  "read_only": ["config.yaml"]
}

root.files.hidden

Files that will be hidden from the file explorer.

type: array[string]
required: no
default: []

example:

"hidden": [".env", ".gitignore"]

Note: Each element of the list should contain relative or absolute path to the project file or directory that should be hidden in the file directory.

root.files.open_at_startup

Files that will be open when the workspace loads.

type: array[string]
required: no
default: []

example:

"open_at_startup": ["README.md"]

Note: Each element of the list should contain relative or absolute path to the project file that should automatically open during workspace's launch.

root.files.read_only

Files that are read only - cannot be modified in the workspace.

For more information, please refer to an extended description of this parameter.

type: array[string]
required: no
default: []

example:

"read_only": ["config.yaml"]

Note: Each element of the list should contain relative or absolute path to the project file whose permissions should be limited to read-only.

root.custom

Any data of any structure.

type: object
required: no
default: {}
parameters: any

example:

{
  "internal_tag": "3317c1863f8dd227c2e438e6cabf466b",
  "categories": ["simple programming", "data processing", "algorithms"]
}
  Contents