Webhooks

Note: The webhook mechanism is not available for the new Sphere Engine Containers module yet. The release of webhook integration for the new module is planned for the first quarter of 2023.

Our implementation of the webhook mechanism has been designed to deliver a reliable and efficient method for event communication between Sphere Engine components. When the execution of the submission finishes, our system instantly sends a message about it to the client's system. Think of it as push notifications from social media services that go directly to your web browser.

The following diagram depicts the idea is a very simple way:

Sphere Engine Webhooks Diagram
Fig. 1. Webhooks diagram

What are webhooks and how to receive them?

Webhook is nothing more than an HTTP request that goes from a service (Sphere Engine) to the client's application. Note that the communication flow in the case of webhooks works opposite to the usual one in which the client is a side that sends the request.

Receiving webhooks consists of:

  • publishing an endpoint to send requests to (e.g. your_awesome_site.com/your_app/webhook_endpoint),
  • receiving and processing incoming requests inside your application.

You can receive webhooks from all Sphere Engine components:

  • Compilers API
  • Compilers Widget
  • Problems API
  • Problems Widget
  • Problems LTI tool
  • (coming soon!) Containers API
  • (coming soon!) Containers Workspaces

Configuring a webhook in Sphere Engine Dashboard

The webhook configuration is global for all Sphere Engine components. Go to Dashboard > Webhooks and provide the required information. For example:

Configuring webhooks in Sphere Engine dashboard
Fig. 2. Configuring webhooks in Sphere Engine dashboard
  • Webhook URL is the space for a complete URL; use this web address to send requests to
  • Choose which notifications you want to subscribe to; requests from all modules are sent to the same webhook endpoint, but you will be able to distinguish between them using the information from the message body

Webhook message specification

Webhook message is an HTTP POST request that goes from Sphere Engine to the client's webhook endpoint. The body of the request contains valid JSON data with its core unified for all Sphere Engine components.

Message core

Name Type Description
origin string

The identifier of the system that can take the following values:

  • sec - Compilers API
  • sep - Problems API
  • secw - Compilers Widget
  • sepw - Problems Wiget
  • se4e - Sphere Engine For Education (Problems LTI tool)
timestamp integer The Unix timestamp generated when the message is created
submission object Information about the submission
widget (present only for origin values: secw, sepw, se4e) object Additional context related to execution within a widget

The structure the submission object (see Message core above) for Compilers API (origin=sec) and Compilers Widget (origin=secw)

Name Type Description
id integer submission id
executing boolean indicates whether a submission is being executed
date string date and time of submission creation [yyyy-mm-­dd hh:mm:ss TZD] note that server time is used
compiler object
compiler.id integer compiler id
compiler.name string compiler name
compiler.version.id integer compiler version id
compiler.version.name string compiler version name
result object
result.status.code integer status code (see section "Submission status")
result.status.name string status name
result.time float execution time [seconds]
result.memory integer memory consumed by the program [kilobytes]
result.signal.code integer signal raised by the program
result.signal.name string description of the raised signal

The structure the widget object (see Message core above) for Compilers Widget (origin=secw)

Name Type Description
hash string widget identifier
submission.id integer submission id in the widget context; note that it's not the same id as for submission object
custom_data string custom data provided during widget initialization in data-custom-data parameter

The structure the submission object (see Message core above) for Problems API (origin=sep), Problems Widget (origin=sepw) and Problems LTI tool aka Sphere Engine For Education (origin=se4e)

Name Type Description
id integer submission id
executing boolean indicates whether a submission is being executed
date string date and time of submission creation [yyyy-mm-­dd hh:mm:ss TZD] note that server time is used
compiler object
compiler.id integer compiler id
compiler.name string compiler name
compiler.version.id integer compiler version id
compiler.version.name string compiler version name
problem object
problem.id integer problem id
problem.code string [deprecated] problem code
problem.name string problem name
result object
result.status.code integer status code (see section "Submission status")
result.status.name string status name
result.score float final score
result.time float execution time [seconds]
result.memory integer memory consumed by the program [kilobytes]
result.signal.code integer signal raised by the program
result.signal.name string description of the raised signal
result.testcases[].number integer test case number
result.testcases[].status.code integer test case status code (see section "Submission status")
result.testcases[].status.name string test case status name
result.testcases[].score float test case score
result.testcases[].time float test case execution time [seconds]
result.testcases[].memory integer test case memory consumed by the program [kilobytes]
result.testcases[].signal integer test case signal raised by the program
result.testcases[].signal_desc string description of the raised test case signal

The structure the widget object (see Message core above) for Problems Widget (origin=sepw)

Name Type Description
hash string widget identifier
submission.id integer submission id in the widget context
submission.grade float submission grade
custom_data string custom data provided during widget initialization in data-custom-data parameter
user object
user.name string user name provided on the welcome page
user.email string user email address provided on the welcome page
user.uuid string user identifier generated by the Sphere Engine system
user.id string user identifier provided during widget initialization (user-id parameter)

The structure the widget object (see Message core above) for Problems LTI tool aka Sphere Engine For Education (origin=se4e)

Name Type Description
consumer_key string consumer key from LTI integration settings
submission.id integer submission id in the widget context
submission.grade float submission grade
custom_data string custom data provided during widget initialization in data-custom-data parameter
user object
user.name string username delivered from LMS; it can be null for strict privacy LMS settings
user.email string user email delivered from LMS; it can be null for strict privacy LMS settings
user.uuid string user identifier generated by the Sphere Engine system
user.id string user identifier generated by LMS
LTI protocol-specific parameters
tool_consumer_instance_guid string parameter tool_consumer_instance_guid from LTI protocol; it can be null
context_id string parameter context_id from LTI protocol; it can be null
resource_link_id string parameter resource_link_id from LTI protocol

Webhook messages examples

{
    "origin": "sec",
    "timestamp": 1610636162,
    "submission": {
        "id": 167124381,
        "executing": false,
        "date": "2021-01-14 14:55:56 +00:00",
        "compiler": {
            "id": 29,
            "name": "PHP",
            "version": {
                "id": 5,
                "name": "php 7.4.3"
            }
        },
        "result": {
            "status": {
                "code": 15,
                "name": "accepted"
            },
            "time": 0.01,
            "memory": 51824,
            "signal": {
                "code": 0,
                "name": ""
            }
        }
    }
}
Compilers API 
{
    "origin": "sep",
    "timestamp": 1610636162,
    "submission": {
        "id": 15778763,
        "executing": false,
        "date": "2021-01-14 14:55:56 +00:00",
        "compiler": {
            "id": 29,
            "name": "PHP",
            "version": {
                "id": 5,
                "name": "php 7.4.3"
            }
        },
        "problem": {
            "id": 170,
            "code": "SETEST",
            "name": "Life, the Universe, and Everything"
        },
        "result": {
            "status": {
                "code": 15,
                "name": "accepted"
            },
            "score": 100,
            "time": 0.01,
            "memory": 52488,
            "signal": 0,
            "signal_desc": "",
            "testcases": [
                {
                    "number": 0,
                    "status": {
                        "code": 15,
                        "name": "accepted"
                    },
                    "score": 0,
                    "time": 0.01,
                    "memory": 52488,
                    "signal": 0,
                    "signal_desc": ""
                }
            ]
        }
    }
}
Problems API 
{
    "origin": "secw",
    "timestamp": 1610636162,
    "submission": {
        "id": 167137007,
        "executing": false,
        "date": "2021-01-14 14:55:56 +00:00",
        "compiler": {
            "id": 29,
            "name": "PHP",
            "version": {
                "id": 5,
                "name": "php 7.4.3"
            }
        },
        "result": {
            "status": {
                "code": 15,
                "name": "accepted"
            },
            "time": 0.01,
            "memory": 82560,
            "signal": {
                "code": 0,
                "name": ""
            }
        }
    },
    "widget": {
        "hash": "a532aa0f61da800f8288c77588a29cd5",
        "submission": {
            "id": 1151666
        },
        "custom_data": null
    }
}
Compilers Widget 
{
    "origin": "sepw",
    "timestamp": 1610636162,
    "submission": {
        "id": 15780466,
        "executing": false,
        "date": "2021-01-14 14:55:56 +00:00",
        "compiler": {
            "id": 29,
            "name": "PHP",
            "version": {
                "id": 5,
                "name": "php 7.4.3"
            }
        },
        "problem": {
            "id": 170,
            "code": "SETEST",
            "name": "Life, the Universe, and Everything"
        },
        "result": {
            "status": {
                "code": 15,
                "name": "accepted"
            },
            "score": 100,
            "time": 0.01,
            "memory": 52488,
            "signal": 0,
            "signal_desc": "",
            "testcases": [
                {
                    "number": 0,
                    "status": {
                        "code": 15,
                        "name": "accepted"
                    },
                    "score": 0,
                    "time": 0.01,
                    "memory": 52488,
                    "signal": 0,
                    "signal_desc": ""
                }
            ]
        }
    },
    "widget": {
        "hash": null,
        "submission": {
            "id": 4458436,
            "grade": 0
        },
        "user": {
            "name": null,
            "email": null,
            "uuid": null,
            "id": null
        },
        "custom_data": null
    }
}
Problems Wiget 
{
    "origin": "se4e",
    "timestamp": 1610636162,
    "submission": {
        "id": 15781550,
        "executing": false,
        "date": "2021-01-14 14:55:56 +00:00",
        "compiler": {
            "id": 29,
            "name": "PHP",
            "version": {
                "id": 5,
                "name": "php 7.4.3"
            }
        },
        "problem": {
            "id": 170,
            "code": "SETEST",
            "name": "Life, the Universe, and Everything"
        },
        "result": {
            "status": {
                "code": 15,
                "name": "accepted"
            },
            "score": 100,
            "time": 0.01,
            "memory": 52488,
            "signal": 0,
            "signal_desc": "",
            "testcases": [
                {
                    "number": 0,
                    "status": {
                        "code": 15,
                        "name": "accepted"
                    },
                    "score": 0,
                    "time": 0.01,
                    "memory": 52488,
                    "signal": 0,
                    "signal_desc": ""
                }
            ]
        }
    },
    "widget": {
        "consumer_key": null,
        "tool_consumer_instance_guid": null,
        "context_id": null,
        "resource_link_id": null,
        "submission": {
            "id": 4458446,
            "grade": 0
        },
        "user": {
            "name": null,
            "email": null,
            "uuid": null,
            "id": null
        },
        "custom_data": null
    }
}
Sphere Engine For Education (Problems LTI tool)

Acknowledging webhooks

Sphere Engine service keeps a webhook message for a while to make sure that the client has received it properly. The elementary rule that the webhook endpoint should implement is that each webhook is an HTTP POST request that expects a specific response:

  • respond with the HTTP status code 200 to acknowledge receiving and processing the webhook message,
  • any other HTTP response code will be interpreted as a failure and webhook message will not be acknowledged.

Retry schedule in the case of failures

Sometimes, the first delivery attempt of the webhook message fails. It can be caused by multiple factors. Just to name a few:

  • Network failure between client's webhook endpoint and Sphere Engine service
  • Unexpected failure on the Sphere Engine side
  • Unexpected failure on the client's application side
  • Unacknowledged webhook message (i.e. non-200 HTTP status code in the response)

To make sure that the webhook message does not get completely lost, we defined a retry schedule. For each message, we will do up to 15 additional delivery attempts. Each attempt is performed with a delay that grows with the total number of attempts according to the table:

# Delay
1 1 second
2 3 seconds
3 9 seconds
4 16 seconds
5 32 seconds
6 1 minute
7 5 minutes
8 15 minutes
9 45 minutes
10 2 hours
11 4 hours
12 8 hours
13 12 hours
14 12 hours
15 12 hours
16 12 hours

There are more than two days for a webhook message to be properly delivered. After a series of 16 failure attempts, the webhook message is lost and will not be delivered.

Testing webhooks without endpoint

If you simply want to test the features, reliability, or behavior of the webhooks you can do it without an endpoint on your side. Such services as https://webhook.site or https://requestbin.com/ allow you to generate a unique custom URL that you can use as your webhook endpoint for testing purposes.

Implementing webhook endpoint

It is possible to implement a webhook endpoint using any technology that can handle HTTP protocol. You can do it as a standalone application as well as part of a bigger one. You can use a web framework or create everything from scratch.

Here is a simple PHP script that implements an elementary webhook endpoint capable of receiving incoming messages and storing them in a messages.dat file.

<?php
    file_put_contents('messages.dat', 'HEADERS: ' . serialize($_REQUEST) . PHP_EOL, FILE_APPEND | LOCK_EX);
    $requestBody = file_get_contents("php://input");
    file_put_contents('messages.dat', 'DATA: ' . $requestBody . PHP_EOL, FILE_APPEND | LOCK_EX);
    http_response_code(200); // response code 200 to acknowledge

Note: In the example, you need to read standard input data to fetch request body data. If you use a web application framework (e.g. Symfony, Django, Laravel) it is very likely that there is a dedicated method that helps you get raw body data from POST requests.

You can configure any webserver to use the above file for processing requests. You can also run a build-in PHP web server using the following command:

php -S 0.0.0.0:8080 webhook_endpoint.php

This command starts a web server on the port 8080. Requests will be processed by the webhook_endpoint.php script.

Note: Keep in mind that your server needs to be accessible from the Internet. You cannot provide “localhost”, “127.0.0.1”, or any local address as a webhook endpoint address.

  Contents