Frequently Asked Questions

The Sphere Engine Compilers module is a service that enables remote execution of computer programs and retrieving the results of their execution.

The Sphere Engine Problems module enables the automated verification of submissions to programming problems. It includes a Problems Manager feature for efficient programming problem creation and testing.

The Sphere Engine Containers module allows executing complex, multi-component programming projects. It supports a vast scope of modern technologies and will bring the source code execution and skill-assessment features to your system effectively, securely, and with large-scaling, future-proofing, and customizability.

copy link

If your project primarily involves running computer programs, such as executing sample source code within a programming course, we recommend selecting the Compilers module.

For scenarios where you need to assess programs, like solutions to programming challenges within a recruitment platform, the Problems module is the ideal choice.

If your project entails running or evaluating complex, multi-stack software projects, such as a web framework with a database, the Workspace module is the most suitable option.

Feel free to mix and match these modules within your application as needed.

More information about the differences between the modules can be found in this module comparison table.

copy link

The Sphere Engine Compilers Widget is a method of integrating the Compilers module through a JavaScript widget embedded on a website.

copy link

The Sphere Engine Problems Widget is a method of integrating the Problems module through a JavaScript widget embedded on a website.

copy link

The Sphere Engine Containers Workspace is a method of integrating the Containers module by embedding a full-fledged development environment on your website using a simple JavaScript SDK.

copy link

Yes - the Problems section of the Problems Manager available after logging in to the Sphere Engine client panel (Menu > Problems > Editor) allows you to filter the problems list by pressing the Shared problems button. The displayed list contains sample programming problems that can be used inside your system right away or for testing purposes.

copy link

Yes, on the Projects page, which can be accessed after logging in to the Sphere Engine Client Panel (Menu > Containers > Projects), you'll find a wide array of samples that cover a diverse range of technologies. These samples serve as an excellent starting point for developing your own customized projects.

copy link

The Sphere Engine API is based on RESTful architecture. This means that any technology that allows for HTTP requests is fully compatible with the API.

copy link

The following integration methods are available for the Sphere Engine modules:

• RESTful API (available for all modules) - is a highly configurable integration method, recommended for building dedicated applications from the ground up,
• Widget integration (available for Compilers and Problems modules) - offers a quick and easy integration method, ideal for expanding existing projects,
• Workspace integration (available for Containers module) - is a convenient and powerful method for integrating an interactive, full-fledged development environment,
• Webhooks (available for all modules) - complement all the aforementioned methods, providing an optimal way to extract information from Sphere Engine.

copy link

For the Compilers and the Problems modules, please use version 4 of our APIs, which is the latest one. We have moved from API Version 3 to API Version 4 in 2018 (full list of changes).

For the Containers module, please use version 1.1.

copy link

Sphere Engine is an online, secure, auto-scalable service with frequently added new features and regular updates of programming languages. If you are interested in an independent instance of the Sphere Engine system running on your servers, please contact us.

copy link

Integration using the Javascript SDK is limited to adding short snippets of HTML and JavaScript code.

Find out more in the following articles:

• Sphere Engine Compilers Widget quickstart
• Sphere Engine Problems Widget quickstart
• Sphere Engine Containers Workspaces quickstart

copy link

Access to the Sphere Engine API is implemented using access tokens.

After logging in to the Sphere Engine client panel, the tokens management section (Menu > API Tokens) allows you to generate any number of access tokens that can be passed to your development team.

Alternatively, go to Menu > Users and create a sub-account with a Developer role.

copy link

To use Sphere Engine as a white-label product, please contact us.

copy link

While it may be tempting to integrate with an API in such a way, there are at least two reasons for not doing so:

• CORS and the Same-origin policy limitations
• security violation by revealing the API access token that should remain secret

CORS mechanism enables the communication between two mutually trusted servers hosted on different domains. The server which handles cross-origin requests (in our scenario it is the Sphere Engine API) should send a proper HTTP header (i.e., Access-Control-Allow-Origin) allowing the request to be handled. Sphere Engine API denies this kind of requests for security reasons.

The mechanism of the API access tokens is the only method of API call authorization. Revealing the access token that is connected to your account will enable everyone to use Sphere Engine API on your credit. Access tokens should remain secret and if they leak out, they should be changed immediately. More precisely: compromised access tokens should be disabled in the proper panel where you can also generate fresh access tokens.

We recommend designing your web applications in such a way that they make Sphere Engine API calls in the back-end. If the front-end layer of your application requires asynchronous communication, please consider creating a custom layer designated to handling the Sphere Engine API (i.e. asynchronous calls should target your back-end application and then your back-end makes the Sphere Engine API call; then pass the response back in the opposite direction).

copy link

Most likely you are trying to make an API call directly from the front-end of your web application. This is not possible due to the Same-origin policy. Please refer to the Can I use Sphere Engine API methods directly from the front-end of my web application? question to learn more.

copy link

The Compilers and the Problems module supports the majority of popular programming languages, including C++, C#, Go, Haskell, Java, Kotlin, Node.js, PHP, Python, Ruby, Scala, or Swift.

A separate part of the documentation contains a complete list of supported languages.

copy link

The high level of customizability offered by the Containers module enables you to utilize a vast array of existing technologies. This includes:

• classical programming languages (like Java, Python),
• front-end technologies and frameworks (e.g. Javascript, HTML, React, Angular),
• multi-stack applications (e.g., Node.js+React+MySQL),
• data science and machine learning (e.g, Pandas, TensorFlow, Jupyter Notebook),
• mobile technologies (e.g., Flutter, React Native),
• databases (e.g., MySQL, MongoDB),
• tools (e.g., git, ansible).

copy link

Yes!

For the Compilers and Problems modules, you can perform disk operations within an isolated workspace dedicated to a single submission. Find out more about performing disk operations from the disk operations article.

With the Containers module, there's no need for specific configurations. Your programs can freely perform disk operations just as they would in a local or server environment.

copy link

For the Compilers and the Problems modules, find out from the submission streams article.

For the Containers module, you can read on here.

copy link

By default, the executed program for the Compilers and Problems modules does not have Internet access. However, it can be enabled for subscribed users.

For the Containers module, access to the Internet is an configurable option in the project settings.

copy link

Multiple factors affect the program's execution time, for example:

• the programming language,
• the complexity of the program,
• the size of input data,
• time limits,
• the number of test cases (only Sphere Engine Problems).

A typical submission is executed within a few seconds. Long execution times are typically associated with the complexity of the program, not with the Sphere Engine infrastructure.

copy link

There is nothing to worry about. Each correctly sent submission is stored in a database. If there are no free resources to execute the submission immediately, it waits in the submission queue. You do not need to implement a queue of submissions in your system - Sphere Engine handles that for you.

copy link

For the Compilers and the Problems modules using integration through RESTful API it is possible to run multi-file projects.

In the Containers module, this is the typical use case. When working in the Workspace, you have convenient access to the directory tree. When making API submissions, you always send a tar.gz archive that can include multiple files or even an entire directory structure. You can find more information about the structure of such a tar.gz archive here.

copy link

We regularly expand the library of supported programming languages and libraries, very often at our clients' requests. Please contact us if you feel that this is what you need.

copy link

We regularly expand the capabilities of Sphere Engine, very often at our clients' requests. Please contact us if you feel that this is what you need.

copy link

The Sphere Engine Problems module equips you with all the necessary tools to create your online programming competition or recruitment platform. However, it is not designed to directly support or host such an event.

Similarly, the Sphere Engine Containers module provides you with essential tools for constructing a platform to manage such events. Yet, as middleware, it is not designed to independently host or manage these scenarios.

For running online contests and hackathons, you can also use Spoj (Sphere Engine Online Judge) platform.

copy link

The default limit of the number of submissions depends on the method of integration (JavaScript widget or RESTful API) and the type of subscription.

copy link

In the case of the Compilers module, the default limitation for the program execution time depends on the configuration and can range from 1 to 15 seconds.

In the Problems module, the default program execution time limit results from the test case parameters and can also range from 1 to 60 seconds.

In the Containers module, the default time limit can be set arbitrally in the project configuration for each execution stage. You can read on about the timeout parameter here.

copy link

The default maximum number of test cases is 16. If the test case construction is correct, this number should be sufficient. Please refer to the "Good test case design article" from the content manager's handbook which explains how to create correct and valuable test cases for problems.

It is also possible to increase the maximum number of test cases - please contact us if you feel that this is what you need.

copy link

The maximum size of uploaded source files as part of a single submission is:

• 64kB for the Sphere Engine Compilers module,
• 32kB for the Sphere Engine Problems module,
• 256kB for the Sphere Engine Containers module.

copy link

The maximum size of the input data file in the Sphere Engine Compilers service is 64kB.

copy link

Data generated when executing submissions can have any size (OS limitations may apply). However, Sphere Engine saves only a shortened data stream of the maximum allowed stream size.

For the Sphere Engine Compilers module, the maximum stream size is 1MB for both streams.

For the Sphere Engine Problems module, the maximum stream size is:

• 5kB for the "stdout" stream (for each test case),
• 1kB for the "stderr" stream (for each test case),
• 64kB for the "psinfo" stream.

For the Sphere Engine Containers module, the maximum stream sizes are:

• 100kB for each stage output stream,
• 100kB for each stage error stream,
• 1MB for the tests report stream,
• 1MB for the auxiliary data stream (please note that this stream always consists of a tar.gz archive, and if truncated, it is highly likely to become corrupted),
• 100kB for the debug logs stream,
• 100kB for the workspace initialization process output stream,
• 100kB for the workspace initialization process error stream,

copy link

Each file with test case data can weigh up to 20MB. It is also possible to increase the maximum size of test case files - please contact us if you feel that this is what you need.

copy link

For the Compilers and the Problems modules, the program has access to 1.5GB of RAM. If this limit is exceeded, the execution of the program is interrupted.

In the Containers module, the amount of RAM memory available depends on the number of "slots" assigned to the project. Each slot grants access to 2GB of RAM for execution. For different slot values, the RAM allocation is proportional, for example, 512MB for 0.5 slots, and 8GB for 4 slots.

copy link

For the Compilers and Problems modules, each program is allocated access to a single physical CPU core, providing exclusive usage during execution.

In the Containers module, the number of CPU cores accessible is contingent upon the number of 'slots' assigned to the project. Each slot provides access to 1 CPU for execution. For varying slot values, the CPU allocation scales proportionally; for instance, 0.5 CPU for 0.5 slots, and 4 CPUs for 4 slots.

copy link

The default limit is up to 1000 problems, however, it is possible to remove the limit of the number of problems. Please contact us if you feel that this is what you need.

copy link

After submitting the source code, it is placed in a queue and awaits execution by the checker machine, which is responsible for running the submitted programs.

For the Compilers module, the checker machine processes the submission by following these steps:

• compilation of the program (applies only to compiled languages like C++),
• execution of the program,
• provision of input data to the executing program (if applicable),
• generation and storage of output data, such as stdout, stderr streams, and execution results, including status (e.g., "successful" or "not successful"), exit code, and execution time.

In the Problems module, the checker machine processes the submission as follows:

• compilation of the program (applies only to compiled languages like C++),
• execution of the program, performed independently for each test case,
• verification of the results of each test case execution by the "test case judge",
• summarization of the results from all test cases by the "master judge",
• generation and storage of output data, such as stdout, stderr streams, and execution results, including status, score, and execution time.

For the Containers module, the checker machine processes the submission by executing a scenario defined in the project configuration. This involves:

• initializing the environment using the workspace init script,
• executing a sequence of stages in the following order:
  • init stage for project initialization,
  • build stage for building the project,
  • run stage for executing the project,
  • test for testing or assessment procedures,
  • post for cleanup, summaries, output gathering, and other post-execution operations,
• generating and storing output data, such as output from stages, test results, and execution results, including status, score, and execution time.

copy link

There are many reasons why a seemingly correct submission may not get the maximum points.

First, it is worth verifying the following aspects of the submission:

• the correctness of handling edge cases,
• program's performance for large input data,
• the correctness of the program for atypical input data,
• the correctness of the program for input data realizing maximum ranges for variables occurring in input data.

copy link

There are many reasons why a program sent to Sphere Engine does not run correctly.

After executing the submission, information that can help identify the right reason is available in:

• the status of the submission (e.g. "compilation error"/"build error", "run-time error", "wrong answer"),
• the content of data streams (e.g. compilation errors and run-time errors).

copy link

Some code written in the languages supported by Sphere Engine requires specific naming or structure for the Compilers and the Problems modules. For example, a Java application should contain a public class named Main (or a private class named as you desire).

Familiarizing yourself with sample source code files for the selected programming language (available in the public GitHub repository with common configuration for programming languages) may be useful in identifying the reason for the error.

copy link

Data regarding compilation errors can be found in the stream containing the compilation errors.

copy link

Data regarding build errors can be found in the output stream and error stream of the build stage.

copy link

If the program has been running for too long (i.e. longer than the execution time limit), its execution was interrupted by the system. In this situation, the submission (or a single test case in case of the Problems module) receives the TLE status.

Examples of reasons why a program might be executing for a long time:

• the occurrence of infinite loops,
• the high computational complexity of the algorithm used in the program,
• a large amount of data processed by the program.

copy link

The most common reason is an error in solving the problem. First of all, check the correctness of handling edge cases and tricky, rare or specific data. Next, it is recommended to read the description of the problem again, to make sure that the answer is correct.

copy link

There are many reasons why the processing of a submission may be interrupted with an error. The most common causes are:

• arithmetic errors (e.g. dividing by zero),
• unauthorized memory access,
• allocation of too many resources.

More information on runtime errors can be found in Wikipedia.

copy link

The most important components of a programming problem are its test cases. Test cases consist of input data and reference output data. The submitted program is executed with the input data of the test case and then the output data it generates is compared to the reference output data.

A detailed description of processing submissions in the Problems module is described in the Submission flow chapter of the content manager's handbook.

copy link

There are various approaches to verify the correctness of submissions in the Containers module. The most straightforward, similar to the one used in the Problems module, involves comparing the produced output data with the expected output data of the application.

An alternative approach is based on unit testing, where the submission comprises the application's source code, and predefined unit tests are executed to check if the application passes them.

In another scenario, the reverse approach to unit testing is employed. Here, the task involves writing unit tests, and the verification process checks whether these tests correctly identify injected issues in the predefined application.

One of the advantages of the Sphere Engine Containers module is its high customizability. The list of verification methods is not limited to the ones mentioned above. Any type of verification can be implemented as part of the test stage in the project's scenario.

copy link

For each test case, the "test case judge" sets a partial score. Then, after running the program for all test cases, the "master judge" summarizes the partial results and determines the final result.

Both the "test case judge" and the "master judge" operate in a way designed by the author.

Example

Master judge "Score is % of correctly solved sets" gives a score proportional to the number of correctly solved test cases. For a problem with five test cases, a user's submission that correctly solves three test cases will receive a final result of 60, which should be interpreted as 60% (3 out of 5 = 60%).

copy link

The score is determined during the test stage of the project's scenario. While it can be set arbitrarily, there are some common and useful approaches.

For example, in projects where correctness verification is based on the results of a unit testing procedure, a reasonable score is the percentage of passed unit tests. Alternatively, when the difficulty of unit tests varies, one can apply a modified approach by assigning weights to each unit test.

copy link

All data related to submissions (such as source code and data streams) in the Compilers and Problems modules is stored on Sphere Engine servers for one month. If you require data storage for a longer duration, please don't hesitate to contact us.

For the Containers module, the availability of data varies based on the stream:

• data is stored for 30 days for the following:
  • source code,
  • stage streams (stage output and error stream),
  • debug log
  • test report,
  • workspace initialization data (output and error streams),
  • custom files, also known as "auxiliary data streams", for API submissions,
• custom files ("auxiliary data streams") for webhooks from Workspace actions are stored for 24 hours.

copy link