INGInious’ documentation¶

Installation, configuration and upgrade¶

Installation and deployment¶

Supported platforms¶

INGInious is intended to run on Linux (kernel 3.10+), but can also be run on Windows and macOS thanks to the Docker toolbox.

Dependencies setup¶

INGInious needs:

Python (with pip) 3.5+
Docker 1.12+
MongoDB
Libtidy
LibZMQ

Danger

This version relies on older web framework and won’t work correctly with Python >= 3.7. Consider installer a newer version for compatibility with the most recent Python interpreters.

RHEL/Cent OS 7.0+, Fedora 24+¶

The previously mentioned dependencies can be installed, for Cent OS 7.0+ :

# curl -fsSL https://get.docker.com/ | sh #This will setup the Docker repo
# yum install -y epel-release
# yum install -y git mongodb mongodb-server gcc libtidy python36 python36-pip python36-devel zeromq-devel

Or, for Fedora 24+:

# curl -fsSL https://get.docker.com/ | sh #This will setup the Docker repo
# dnf install -y git mongodb mongodb-server gcc libtidy python3 python3-pip python3-devel zeromq-devel

You may also add openldap-devel if you want to use the LDAP auth plugin and xmlsec1-openssl-devel libtool-ltdl-devel for the SAML2 auth plugin.

Danger

Due to compatibility issues, it is recommended to disable SELinux on the target machine.

You can now start and enable the mongod and docker services:

# systemctl start mongod
# systemctl enable mongod
# systemctl start docker
# systemctl enable docker

Ubuntu 16.04+¶

The previously mentioned dependencies can be installed, for Ubuntu 16.04+:

# curl -fsSL https://get.docker.com/ | sh #This will setup the Docker repo
# apt-get install git mongodb gcc tidy python3 python3-pip python3-dev libzmq3-dev

You may also add libldap2-dev libsasl2-dev libssl-dev if you want to use the LDAP auth plugin and libxmlsec1-dev libltdl-dev for the SAML2 auth plugin

You can now start and enable the mongod and docker services:

# systemctl start mongodb
# systemctl enable mongodb
# systemctl start docker
# systemctl enable docker

OS X 10.9+¶

We use brew to install some packages. Packages are certainly available too via macPorts.

$ brew install mongodb
$ brew install python3

Follow the instruction of brew to enable mongodb.

The next step is to install Docker for Mac.

Windows¶

Danger

INGInious rely on Docker to run containers. While Docker is supported on Windows 10 (version 1607), INGInious does not provide support for Windows containers yet.

The recommended way to run INGInious under Windows is by using a Linux virtual machine, for much more simplicity. One can also only run the Docker agent under a Linux virtual machine and run the backend and selected frontend under Windows.

In the later case, you’ll need to install Python 3.5+, MongoDB, LibTidy and LibZMQ.

Installing INGInious¶

The recommended setup is to install INGInious via pip and the master branch of the INGInious git repository. This allows you to use the latest development version. This version is currently the supported one for issues.

$ pip3 install --upgrade git+https://github.com/UCL-INGI/INGInious.git@inginious-0.6

This will automatically upgrade an existing version.

Note

You may want to enable the LDAP/SAML2 plugin or use FCGI/UWSGI instead of the web.py default webserver. In this case, you have to install more packages: simply add [cgi], [uwgsi], [ldap] or [saml2] to the above command, depending on your needs:

$ pip3 install --upgrade git+https://github.com/UCL-INGI/INGInious.git@inginious-0.6#egg=INGInious[cgi,ldap]

Configuring INGInious¶

INGInious comes with a mini-LMS web app that provides statistics, groups management, and the INGInious studio, that allows to modify and test your tasks directly in your browser. It supports the LTI interface that allows to interface with Learning Management System via the LTI specification. Any LMS supporting LTI is compatible. This includes Moodle, edX, among many others.

To configure the web app automatically, use the inginious-install CLI.

$ inginious-install

This will help you create the configuration file in the current directory. For manual configuration and details, see Configuration reference.

The detailed inginious-install reference can be found at inginious-install.

Running INGInious¶

During the configuration step, you were asked to setup either a local or remote backend. In the former case, the frontend will automatically start a local backend and grading agents.

With local backend/agent¶

To run the frontend, please use the inginious-webapp CLI. This will open a small Python web server and display the url on which it is bind in the console. Some parameters (configuration file, host, port) can be specified. Details are available at inginious-webdav.

With remote backend/agent¶

To run INGInious with a remote backend (and agents), do as follows:

On the backend host, launch the backend (see inginious-backend) :
```
inginious-backend tcp://backend-host:2001 tcp://backend-host:2000
```
The agents will connect on tcp://backend-host:2001 and clients on tcp://backend-host:2000
Possibly on different hosts, launch the Docker and MCQ agents (see inginious-agent-docker and inginious-agent-mcq) :
```
inginious-agent-docker tcp://backend-host:2001
inginious-agent-mcq tcp://backend-host:2001
```
In your INGInious frontend configuration file (see Configuration reference), set backend to :
```
backend: tcp://backend-host:2000
```

Run the frontend using inginious-webdav.

inginious-webapp --config /path/to/configuration.yaml

WebDAV setup¶

An optional WebDAV server can be used with INGInious to allow course administrators to access their course filesystem. This is an additional app that needs to be launched on another port or hostname. Run the WebDAV server using inginious-webdav.

inginious-webdav --config /path/to/configuration.yaml --port 8000

In your configuration file (see Configuration reference), set webdav_host to:

<protocol>://<hostname>:<port>

where protocol is either http or https, hostname and port the hostname and port where the WebDAV app is running.

Webterm setup¶

An optional web terminal can be used with INGInious to load the remote SSH debug session. This rely on an external tool.

To install this tool :

$ git clone https://github.com/UCL-INGI/INGInious-xterm
$ cd INGInious-xterm && npm install

You can then launch the tool by running:

$ npm start bind_hostname bind_port debug_host:debug_ports

This will launch the app on http://bind_hostname:bind_port. The debug_host and debug_ports parameters are the debug paramaters on the local (see Configuration reference) or remote (see inginious-agent-docker) Docker agent.

To make the INGInious frontend aware of that application, update your configuration file by setting the webterm field to http://bind_hostname:bind_port (see Configuration reference).

For more information on this tool, please see INGInious-xterm. Please note that INGInious-xterm must be launched using SSL if the frontend is launched using SSL.

Webserver configuration¶

The following guides suggest to run the INGInious webapp on http port and WebDAV on port 8080 on the same host. You are free to adapt thm to your use case (for instance, adding SSL support or using two hostnames).

Warning

In configurations below, environment variables accessible to the application must be explicitly repeated. If you use a local backend with remote Docker daemon, you may need to set the DOCKER_HOST variable. To know the value to set, start a terminal that has access to the docker daemon (the terminal should be able to run docker info), and write echo $DOCKER_HOST. If it returns nothing, just ignore this comment. It is possible that you may need to do the same for the env variable DOCKER_CERT_PATH and DOCKER_TLS_VERIFY too.

Using lighttpd¶

In production environments, you can use lighttpd in replacement of the built-in Python server. This guide is made for CentOS 7.x.

Install lighttpd with fastcgi:

# yum install lighttpd lighttpd-fastcgi

Add the lighttpd user in the necessary groups, to allow it to launch new containers and to connect to mongodb:

# usermod -aG docker lighttpd
# usermod -aG mongodb lighttpd

Create a folder for INGInious, for example /var/www/INGInious, and change the directory owner to lighttpd:

# mkdir -p /var/www/INGInious
# chown -R lighttpd:lighttpd /var/www/INGInious

Put your configuration file in that folder, as well as your tasks, backup, download, and temporary (if local backend) directories (see Configuring INGInious for more details on these folders).

Once this is done, we can configure lighttpd. First, the file /etc/lighttpd/modules.conf, to load these modules:

server.modules = (
    "mod_access",
    "mod_alias"
)

include "conf.d/compress.conf"
include "conf.d/fastcgi.conf"

You can then add virtual host entries in a /etc/lighttpd/vhosts.d/inginious.conf file and apply the following rules:

server.modules   += ( "mod_fastcgi" )
server.modules   += ( "mod_rewrite" )

$SERVER["socket"] == ":80" {
    alias.url = (
        "/static/" => "/usr/lib/python3.5/site-packages/inginious/frontend/static/"
    )

    fastcgi.server = ( "/inginious-webapp" =>
        (( "socket" => "/tmp/fastcgi.socket",
            "bin-path" => "/usr/bin/inginious-webapp",
            "max-procs" => 1,
            "bin-environment" => (
                "INGINIOUS_WEBAPP_HOST" => "0.0.0.0",
                "INGINIOUS_WEBAPP_PORT" => "80",
                "INGINIOUS_WEBAPP_CONFIG" => "/var/www/INGInious/configuration.yaml",
                "REAL_SCRIPT_NAME" => ""
            ),
            "check-local" => "disable"
        ))
    )

    url.rewrite-once = (
        "^/favicon.ico$" => "/static/icons/favicon.ico",
        "^/static/(.*)$" => "/static/$1",
        "^/(.*)$" => "/inginious-webapp/$1"
    )
}

$SERVER["socket"] == ":8080" {
    fastcgi.server = ( "/inginious-webdav" =>
        (( "socket" => "/tmp/fastcgi.socket",
            "bin-path" => "/usr/bin/inginious-webdav",
            "max-procs" => 1,
            "bin-environment" => (
                "INGINIOUS_WEBDAV_HOST" => "0.0.0.0",
                "INGINIOUS_WEBDAV_PORT" => "8080",
                "INGINIOUS_WEBAPP_CONFIG" => "/var/www/INGInious/configuration.yaml",
                "REAL_SCRIPT_NAME" => ""
            ),
            "check-local" => "disable"
        ))
    )

    url.rewrite-once = (
        "^/(.*)$" => "/inginious-webdav/$1"
    )
}

The INGINIOUS_WEBAPP and INGINIOUS_WEBDAV prefixed environment variables are used to replace the default command line parameters. See inginious-webdav for more details.

The REAL_SCRIPT_NAME environment variable must be specified under lighttpd if you plan to access the application from another path than the specified one. In this case, lighttpd forces to set a non-root path /inginious-webapp, while a root access if wanted, in order to serve static files correctly. Therefore, this environment variable is set to an empty string in addition to the rewrite rule.

Finally, start the server:

# systemctl enable lighttpd
# systemctl start lighttpd

Using Apache¶

You may also want to use Apache. You should install mod_wsgi. WSGI interfaces are supported through the inginious-webapp script. This guide is made for CentOS 7.x.

Install the following packages (please note that the Python3.5+ version of mod_wsgi is required):

# yum install httpd httpd-devel
# pip3.5 install mod_wsgi

Add the apache user in the necessary groups, to allow it to launch new containers and to connect to mongodb:

# usermod -aG docker apache
# usermod -aG mongodb apache

Create a folder for INGInious, for example /var/www/INGInious, and change the directory owner to apache:

# mkdir -p /var/www/INGInious
# chown -R apache:apache /var/www/INGInious

Put your configuration file in that folder, as well as your tasks, backup, download, and temporary (if local backend) directories (see Configuring INGInious for more details on these folders).

Set the environment variables used by the INGInious CLI scripts in the Apache service environment file (see lighttpd for more details):

# cat  << EOF >> /etc/sysconfig/httpd
INGINIOUS_WEBAPP_CONFIG="/var/www/INGInious/configuration.yaml"
INGINIOUS_WEBAPP_HOST="0.0.0.0"
INGINIOUS_WEBAPP_PORT="80"
EOF
# rm /etc/httpd/conf.d/welcome.conf

Please note that the service environment file /etc/sysconfig/httpd may differ from your distribution and wether it uses systemd or init.

You can then add virtual host entries in a /etc/httpd/vhosts.d/inginious.conf file and apply the following rules:

<VirtualHost *:80>
    ServerName my_inginious_domain
    LoadModule wsgi_module /usr/lib64/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

    WSGIScriptAlias / "/usr/bin/inginious-webapp"
    WSGIScriptReloading On

    Alias /static /usr/lib/python3.5/site-packages/inginious/frontend/static

    <Directory "/usr/bin">
        <Files "inginious-webapp">
            Require all granted
        </Files>
    </Directory>

    <DirectoryMatch "/usr/lib/python3.5/site-packages/inginious/frontend/static">
        Require all granted
    </DirectoryMatch>
</VirtualHost>

<VirtualHost *:8080>
    ServerName my_inginious_domain
    LoadModule wsgi_module /usr/lib64/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so

    WSGIScriptAlias / "/usr/bin/inginious-webdav"
    WSGIScriptReloading On

    <Directory "/usr/bin">
        <Files "inginious-webdav">
            Require all granted
        </Files>
    </Directory>
</VirtualHost>

Please note that the compiled wsgi module path may differ according to the exact Python version you are running.

Updating INGInious¶

Updating the sources¶

To fetch the latest updates on the Git repository master branch :

$ pip3 install --upgrade git+https://github.com/UCL-INGI/INGInious.git

Updating your containers¶

The provided containers can be automatically updated using:

$ inginious-container-update

For your own or third-party containers, please refer to Creating a new container image.

Updating the configuration¶

Most of the time, you won’t need to update your configuration. If something goes wrong, backup your existing configuration file(s) and run inginious-install again. For further details, please refer to inginious-install or Configuration reference.

Updating the database¶

The database scheme may have changed since the last INGInious release. A tool is available to do this migration automatically from your configuration file. Please refer to inginious-database-update.

Configuration reference¶

Hint

The best way to configure INGInious is to use inginious-install. See Configuring INGInious.

Configuring INGInious is done via a file named configuration.yaml or configuration.lti.yaml. To get started, files named configuration.example.yaml and configuration.lti.example.yaml are provided.

The different entries are :

allow_deletion

false if users cannot delete their accounts (and all related data from database), true otherwise.

allow_registration

false if database registration should be disabled. In this mode no password can be set and accounts are only created via the external authentication systems. true otherwise.

backend

The link to the backend used. You can either set it to local or indicate the address of your manually-managed backend.

local. In this mode, which is the default, you have to ensure the docker daemon is local to your machine, or, at least, share the same directory structure. This is typically the case if you use Linux and have a local Docker daemon, or if you use Docker for Mac/Windows, or even docker-machine with local machines. This is the configuration described in this tutorial. You will need a running docker daemon on your machine for this to work. If you can use any Docker client command, like docker info, INGInious should run flawlessly.

In this mode, a supplementary config option is available, local-config.
tcp://xxx.yyy.zzz.aaa:bbbb, udp://xxx.yyy.zzz.aaa:bbbb or ipc:///path/to/your/sock, where the adresses are the ip/socket path of the backend you started manually. This is for advanced users only. See commands inginious-backend and inginious-agent for more information.

backup_directory

Path to the directory where are courses backup are stored in cases of data wiping.

local-config

These configuration options are available only if you set backend:local.

concurrency: Number of concurrent task that can be run by INGInious. By default, it is the number of CPU in your host.
debug_host: Host to which the users should connect in order to access to the debug ssh for containers. Most of the time, just do not indicate this option: the address will be automatically guessed.
debug_ports: Range of port, in the form 64100-64200, to which INGInious can bind SSH debug containers, to allow remote debugging. By default, it is 64100-64200.
tmp_dir: A directory whose absolute path must be available by the docker daemon and INGInious at the same time. By default, it is ./agent_tmp.

log_level

Can be set to INFO, WARN, or DEBUG. Specifies the logging verbosity.

maintenance

Set to true if the webapp must be disabled.

mongo_opt

MongoDB client configuration.

host: MongoDB server address. If your database is user/password-protected, use the following syntax: mongodb://USER:PASSWORD@HOSTNAME/DB_NAME
database: You can change the database name if you want multiple instances or in the case of conflict.

plugins

A list of plugin modules together with configuration options. See Plugins for detailed information on available plugins, including their configuration. Please note that the usage of at least one authentication plugin is mandatory for the webapp.

smtp

Mails can be sent by plugins.

sendername: Email sender name, e.g. : INGInious <no-reply@inginious.org>
host: SMTP server.
port: SMTP port.
username: SMTP username.
password: SMTP password.
starttls: Set to true if TLS is needed.

static_directory

Path to the directory where YAML-defined static pages are located.

superadmins

A list of super-administrators who have admin access on the whole stored content.

tasks_directory

The path to the directory that contains all the task definitions, grouped by courses. (see Creating a new task)

use_minified_js

Set to true to use the minified version of Javascript scripts, false otherwise.

webterm

Link to the INGInious xterm app with the following syntax: http[s]://host:port. If set, it allows to use in-browser task debug via ssh. (See _webterm_setup for more information)

webdav_host

Link to the INGInious webdav app with the following syntax: http[s]://host:port. If set, a new page displays a WebDAV URL and login/password for administrators to access the course filesystem.

sentry_io_url

The Sentry.io Data Source Name to use for error reporting in the frontend. If not set, sentry.io is not loaded.

Plugins¶

Several plugins are available to complete the INGInious feature set.

External authentication plugins¶

You can allow account creation from an external authentication source. This will link the external credentials to the INGInious account so that the user can log in INGInious using these credentials in the future. Several authentication plugins are available.

LDAP¶

Uses an LDAP server to authenticate users.

To enable this plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.auth.ldap_auth
      id: <some_id_for_ldap>
      host: "your.ldap.server.com"
      encryption: "ssl" #can be tls or none
      base_dn: "ou=People,dc=info,dc=ucl,dc=ac,dc=be"
      request: "(uid={})",
      name: "LDAP Login"

Most of the parameters are self-explaining, but:

id: is the authentication method id. It must be alphanumerical and different from other external authentication methods.
request: is the request made to the LDAP server to search the user to authentify. “{}” is replaced by the username indicated by the user.

SAML2/Shibboleth¶

Uses a SAML2-compliant identity provider (such as Shibboleth IdP) to authenticate users.

To enable this plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.auth.saml2_auth
        id: <some_id_for_saml2>
        strict: true
        sp:
            entityId: "<your_entity_id>"
            x509cert: "<your_cert>"
            privateKey: "<your_private_key>"
        idp:
            entityId: "https://idp.testshib.org/idp/shibboleth"
            singleSignOnService:
                url: "https://idp.testshib.org/idp/profile/SAML2/Redirect/SSO"
                binding: "urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect"
            x509cert: "<idp_cert>"
            additionalX509certs:
                - "<idp_cert>"
        security:
             metadataValidUntil: ""
             metadataCacheDuration: ""
        attributes:
             cn: "urn:oid:2.5.4.3"
             email: "urn:oid:1.3.6.1.4.1.5923.1.1.1.6"
             uid: "urn:oid:0.9.2342.19200300.100.1.1"

id is the authentication method id. It must be alphanumerical and different from other external authentication methods. Your IdP is required to provide at least attributes corresponding to the username, the complete name and the email address. Use the attributes entry for the mapping. The additionalX509certs is a plugin-specific entry to specify several certificates in case your IdP is able to use more than one.

This plugin mainly relies on python3-saml package and configuration parameters are interoperable. Please refer to the package documentation for more detailed configuration parameters. The SP Attribute Consuming Service (ACS) is automatically configured by the plugin.

Facebook/LinkedIn/GitHub/Google¶

Uses a Facebook/LinkedIn/GitHub/Google application to allow authentication (and possibly sharing) via the network. You need to create an app on the appropriate developer platform in order to use this plugin.

To enable this plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.auth.facebook_auth
        id: <some_id_for_facebook>
        debug: false
        client_id: <your_app_id>
        client_secret: <your_app_secret>

id is the authentication method id. client_id and client_secret are the OAuth identifier and secret of the created app. Replace facebook_auth by linkedin_auth, github_auth or google_auth according to your case.

Set debug to true to allow OAuth to be run in debug mode (for instance, if SSL is not yet set up).

Twitter¶

Uses a Twitter application to allow authentication and sharing via the network. You need to create two apps on the appropriate developer platform in order to use this plugin. One will only have authentication capabilities and the other one will be able to write posts for the user in order to share results.

To enable this plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.auth.twitter_auth
      id: twitter
      debug: false
      client_id: <app_id_auth_only>
      client_secret: <app_secret_auth_only>
      share_client_id: <app_id_with_share_rights>
      share_client_secret: <app_secret_with_share_rights>
      user: <user_who_created_the_app>

id is the authentication method id. client_id and client_secret are the OAuth identifier and secret of the created app. Set debug to true to allow OAuth to be run in debug mode (for instance, if SSL is not yet set up).

Scoreboard plugin¶

This plugin allows to generate course/tasks scoreboards. To enable the plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.scoreboard

To define a new scoreboard, an additional field scoreboard must be defined in the course.yaml file associated to a course (See Creating a new course). For instance:

scoreboard:
    - content: ["taskid1"]
      name: "Scoreboard task 1"
    - content: ["taskid2", "taskid3"] # sum of both score is taken as overall score
      name: "Scoreboard for task 2 and 3"
    - content: {"taskid4": 2, "taskid5": 3} # overall score is 2*score of taskid4 + 3*score of taskid5
      name: "Another scoreboard"
      reverse: True

This defines three scoreboards for the course. The first one will create a scoreboard for task id taskid1 and will be displayed as Scoreboard task 1. The second one will create a scoreboard for taskid2 and taskid3 where both scores are added. The last one is more complex and will create a reversed scoreboard for task taskid4 and taskid5 where both scores are wieghted by factor 2 and 3, respectively.

The score used by this plugin for each task must be generated via a key/value custom feedback (see feedback-custom) using the score key. Only the succeeded tasks are taken into account.

Contests plugin¶

This plugin allows to manage an ACM/ICPC like contest inside a course between students. To enable the plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.contests

A new configuration page named Contest appears on the administration page. To enable the contest mode, check the Enable contest plugin box on the appropriate course. Please note that the plugin will override the task accessibility dates.

Simple grader plugin¶

This simple grader allows anonymous POST requests without storing submissions in database.

To enable the plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.simple_grader
      courseid : "external"
      page_pattern: "/external"
      return_fields: "^(result|text|problems)$"

courseid is the course id you want to expose to the simple grader.
page_pattern is the URL at which you want to make the simple grader available.
return_fields is a regular expression matching the submission fields that can be returned via the simple grader.

A demonstration POST form will be available at the page_pattern specified URL.

New synchronized job¶

External submissions must take the form of a POST request on the url defined by page_pattern. This POST must contains two data field:

taskid: the task id of the task
input: the input for the task, in JSON. The input is a dictionary filled with problemid:problem_answer pairs.

The return value will contains the standard return fields of an INGInious inginious.backend job plus a “status” field that will contain “ok”.

If an internal error occurs, it will return a dictionary containing

{
    "status": "error",
    "status_message": "A message containing a simple description of the error"
}

New asynchronous job¶

This POST request allows new jobs to be treated asynchronously. It must contains three data fields:

taskid: the task id of the task
input: the input for the task, in JSON. The input is a dictionary filled with problemid:problem_answer pairs.
async: field that indicate that the job must be launched asynchronously. Only have to be present, content is not read.

The return value will be a dictionnary containing:

{
    "status": "done",
    "jobid": "the jobid of the async job. Will be needed to get the results."
}

or

{
    "status": "error",
    "status_message": "A message describing the error"
}

Get status of asynchronous job¶

Given a jobid in input (as field of the POST request) and will return either:

{
    "status": "waiting"
}

or

{
    "status": "error",
    "status_message": "A message describing the error"
}

or

{
    "status": "done",
    "...":"..."
}

where ... are the results of the job, as defined in the return_fields configuration value.

Git Repo plugin¶

This plugin allows saving submissions history in a Git repository, according to the following path pattern : courseid/taskid/username. The version kept in the head of branch is the latest submission made.

To enable this plugin, add to your configuration file:

plugins:
    - plugin_module: inginious.frontend.plugins.git_repo
      repo_directory: "./repo_submissions"

The repo_directory parameter specify the path to the repository that must be initialized before configuration.

JSON task file readers plugin¶

It is possible to store task files in other formats than YAML. However, these plugins are provided for retro-compatibility with previous supported formats, which are deprecated. You therefore use these plugins at your own risks.

To enable the JSON task file format:

plugins:
    - plugin_module: inginious.frontend.plugins.task_file_readers.json_reader

Troubleshooting common problems¶

Solving problems hangs, on OS X with docker-machine or VirtualBox to run Docker¶

There is a known problem with VirtualBox shared folders: it is impossible for the VM to create a unix socket inside (for strange reasons). To solve this, you can mount instead your /Users directory using docker-machine-nfs:

LTI grades are not pushed back using LetsEncrypt¶

INGInious uses PyLTI which uses oauth, oauth2 and libhttp2. The list of certificate authorities known to libhttp2 may be out of day with your host operating system. In particular, as of August 2016, it does not include the LetsEncrypt CA, and thus websites protected with a LetsEncrypt certificate won’t work (you won’t be able to push grades back).

LTI frontend keeps on OAuth errors¶

LTI uses OAuth which uses time-based replay prevention. You need to insure that your webserver (LTI consumer) and LTI producer have reasonably synchronous clocks.

Impossible to get the LTI frontend work¶

You may find `http://ltiapps.net/test/tp.php`_ and `http://ltiapps.net/test/t.php`_ useful when debugging producers and consumers.

It is impossible to modify the course.yaml from the webdav interface¶

Some editors/webdav clients attempt to first move/delete a file before modifying it. It is forbidden to remove or rename the course.yaml, so the modification will fail.

Use simpler editors (such as nano/vim) that directly edit the file rather than doing strange things.

Commands reference¶

Frontend commands¶

inginious-install¶

Assistant to create a complete configuration file for the INGInious frontend.

Read and follow screen instructions.

If you want to configure the frontend by hand, see Configuration reference for a list of entries.

inginious-install [-h] [--file FILE]

--file¶: Specify the configuration file to use. By default, it is configuration.yaml or configuration.lti.yaml, depending on which backend you use

-h, --help¶: Display the help message.

inginious-webapp¶

Start the Web App Frontend. This command can run a standalone web server (see --host and --port options), but also as a FastCGI or WSGI backend.

inginious-webapp [-h] [--config CONFIG] [--host HOST] [--port PORT]

--config¶: Specify the configuration file to use. By default, it is configuration.yaml or configuration.json, depending on which is found first. This can also be specified via the INGINIOUS_WEBAPP_CONFIG environment variable.

--host HOST¶: Specify the host to which to bind to. By default, it is localhost. This can also be specified via the INGINIOUS_WEBAPP_HOST environment variable.

--port PORT¶: Specify the port to which to bind to. By default, it is 8080. This can also be specified via the INGINIOUS_WEBAPP_PORT environment variable.

-h, --help¶: Display the help message.

inginious-webdav¶

Start the Web App Frontend. This command can run a standalone web server (see --host and --port options), but also as a FastCGI or WSGI backend.

inginious-webdav [-h] [--config CONFIG] [--host HOST] [--port PORT]

--config¶: Specify the configuration file to use. By default, it is configuration.yaml or configuration.json, depending on which is found first. This can also be specified via the INGINIOUS_WEBAPP_CONFIG environment variable.

--host HOST¶: Specify the host to which to bind to. By default, it is localhost. This can also be specified via the INGINIOUS_WEBDAV_HOST environment variable.

--port PORT¶: Specify the port to which to bind to. By default, it is 8080. This can also be specified via the INGINIOUS_WEBDAV_PORT environment variable.

-h, --help¶: Display the help message.

Backend commands¶

inginious-agent-docker¶

Start a Docker grading agent. This is typically only used when running the INGInious backend remotely. If you configured INGInious to use a local backend, it is automatically run by inginious-webapp or inginious-lti.

inginious-agent-docker [-h] [--debug-host DEBUG_HOST]
                       [--debug-ports DEBUG_PORTS] [--tmpdir TMPDIR]
                       [--tasks TASKS] [--concurrency CONCURRENCY] [-v]
                       backend

-h, --help¶: Display the help message.

--debug-host DEBUG_HOST¶: The agent hostname for SSH debug. If not specified, the agent autodiscover the public IP address.

--debug-ports DEBUG_PORTS¶: Range of port for job remote debugging. By default it is 64120-64130

--tmpdir TMPDIR¶: Path to a directory where the agent can store information, such as caches. Defaults to ./agent_data

--tasks TASKS¶: The path to the directory containing the courses. Default to ./tasks.

--concurrency CONCURRENCY¶: Maximal number of jobs that can run concurrently on this agent. By default, it is the two times the number of cores available.

-v, --verbose¶: Increase output verbosity: logging level to DEBUG.

backend¶: The backend port, using the following syntax : protocol://host:port. E.g. tcp://127.0.0.1:2001. The agent will connect to the backend listening on that port.

inginious-agent-mcq¶

Start a MCQ grading agent. This is typically only used when running the INGInious backend remotely. If you configured INGInious to use a local backend, it is automatically run by inginious-webapp or inginious-lti.

inginious-agent-mcq [-h] [--tasks TASKS] [-v] backend

-h, --help¶: Display the help message.

--tasks TASKS¶: The path to the directory containing the courses. Default to ./tasks.

-v, --verbose¶: Increase output verbosity: logging level to DEBUG.

backend¶: The backend port, using the following syntax : protocol://host:port. E.g. tcp://127.0.0.1:2001. The agent will connect to the backend listening on that port.

inginious-backend¶

Start an INGInious backend. This is typically only used when running the INGInious backend remotely. If you configured INGInious to use a local backend, it is automatically run by inginious-webapp or inginious-lti.

inginious-backend [-h] [-v] agent client

-h, --help¶: Display the help message.

-v, --verbose¶: Increase output verbosity: logging level to DEBUG.

agent¶: The agents port, using the following syntax : protocol://host:port. E.g. tcp://127.0.0.1:2001. The backend will listen for grading agents on that port.

client¶: The clients port, using the following syntax : protocol://host:port. E.g. tcp://127.0.0.1:2000. The backend will listen for client frontend on that port.

Utilities¶

inginious-synchronize¶

Synchronization tool for INGInious Git repos. Each repository is suppposed to content the files required for a course. When run, the tool pulls the modifications done remotely and force-merge the the local version if conflicts cannot be resolved automatically.

A configuration file synchronize.json must be provided or specified using environment variable INGINIOUS_SYNC_CONFIG. This file contains the main task directory as well as the course identifier, private key for pulls and repo url, as follows:

{
    "maindir":"../tasks",
    "repos":
    [
        {
            "course":"TEST0000",
            "keyfile":"TEST0000.key",
            "url":"git@github.com:user/TEST0000.git"
        },
        {
            "course":"TEST0001",
            "keyfile":"TEST0001.key",
            "url":"git@github.com:user/TEST0001.git"
        }
    ]
}

For more compatibility, please run this command in an ssh-agent session.

Before adding in crontab, add the following lines to .ssh/config for user who runs the scripts :

Host *
    StrictHostKeyChecking no

This tells SSH not to check host keys, we always trust the remote servers

inginious-containers-update¶

Update all the containers created and/or maintained by the INGInious team.

Takes no argument, but needs a properly configured Docker environment. If you use a remote Docker instance, please check the DOCKER_HOST environment variable.

inginious-database-update¶

Update the database to use it with the latest INGInious version.

-h, --help¶: Display the help message.

-c, --config¶: Specify the INGInious config file to use. If not specified, looks for a configuration file in the current directory

Teacher’s documentation¶

Contents:

What is INGInious?¶

INGInious provides a simple and secure way to execute and test untrusted code. It has been developed by the INGI department (Université catholique de Louvain) to automatic grading of programming assignments. The whole tool is written in Python (version 3.5+) and relies on Docker to provide secure execution environments and on MongoDB to keep track of submissions.

INGInious is completely language-agnostic and is able to run anything. Currently, this is limited to Linux programs as only Linux containers are provided and supported.

INGInious also provides an LTI module, allowing its integration to your existing (Open) edX, Moodle,… courses.

How does INGInious work?¶

INGInious is based on the concept of tasks (see Creating a new task). A task is a set of one or more related (sub)questions. For each task, an infinite number of submissions is allowed, but a user must wait for the result of its current submission before trying a new one.

For simplicity, tasks are grouped by courses (see Creating a new course). Usually, an INGInious course has one task per assignment.

A submission is a set of deliverables (chunks of code, files, archives, etc.) that correspond each to one of the (sub)questions of the task. These files are made available to the run file (see Run file), a special script provided by the task. That script is responsible for providing feedback on the submission by compiling, executing or applying any form of checking and testing to the deliverables. In its simplest form, the feedback consists of either success or failed.

This run file is run inside a container (precisely, a grading container), that completely jails the execution of the script, because even teachers and assistants are never fully trusted. Grading containers are able to start sub-containers, called student containers, that runs the scripts that the students sent with their submission, in another jailed environment.

This separation in two step of the grading is mandatory to ensure a complete security for the server hosting INGInious and a complete security of the grading process, making impossible for the student to interact “badly” with the run script.

These containers are created/described by very simple files called Dockerfile. They allow to create containers for anything that runs on Linux. For details about to create new containers and add new languages to INGInious, see Creating a new container image.

Architecture¶

INGInious comes with three distinct parts, the backend (and its agent) and a frontend.

The backend (see ../dev_doc/backend) receives the code of the students and sends it to its agent (see ../dev_doc/agent), which is then responsible to send it to a Docker container, and interact with the request made by the container.

That container then makes some verifications on the submission and returns one of the following four possible status : success, crash, timeout, or failed.

INGInious also provides a frontend (see ../dev_doc/frontend). Made with MongoDB as database, the frontend is in fact an extension of the backend and allows students to work directly on a website. This frontend also provides statistics and management tools for the teachers.

Most of these functionalities can be extended through plugins.

For a more advanced view of the architecture of INGInious, see Understand INGInious.

Docker containers¶

Docker containers are small virtual operating systems that provides isolation between the processes and resources of the host operating system. Docker allow to create and ship any software on any free Linux distribution.

As there are no hypervisor, the processes launched in the container are in fact directly run by the host operating system, which allows applications to be amazingly fast.

Docker allows teachers to build new containers easily, to add new dependencies to the tests applied on the student’s code (see Creating a new container image)

Isolation¶

Isolation allows teachers and system administrators to stop worrying about the code that the students provides.

For example, if a student provides a forkbomb instead of a good code for the test, the forkbomb will be contained inside the container. The host operating system (the computer that runs INGInious) won’t be affected.

The same thing occurs with memory consumption and disk flood. The running time of a code is also limited.

Compatibility¶

INGInious provides two compatibility layers with Pythia v0 and v1. Except the task description file which has to be updated, everything is 100% compatible with INGInious.

Creating a new course¶

Courses are defined by subdirectories found in the tasks directory, which has been specified in the configuration. See Configuration reference. These subdirectories are composed of a course.yaml file describing the course parameters and other subdirectories corresponding to tasks (See Creating a new task).

Here is an example of the content of a tasks folder:

tasks/
    course_id_1/
        course.yaml
        task_id_1/
            task.yaml
            run
            ...
        ...
    ...

Ideally, you should only give permissions to a course folder to the course administrator if needed. The webapp task editor should not require you to give this access. If needed, several methods exist. See inginious-synchronize for Git repository synchronization.

Tutorial¶

Creating courses is reserved to the super-administrators (See Configuration reference). Course administrators are then able to configure the course by themselves.

Note

Demonstration tasks are made available for download here. They can also be downloaded and installed automatically via the inginious-install script.

Using the webapp¶

As a super-administrator, go to the bottom of the course list and enter a new course id, for instance demo, and click on Create new course. A newly created hidden course named demo appears on the list.
Click on that course, and then on Course administration to change the course parameters, add course administrators and tasks.

Please note that, if you give access to the course directory to course administrators, you still have to do some manual work for this to be effective.

Manually¶

The course description is a YAML file containing all the course parameters used by INGInious. Here is a simple course description. Put this file with the name course.yaml in a newly created demo folder in your tasks directory.

name: "[DEMO] Demonstration course"
admins:
- demouser

This elementary course description file will make a new publicly visible course with id demo appear as [DEMO] Demonstration course on the course list.

Course description files¶

Inside the task folder, courses are identified by subdirectories name by their course id and containing a course.yaml file. For instance, this file, for a course with id courseid1, should be placed in a courseid1 subdirectory.

course.yaml is a YAML file containing the course configuration.

admins:
  - demouser
name: "[DEMO] Demonstration course"
tutors: []
groups_student_choice: false
use_classrooms: true
accessible: true
registration: true
registration_password: null
registration_ac: null

While the course.yaml file must be present at the course root dir, all the fields inside are actually only used by the webapp. Here are the possible fields to set:

name Displayed name of the course on the course list.
admins List of administrators usernames. These users will have complete administrations right on the course.
tutors List of tutors usernames (restricted-rights teaching assistants). These users will have read-only rights on the course content. They cannot change course parameters nor tasks, cannot replay submissions or wipe the course data. However, they can manage the classroom composition and download all the student submissions.
accessible When this field is defined, the course is only visible if within the defined period. A course is always accessible to its admins, and is only hidden to normal users, even if they are registered to the course. This field can contain theses values:
true

the task is always accessible;

false

the task is never accessible;

"<start>/<end>"
where <start> and <end> are either empty or valid dates like “2014-05-10 10:11:12” or “2014-06-18”. The task is only accessible between <start> and <end>. If one of the values is empty, the corresponding limit does not apply.

Dates are always considered as a precise instant (to te lowest resolution of the clock). For example, “2014-05-21” is expanded to “2014-05-21 00:00:00”. This means that start limits are inclusive, while end limits are exclusive.

Some examples:
"2014-05-21 / 2014-05-28" "/ 2014-01-01 " # (strictly) before january the first "2030-01-01 /" # opens in 2030 "/" # Always open "/ 2013-12-31 23:59:59" # closes one minute before "/ 2014-01-01"
registration When this field is defined, users can only register to the course between the defined period. It takes the same arguments as accessible.
allow_unregister If this field is defined and set to false, then students are not allowed to auto-unregister from the course.
registration_password A password that is asked upon registration to the course. If empty or not defined, no password will be asked.
registration_ac Access control (AC) method. Can be null (anyone can register), username (filter by username), realname (filter by real name) or email (filter by email address). If AC is activated, the allowed values for the filter should be set in the registration_ac_list key.
registration_ac_list If AC is activated, registration_ac_list should contain a list of values for the filter.
nofrontend If this field is defined and set to true, then the course won’t be displayed on the webapp course list.
groups_student_choice If this field is defined and set to true and if collaborative work is activated for a given task, students will be invited to register by themselves for a group or team before submitting.
use_classrooms If this field is set to true, the classroom model will be used, otherwise, the team model will be used. The default value for this field is true. (See Classrooms and teams)

Creating a new task¶

Tutorial¶

In this document we will describe how to create a simple task, that checks that a code in Python returns “Hello World!”.

Note

Demonstration tasks are made available for download here. They can also be downloaded and installed automatically via the inginious-install script.

Creating the task description¶

Using the webapp¶

If you are using the webapp, this procedure can be done using the graphical interface:

Go to the Course administration/Tasks page, enter helloworld as a new task id and click on Create new task.
In the Basic settings tab, set the task name to Hello World! and put some context and author name. Container setup can be left with default parameters.
In the Subproblems tab, add a new code-type problem with problem id question1.
Set some problem name and context, and set language to python.
Save changes and go to Task files tab.

Manually¶

This is only possible if the administrator has given access to the course directory to the course administrator.

The task description is a YAML file describing everything that INGInious needs to know to verify the input of the student. Here is a simple task description. Put this file with the name task.yaml in a newly created helloworld folder in your course directory.

author: "The INGInious authors"
accessible: true
name: "Hello World!"
context: "In this task, you will have to write a python script that displays 'Hello World!'."
problems:
    question1:
        name: "Let's print it"
        header: "def func():"
        type: "code"
        language: "python"
limits:
    time: 10
    memory: 50
    output: 1000
environment: default

Most of the fields are self-explanatory. Some remarks:

The field problems is a dictionary of problems. Each problem must have an unique id, for example here question1.
Problem question1 have its type field that equals to code, which means the student must enter some code to answer the question. Other types exists, such as multiple-choice.
The field limits are the limits that the task cannot exceed. The time is in seconds, and memory and output are in MB.
The environment field is intended to change the environment where the tasks run. The available environments are those you downloaded during installation or those you created by creating a grading container. Please see Creating a new container image.

More documentation is available here: Creating a new task.

Creating the run file¶

In your task folder, you will put every file needed to test the input of the student. This folder content can be shown in the webapp in the Task files tab of the Edit task page.

Create a template file template.py, where we will put the code of the student.
```
def func():
    @    @question1@@

    func()
```
The syntax is very simple: put a first @ on the line where you want to put the code of the student. Then indent the line and write a second @. Now write the problem id of the problem you want to take the input from (question1) then write another @, write a possible suffix (not used here), and then finish the line with a last @.
Create the run file. This file will be the script that is launched when the task is started. Here we will create a bash script, that parses the template and verifies its content.

Here we use four commands provided by INGInious, parse_template, run_simple, set_global_result and set_global_feedback. The code is self-explanatory; just notice the usage of run_student_simple (a version of run_student) that ask INGInious (precisely the Docker agent) to start a new student container and run inside the command python studentcode.py.

Please note that the run_student_simple command is fully configurable: you can change the environment on which you run the task, define new timeouts, memory limits, … See run_student for more details.
If not using the webapp, don’t forget to give the run file the execution rights:
```
$ chmod +x helloworld/run
```

More documentation is available here: Run file.

Run file¶

When the student have submit his/her code, INGInious starts a new Docker container with the right environment for the task (as given in the task.yaml file). Inside this container is launched a script, that we call the run script, that you have to provide in the directory of your task.

Naming your run script¶

Tip

TL;DR: name your script run.py if you use Python or run.sh if you use bash. You can put a run script inside the common folder of a course, and it will be used by default if no run file exists in a task.

The file chosen by INGInious as your run file is dependent on the environment. Here is how INGInious resolves the best file to run (the resolution ends a soon a one of the rule is respected):

If a custom run command was provided, this command is run.
If a file named `run` exists in the task folder, this file is run. Note that in this case the file will need a shebang (see below)
Depending on the container environment, files named `run.EXT` in the task folder, where EXT is a container-specific extension, will be run.

Container EXT Language

all py3 IPython3

all py IPython3

all sh bash

It is possible to add your own extensions/languages in new containers.
If a file named `run` exists in the common course folder, this file is run. Note that in this case the file will need a shebang (see below)
Depending on the container environment, files named `run.EXT` in the common course folder, where EXT is a container-specific extension, will be run. See table above.

Tip

If you simply name you run script `run`, don’t forget to indicate which interpreter should be used to execute the script. To do that, use a shebang:

#!/bin/bash
feedback-result success

Here is a simple example of a run.py file, compatible with the default environment, that simply returns that the student’s code is OK:

set_global_result("success")

This is actually an IPython code.

In general, the run script is simply an executable application (a bash script, a python script, or a compiled executable runnable by the container). INGInious’ default containers provides commands (also available as python libraries) to interact with the backend.

By default, the script is run by a non-root user. You can modify the container to change this (and everything else).

Python run scripts are run with IPython¶

IPython is a Python interpreter that adds some very useful features to Python, notably magic commands.

The main feature that you will use is probably the bang (!) magic command, that allows your to run a command like if you were in bash (or any “basic” shell):

# run a command
! touch hello.txt

# you can store the output, as an array of line
out = !ls -1

# we are still in python
length = len(out)
print(length, ";".join(out))

By default, the INGInious version of IPython loads utility libraries of INGInious (feedback, input, lang, rst) into the global namespace, so you don’t have to.

If you want to use the INGInious IPython interpreter in another script, the interpreter is located at /bin/inginious-ipython.

Check the API documentation¶

The prefered way to create run script is via the IPython interpreter. The full description of the API available from Python is available here.

Feedback commands¶

feedback-result¶

The feedback-result command sets the submission result of a task, or a problem.

# set the global result
set_global_result("success")  # Set global result to success

# set the result of a specific suproblem
set_problem_result("failed", "q1")  # Set 'q1' subproblem result to failed

from inginious_container_api import feedback

# set the global result
feedback.set_global_result("success")  # Set global result to success

# set the result of a specific suproblem
feedback.set_problem_result("failed", "q1")  # Set 'q1' subproblem result to failed

# format: feedback-result [-i|--id PROBLEM_ID] RESULT
feedback-result success  # Set global result to success
feedback-result -i q1 failed  # Set 'q1' subproblem result to failed

The execution result can be of different types:

success : the student succeeded the task
failed : there are error in the student answer
timeout : the tests timed out
overflow :there was a memory/disk overflow
crash : the tests crashed

Any other type will be modified to “crash”.

feedback-grade¶

The feedback-grade command sets the submission grade.

set_grade(87.8) # Set the grade to 87.8%

from inginious_container_api import feedback
feedback.set_grade(87.8) # Set the grade to 87.8%

# format: feedback-grade GRADE
feedback-grade 87.8

If no grade is specified (i.e. the command is never called), the result score will be binary. This means that a failed submission will give a 0.0% score to the student, while a successful submission will give a 100.0% score to the student.

feedback-msg-tpl¶

The feedback-msg-tpl sets the feedback message associated to the task or a subproblem, using a Jinja2 <http://jinja.pocoo.org/docs/2.9/> template.

It needs the name of a template. The command attempt to use a translated version of the template first; given that you give TPLNAME as first argument to the command, feedback-msg-tpl will attempt to find the template, by search in this order:

[local_dir]/TPLNAME.XX_XX.tpl
[task_dir]/lang/XX_XX/TPLNAME.tpl (preferred way)
[local_dir]/TPLNAME.tpl

Once found, the template is parsed using Jinja2 <http://jinja.pocoo.org/docs/2.9/>, which allows you to send parameters to the template.

# format:
# set_feedback_from_tpl(template_name, template_options, problem_id=None, append=False)
# template_name is the file to format. See above for details.
# template_options is a dict in the form {name: value}. See below
# problem_id is the problem id to which the feedback must be assigned. If None, the feedback is global
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

set_feedback_from_tpl("feedback.tpl", {"option1":"value1", "anothername":"anothervalue"})

from inginious_container_api import feedback

# format:
# feedback.set_feedback_from_tpl(template_name, template_options, problem_id=None, append=False)
# template_name is the file to format. See above for details.
# template_options is a dict in the form {name: value}. See below
# problem_id is the problem id to which the feedback must be assigned. If None, the feedback is global
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

feedback.set_feedback_from_tpl("feedback.tpl", {"option1":"value1", "anothername":"anothervalue"})

# format: feedback-msg-tpl [-a | --append] [-i | --id PROBLEM_ID] TPLNAME [option1=value1 option2=value2 ...]
# TPLNAME is the file to format. See above for details.
# Options can be indicated at the end of the command, and will be passed to the template (see below)
# --append is a boolean flag indicating if the feedback must be appended or not (overwritting the current feedback)
# --id PROBLEM_ID. PROBLEM_ID is the problem id to which the feedback must be assigned.
#                  If not indicated, the feedback is global

feedback-msg-tpl "feedback.tpl" option1=value1 anothername=anothervalue

Inside your template (named feedback.tpl in the examples above), you can use these parameters like this:

Option 1 was {{ option1 }} and the option 2 was {{ anothername }}

Which will return

Option 1 was value1 and the option 2 was anothervalue

See the Jinja2 documentation to discover all possibilities.

Your template must return a valid RestructuredText.

feedback-msg¶

The feedback-msg command sets the feedback message associated to the task or a subproblem.

# format:
# set_global_feedback(feedback, append=False)
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

set_global_feedback(
    """This is the correct answer.

    Well done!"""
)

# format:
# set_problem_feedback(feedback, problem_id, append=False)
# problem_id is the problem id to which this feedback must be associated
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

set_problem_feedback(
    """This is the correct answer.

    Well done!"""
, "q1")

from inginious_container_api import feedback

# format:
# set_global_feedback(feedback, append=False)
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

feedback.set_global_feedback(
    """This is the correct answer.

    Well done!"""
)

# format:
# set_problem_feedback(feedback, problem_id, append=False)
# problem_id is the problem id to which this feedback must be associated
# append is a boolean indicating if the feedback must be appended or not (overwritting the current feedback)

feedback.set_problem_feedback(
    """This is the correct answer.

    Well done!"""
, "q1")

feedback-msg -ae -m "This is the correct answer.\n\nWell done!"

# It has several
# optional parameters:
#
# -a, --append                        append to current feedback, if not specified, replace the
#                                     current feedback.
# -i, --id PROBLEM_ID                 problem id to which associate the feedback, leave empty
#                                     for the whole task.
# -e, --escape                        interprets backslash escapes
# -m, --message MESSAGE               feedback message
# If the message is not specified, the feedback message is read from stdin.

feedback-custom¶

The feedback-custom command sets a pair of key/value custom feedback, mainly used with plugins.

# format: set_custom_value(key, value)
# Please refer to the plugin documentation to know which value you have to set for ``key`` and ``value`` parameters.
# value can be anything that can be encoded to JSON by the default python library.
set_custom_value("score", 56) # Set the `score` key to value 56

# format: set_custom_value(key, value)
# Please refer to the plugin documentation to know which value you have to set for ``key`` and ``value`` parameters.
# value can be anything that can be encoded to JSON by the default python library.
feedback.set_custom_value("score", 56) # Set the `score` key to value 56

# format: feedback-custom [-j|--json] key value

# The ``--json`` parameter indicates if ``value`` must be parsed as a JSON string.
# Please refer to the plugin documentation to know which value you have to set for ``key`` and ``value`` parameters.

# For instance, the following command set the value ``56`` to the ``score`` key:
feedback-custom score 56

tag-set¶

The tag-set command sets the value of the tag specified by the tag identifier to True or False.

# format: set_tag(tag, value):
# Set the tag 'tag' to the value True or False.
# :param value: should be a boolean
# :param tag: should be the id of the tag. Can not starts with '*auto-tag-'

# For instance, the following command set the value of the ``my_tag`` tag to ``True``:
set_tag("my_tag", True)

from inginious_container_api import feedback

# format: set_tag(tag, value):
# Set the tag 'tag' to the value True or False.
# :param value: should be a boolean
# :param tag: should be the id of the tag. Can not starts with '*auto-tag-'

# For instance, the following command set the value of the ``my_tag`` tag to ``True``:
feedback.set_tag("my_tag", True)

# format: tag-set tag value

# For instance, the following command set the value of the ``my_tag`` tag to ``True``:
tag-set my_tag true

tag¶

The tag command defines a new unexpected tag to appear in the submission feedback.

# format: set_tag(tag, value):
# Set the tag 'tag' to the value True or False.
# :param value: should be a boolean
# :param tag: should be the id of the tag. Can not starts with '*auto-tag-'

# # For instance, the following command defines a new ``A new tag`` tag that will appear in the submission feedback:
tag("A new tag") # Sets a new unexpected tag

from inginious_container_api import feedback

# format: set_tag(tag, value):
# Set the tag 'tag' to the value True or False.
# :param value: should be a boolean
# :param tag: should be the id of the tag. Can not starts with '*auto-tag-'

# # For instance, the following command defines a new ``A new tag`` tag that will appear in the submission feedback:
feedback.tag("A new tag") # Sets a new unexpected tag

# format: tag value

# For instance, the following command defines a new ``A new tag`` tag that will appear in the submission feedback:
tag "A new tag"

reStructuredText helper commands¶

Several helper commands are available to format the feedback text, which format is reStructuredText.

rst-code¶

The rst-code command generates a code-block with the specified code snippet and language to enable syntax highlighting.

codeblock = get_codeblock("java", "int a = 42;") # Java codeblock with `int a = 42;` code

set_global_feedback(codeblock, True) # Appends the codeblock to the global feedback

from inginious_container_api import rst, feedback

codeblock = rst.get_codeblock("java", "int a = 42;") # Java codeblock with `int a = 42;` code

feedback.set_global_feedback(codeblock, True) # Appends the codeblock to the global feedback

# format: rst-code [-l | --language LANGUAGE] [-e | --escape] [-c | --code CODE]

# -l, --language LANGUAGE    snippet language, leave empty to disable syntax highlighting
# -e, --escape               interprets backslash escapes
# -c, --code CODE            snippet code

# If the code parameter is not specified, it is read on standard input. The result is written on standard output.
# For instance, the command can be used as follows:
cat test.java | rst-code -l java | feedback-msg -a

rst-image¶

The rst-image command generates a raw reStructuredText block containing an image to display.

# get_imageblock(filename, format='')
imgblock = get_imageblock("smiley.png") # RST block with image
set_global_feedback(imgblock, True) # Appends the image block to the global feedback

from inginious_container_api import rst, feedback

# get_imageblock(filename, format='')
imgblock = rst.get_imageblock("smiley.png") # RST block with image
feedback.set_global_feedback(imgblock, True) # Appends the image block to the global feedback

# format: rst-image [-f|--format FORMAT] FILEPATH

# Appends the image block to the global feedback
rst-image smiley.png | feedback-msg -a

The optional format parameter is used to specify the image format (jpg, png,…) if this is not explicitly specified the the image filename. The output is written on the standard output. For instance, the command can be used as follows:

get_admonition / rst-msgblock¶

The get_admonition (python) / rst-msgblock (bash) command is used to generate a reStructuredText admonition in a specific colour according to the message type.

You must indicate a type for the admonition (via the first arg in Python, or via the -c arg in bash). The type can be:

success (green box)
info (blue box)
warning (orange box)
danger (red box)

You can also indicate a title (second parameter in Python, -t in bash). It can be empty.

# RST message block of class "success" and title "Yeah!"
admonition = get_admonition("success", "Yeah!", "Well done!")
set_global_feedback(admonition, True) # Appends the block to the global feedback

from inginious_container_api import rst, feedback

# RST message block of class "success" and title "Yeah!"
admonition = rst.get_admonition("success", "Yeah!", "Well done!")
feedback.set_global_feedback(admonition, True) # Appends the block to the global feedback

# format: rst-image [-c | --class CSS_CLASS] [-e | --escape] [-t | --title TITLE] [-m | --message MESSAGE]
# -c, --class CSS_CLASS    Type (Bootstrap alert CSS class). See above for details.
# -e, --escape             interprets backslash escapes
# -t, --title TITLE        message title
# -m, --message MESSAGE    message text
# If the message parameter is not set, the message is read from standard input.

rst-msgblock -c info -m "This is a note" | feedback -ae

rst-indent¶

The rst-indent command is used to add indentation to a given text.

rawhtml = indent_block(1, "<p>A paragraph!</p>", "\t") # Indent the HTML code with 1 unit of tabulations
set_global_feedback(".. raw::\n\n" + rawhtml, True) # Appends the block to the global feedback

from inginious_container_api import rst, feedback

rawhtml = rst.indent_block(1, "<p>A paragraph!</p>", "\t") # Indent the HTML code with 1 unit of tabulations
feedback.set_global_feedback(".. raw::\n\n" + rawhtml, True) # Appends the block to the global feedback

# format: rst-image [-c | --class CSS_CLASS] [-e | --escape] [-t | --title TITLE] [-m | --message MESSAGE]
# -e, --escape                      interprets backslash escapes
# -c, --indent-char INDENT_CHAR     indentation char, default = tabulation
# -a, --amount AMOUNT               amount of indentation, default = 1
# -m, --message MESSAGE             message text

# If the message parameter is not set, the message is read from standard input.

# For instance, the command can be used as follows, to add an image to the feedback,
# (inside a list item, for instance):
rst-msgblock -c info -m "This is a note" | feedback -ae

The amount of indentation can be negative to de-indent the text.

Input commands¶

get_input¶

The get_input command/function returns the input given by the student for a specific problem id. For example, for the problem id “pid”:

thecode = get_input("pid")

from inginious_container_api import input
thecode = input.get_input("pid")

getinput pid

When a problem is defined with several boxes, the argument becomes pid/bid where “pid” stands for the problem id and “bid” for “box id”. If the problem is a file upload, the problem id can be appended with :filename or :value to retrieve its filename or value.

Note that get_input can also retrieve the username/group of the user that submitted the task. You simply have to run

username = get_input("@username")

username = input.get_input("@username")

getinput @username

If the submission is made as a user, it will contain the username. It it’s made as a group, it will contain the list of the user’s usernames in the group, joined with ‘,’.

The four letter code of the student’s language (for example en_US or fr_FR) can also be retrieved using

lang = get_input("@lang")

lang = input.get_input("@lang")

getinput @lang

Note that plugins are free to add new @-prefixed fields to the available input using the new_submission hook.

parsetemplate¶

The parsetemplate command injects the input given by the student in a template.

A template file must be given to the function/command. An output file can also be given, and if none is given, the template will be replaced.

thecode = parse_template("student.c") # Parse the `student.c` template file
thecode = parse_template("template.c", "student.c") # Parse the `template.c` template file and save the parsed file into `student.c`

from inginious_container_api import input
thecode = input.parse_template("student.c") # Parse the `student.c` template file
thecode = input.parse_template("template.c", "student.c") # Parse the `template.c` template file and save the parsed file into `student.c`

# parsetemplate [-o|--output outputfile] template
parsetemplate "student.c" # Parse the `student.c` template file
parsetemplate -o "student.c" "template.c" # Parse the `template.c` template file and save the parsed file into `student.c`

The markup in the templates is very simple: @prefix@problemid@suffix@. Prefix allows to correct the indentation when needed (this is useful in Python).

Example of template file (in java)

public class Main
{
    public static void main(String[] args)
    {
@        @problem_one@@
    }
}

To access the filename and text content of a submitted file, the problemid can be followed by a :filename or :value suffix.

run_student¶

run_student allows the run file to start, at will, sub-containers. This makes you able to secure the grading, making sure the untrusted code made by the student don’t interact with yours.

The sub-container is launched with a different user who has read-write accesses to the task student subdirectory. Only the changes made in that directory will remain in the main container.

run_student is fully configurable; you can change the container image (environment), set new timeouts, new memory limits, … And you can call it as many time as you want.

Here is the list of the main parameters:

container (–container in the run_student command)

Name of the container to use. The default is the same as the current container.
time limit (–time)

Timeout (in CPU time) for the container, in seconds. The default is the same as the current container.
hard time limit (–hard-time)

Hard timeout for the container (in real time), in seconds. The default is three times the value indicated for the time limit.
memory limit (–memory)

Maximum memory for the container, in Megabytes. The default is the same as the current container.
network sharing (–share-network)

Share the network stack of the grading container with the student container. This is not the case by default. If the container container has network access, this will also be the case for the student!

Beyond these optionals args, run_student various commands also takes an additional (mandatory) argument: the command to be run in the new container.

More technically, please note that:

the run_student command (accesible in bash) proxies stdin, stdout, stderr, most signals and the return value
There are special return values:
- 252 means that the command was killed due to an out-of-memory
- 253 means that the command timed out
- 254 means that an error occurred while running the proxy

In Python, two flavours of run_student are available: run and run_simple. The first is a low-level function, which allows you to modify most of the behavior of the behavior of the function. The second aims to solve the most used use case: run a command with a given input, and returns its output. A small description of run_simple is available in the examples below, please check the API directly for more information.

# runs student/script.sh in another safe container, with a timeout of 60 seconds,
# and stores the output in the variables `stdout` and `stderr`, and the return value
# inside the variable `retval`.
stdout, stderr, retval = run_student_simple("student/script.sh", time=60)

from inginious_container_api import run_student

# runs student/script.sh in another safe container, with a timeout of 60 seconds,
# and stores the output in the variables `stdout` and `stderr`, and the return value
# inside the variable `retval`.
stdout, stderr, retval = run_student.run_student_simple("student/script.sh", time=60)

# runs student/script.sh in another safe container, with a timeout of 60 seconds,
# and stores the output in the variable `output`, as an array of lines.
output=`run_student --time 60 student/script.sh`

Archiving files¶

The folder /archive inside the container allows you to store anything you may need outside the container. The content of the folder will be automatically compressed and saved in the database, and will be downloadable in the INGInious web interface.

This feature is useful for debug purposes, but also for analytics and for more complex plugins.

Task description files¶

Inside a course folder (see Creating a new course), tasks are identified by subdirectories named by their task id and containing a task.yaml file. For instance, this file, for a task with id taskid1, should be placed in a taskid1 subdirectory.

task.yaml is a YAML file containing information about the task.

author: Your name
context: |-
    The context of this task. Explain here what the students have to do.
order: 1
groups: false
name: The complete name of this task
accessible: true
problems:
    a_problem_id:
        name: The title of this question
        header: A header for this question
        type: code
        language: c
limits:
    time: 30
    memory: 128
environment: default
network_grading: False

author, context, order, name, language and header are only needed if you use the frontend. context and header are parsed using restructuredText.
order is an integer, used by the frontend to sort the task list. Task are sorted in increasing value of order.
weight is a decimal value indicating the weight of the task score to use to compute the total course score.
accessible describes when the task is accessible to student. This field is not mandatory (by default, the task is visible) and can contain the following values:

true

the task is always accessible

false

the task is never accessible

"START"

where START is a valid date, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible after START.

"/END"

where END is a valid date, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible before END.

"START/END"

where START and END are valid dates, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible between START and END.

"START/SOFT_DEADLINE/END"

where START, SOFT_DEADLINE and END are valid dates, like “2014-05-10 10:11:12”, or “2014-06-18”. The task is only accessible between START and END, but the publicly communicated end time is SOFT_DEADLINE.
problems describes sub-problems of this task. This field is mandatory and must contain at least one problem. Problem types are described in the following section Problem types. Each problem must have an id which is alphanumeric and unique.
limits contains the limits that will be applied on the grading container. time is the CPU timeout in seconds, and hard_time is the timeout in real time.

By default, hard_time is defined to be to 3*``time``. This can leads to problems when INGInious is under heavy load, but allow to detect processes that do too much system interruptions (sleep calls or IO)

memory is the maximum memory allowed to the container.

Please note that the limits of the student containers (container that you start inside the grading container) will use these limits by default.
environment is the name of the Docker container in which the grading code will run. This field is only needed if there is code to correct; a multiple-choice question does not need it. This environment will be used by default for the student containers.
groups allows to indicate if the submission is to be done individually or per groups/teams. (see Classrooms and Teams).
network_grading indicates if the grading container should have access to the net. This is not the case by default.
evaluate indicates the submission that must be used for evaluation. This can be either:

best

This is the default value. In this case, the best submission is used.

last

In this case, the last submission is used.

student

In this case, the student can select the submission for evaluation. This allows student to select the submission for evaluation without submitting it again or if submission replays are planned. This feature is not available in the LTI module due to LTI specifications limitations, and will be considered as best submission.
submission_limit indicates the amount of submissions a student can make within a certain period of time. It is composed of two fields:
- amount is an integer value indicating the amount of submission. A value of -1 corresponds to an infinite amount of submissions.
- period is an integer value indicating the length of the submission period in hours. A value of -1 corresponds to an infinite period. At the end of this period, the student can submit amount submissions again during period hours.
stored_submissions indicates the amount of submissions that must be saved in the submission history. A value of 0 keeps all the submissions.

Problem types¶

Code problems¶

type: code problems allows students to submit their code. The code is then sent to a container where a script made by the teaching team corrects it.

Here is a simple example for a code problem

type: code
language: c
header: |-
    Hello dear student!
    I'm a multiline header!
name: A name
optional: false

header and language are only needed when using the frontend and are not mandatory. This description typically displays on the frontend a box where student can put their code.

optional is an optional field, that defaults to false, that indicates if this problem is mandatory or not.

Code problem input’s are available in the run script (see Run file) directly with the id of the problem.

Single code line problems¶

type: code_single_line is simply a code box that allows a single line as input.

type: code_single_line
language: c
header: |-
    Hello dear student!
    I'm another multiline header, parsed with *RST*!
name: Another problem
optional: false

Single line code problem input’s are available in the run script (see Run file) directly with the id of the problem.

Advanced code problem¶

Advanced code problems are available:

type: code
header: some text
name: And again, another name
boxes:
    boxId1:
        type: text
        content: Some additional text
    boxId2:
        type: input-text
        maxChars: 10
        optional: true
    boxId3:
        type: multiline
        maxChars: 1000
        lines: 8
        language: java

Boxes are displayable (on the frontend) input fields that allows the student to fill more than one entry per problem. Different box types are available, all of them are demonstrated above. Every configuration in the boxes (maxChars,*lines*,*language*) is not mandatory, except content if the box type is text, and the field optional (default to false), that indicates if the box is mandatory or not.

In the run file (see Run file), boxes input are available with the name problem_id/box_id

Match problems¶

Match problem are input that allows a single-line input from the student and that returns if the student entered exactly the text given in the “answer” field.

name: The answer
type: match
header: some text describing this problem
answer: 42

Match problem input’s are available in the run script (see Run file) directly with the id of the problem.

Multiple choice problems¶

name: An exercice
type: multiple_choice
header: The answer to life, the universe and any other things is
multiple: true
limit: 2
error_message: "Wrong answer. Don't panic, and read Hitchhiker's Guide to the Galaxy."
success_message: "You're right! But don't forget to always take your towel with you."
choices:
  - text: It is, of course, 42!
    valid: true
  - text: It should be *42*
    valid: true
  - text: 43!
    feedback: "43 isn't the answer. Maybe can you try to substract one?"
  - text: 41?
    feedback: "41 isn't the answer. Maybe can you try to add one?"

Choices are described in the choices section of the YAML. Each choice must have a text field (on the frontend) that will be parsed in restructuredText. Valid choices must have a valid: true field. The field feedback is a message that will be displayed when the student check the choice.

multiple indicates if the student may (or not) select more than one response.

Choices are chosen randomly in the list. If the limit field is set, the number of choices taken equals to the limit. There is always a valid answer in the chosen choices.

error_message and success_message are messages that will be displayed on error/success. They are parsed in RST and are not mandatory.

Multiple choice problem input’s are available in the run script (see Run file) directly with the id of the problem. The input can be either an array of integer if multiple is true or an integer. Choices are numbered sequentially from 0.

Allowing students to download files¶

Files stored in the subdirectory “public” of a task are available to download from the frontend.

For example, if you have a course with course id “courseA”, and a task with task id “taskB”, and if you want to distribute an image named “flower.png”, you will have to put it inside the folder “/path/to/INGInious/tasks/courseA/taskB/public/”.

The image will then be available with the url: http://domain.name.com/course/courseA/taskB/flower.png

This allows you to insert images inside the context of your tasks, and to share additional resources like datasets or slides.

Danger

The files stored in your public folder will be available to all users, without authentication needed.

Random tasks parameters¶

Typical INGInious tasks are displayed the same way for all the students. However, it is possible to display some parameters randomly for students to avoid copy/pasting their peers submissions. This can, for instance, be a numerical value for a computation, or the name of a variable.

INGInious allows you to generate one or several random numbers that are stored in database per student. To enable this, specify the number of random numbers (of parameters) you need in the task editor Basic settings tab. By default, the same set of random numbers is kept per student. If you want to generate random numbers each time the student opens the task, check the Regenerate random input box.

Accessing the task random parameters¶

The random numbers generated for the task are accessible through the webapp as well as in the INGInious container and run file.

Through the webapp¶

Several context-specific inputs are accessible through a input Javascript dictionary in the task and subproblems context.

var input = {
    "@lang": "fr",
    "@username": "foobar",
    "@random": [0.9805443622648311, 0.8755252481699163],
    "@state": ""
}

The @random key contains the random number list. In order to use it, declare a ..raw:: html directive in your contexts and include some Javascript code via the HTML <script> tag.

Through the container¶

The random number list can be accessed using the getinput API with the specific id @random.

Using the shell API:

getinput @random

Using the Python API:

from inginious import input as inginious_input
random_list = inginious_input.get_input("@random")

Overriding system files¶

For some particular tasks (that involve networking), it may be necessary to overwrite/append to some system files, such as /etc/hosts and /etc/resolv.conf. In order to do this, create a folder systemfiles in your task, and create these files according to what you want to do:

systemfiles/hosts the content of this file is append to the /etc/hosts of the container at start. systemfiles/resolv.conf the content of /etc/resolv.conf is replaced by the content of this file if it exists.

These files are also used in containers created by run_student.It is even possible to write dynamically on systemfiles/xxx while the grading script is running, and new changes (made before run_student) will be set into the new container.

The grading container can also write directly on its /etc/hosts and /etc/resolv.conf, but it is discouradged as it will not be reflected in student containers.

Debugging tasks¶

There are different ways to get more insight on what’s going wrong with your tasks in case of errors. Generally, those errors come from the fact the running environment may be different from your task development environment (OS configuration, software set,…).

Debug information¶

Every time an administrator submit a new job and receives result on the frontend, a Debug information is made available in the sidebar. Those information contain all the submission metadata (input, result, feedback,…) as well as the grading container standard output and standard error output.

Please note that every manipulation done with those streams will not be visible anymore in those information. Redirected output won’t be shown. This is important as spawning processes in non-shell oriented languages will not redirect the spawned process standard output on the grading container standard output.

SSH debug¶

Debugging tasks is made more easy using SSH debug feature. This aims at providing the same user experience as local development. To make this feature work remotely (regarding the INGInious Docker agent), please make sure you’ve correctly set up the debug hosts and ports (see Configuration reference if needed).

Every administrator is able from the frontend to launch a debugging job. This is done by clicking on the >_ (left-chevron, underscore) button next to the Submit button. According to your configuration, either a SSH command-line with auto-generated password will be given you (you will, in this case, need an SSH client installed), or an embedded SSH console will pop up as the feedback position.

Best practices¶

Use YAML with RST¶

YAML with RST is the recommended way to create task descriptions. Old methods such as JSON+RST, JSON+HTML and RST+RST are deprecated and were dropped. If you really want to use them, these parsing methods are still available as plugins.

Provide a test set for the student to test their code themselves, and grade the code with at least this set¶

This will allow the students to make sure that they submit properly. If a test set that is public works on the student’s computer but does not work on INGInious, this is the sign of a greater problem. Give them a complete feedback on these public tests, but hide the rest to keep the difficulty of the task.

Do not use the “file” subproblem type¶

When you ask student to produce some code, you should always put a “code” subproblem, where the student can copy-paste his code. INGInious is made to receive text as input, no (huge) binary files.

Using Jar, Zip, Tar+Gz and other formats to convey code in binary format is most of the time convenient for the task writer, but it’s not the case for the students; tools for compressing files are not always consistent among all OS, and even inside the same OS, leading to errors that could be easily avoided using text input.

It is also prone to other types of errors: when submitting a project with a zip file or a jar, task writers ask most of the time to resubmit the complete project, with other files that may seem uneeded by the grader and that was maybe modified by the student. There is then two possibilities:

either the task writer effectively uses these files, leading to an error that is not really due to the student
or the task writer does not use them, and there is a waste of disk space

These “common files” should be located not in the code submitted by the student, but directly in the task folder.

Using “code” inputs forces you to only ask the students to submit what is really needed, improving the quality of the tasks. Zip, Jar and other archive formats should only be used when grading huge projects.

If you still have to use a “file” subproblem type, make it lightweight¶

For example, when submitting a Jar file (type of archive for languages that uses the JVM), please do not ask student to add common libraries (such as Scala) in their submission. These libraries can be installed in the task folder, or even better, directly in the container.

INGInious reloads most of the time the whole submission when it needs to read something inside, and reloading a file of 20M can take time.

Students should only have to upload files they modified, and not files common to all submissions.

If you still have to use a lightweight “file” subproblem, ask for code, not for binary executables¶

Asking for binary executable (or JVM bytecode, etc.) is a bad practice from a general point of view: * You cannot read the code of the students to check the grading * Small differences in libs used on the system and in the container can lead to errors. Even in the JVM. * It will force the grader to recompile the code of the student, allowing to make more checks

You should always ask only the code (no binary executable, no JVM .class file, …) and always recompile everything.

Do not delete tasks¶

Instead of deleting a task, it is better to make it unavailable, using the accessible field.

For example, if 2014-05-11 is the date where the task became unavailable:

{
        ...
        "accessible": "/2014-05-11",
        ...
}

Do not put the student’s code directly in your tests¶

Inserting student’s code directly in your tests is dangerous, because the student could make syntax errors that would display the code of your tests.

It is better to put all the student’s functions in a single template file, which one you will import in you test files.

Use student container¶

To be completely secure, run anything you do not trust inside a separate student container. Students may want to interfere with the normal work of your grading script to get better grades…

Make small tests¶

Inside your tasks, do not test everything in a single test file. Using a file for each test is a good practice, and will allow you to debug your code efficiently, while providing students fine-grained error descriptions.

Try to use unit-test libraries provided by your programming language.

The inginious_container_api package (commands inside containers)¶

This package contains all methods accessible from inside the containers.

In the default shell of the `run` scripts, all these function are included in the global namespace.

Gathering the student’s input¶

inginious_container_api.input.get_input(problem)[source]¶: ” Returns the specified problem answer in the form problem: problem id Returns string, or bytes if a file is loaded

inginious_container_api.input.get_lang()[source]¶: Returns the language of the current user (as a two-character ISO-639-1 code) being graded

inginious_container_api.input.get_username()[source]¶: Returns the username of the current user being graded

inginious_container_api.input.parse_template(input_filename, output_filename='')[source]¶

Parses a template file Replaces all occurences of @@problem_id@@ by the value of the ‘problem_id’ key in data dictionary

input_filename: file to parse output_filename: if not specified, overwrite input file

Feedback¶

inginious_container_api.feedback.get_feedback()[source]¶: Returns the dictionary containing the feedback

inginious_container_api.feedback.save_feedback(rdict)[source]¶: Save feedback file

inginious_container_api.feedback.set_custom_value(custom_name, custom_val)[source]¶: Set a custom value to be given back in the feedback :param custom_name: name/key of the entry to be placed in the custom dict :param custom_val: content of the entry to be placed in the custom dict

inginious_container_api.feedback.set_feedback_from_tpl(tpl_name, parameters, problem_id=None, append=False)[source]¶

Parse a template, using the given parameters, and set it as the feedback message.

tpl_name must indicate a file. Given that XX_XX is the lang code of the current user (‘en_US’ or ‘fr_FR’, for example), this function will search template file in different locations, in the following order: - [current_dir]/tpl_name.XX_XX.tpl - [task_dir]/lang/XX_XX/tpl_name.tpl (this is the preferred way, as it contributes to store all translations in the same folder) - [current_dir]/tpl_name.tpl

Note that you can indicate “../otherdir/mytpl” to force the function to search in the “../otherdir” directory. Simply omit the final “.tpl”.

If no file is found or a parsing exception occured, an error is displayed as feedback message, and False is returned. If everything went well, True is returned.

The parsing uses Jinja2.

Parameters is a dictionnary that will be given to the Jinja template.

inginious_container_api.feedback.set_global_feedback(feedback, append=False)[source]¶: Set global feedback in case of error

inginious_container_api.feedback.set_global_result(result)[source]¶: Set global result value

inginious_container_api.feedback.set_grade(grade)[source]¶: Set global grade of this job

inginious_container_api.feedback.set_problem_feedback(feedback, problem_id, append=False)[source]¶: Set problem specific feedback

inginious_container_api.feedback.set_problem_result(result, problem_id)[source]¶: Set problem specific result value

inginious_container_api.feedback.set_state(state)[source]¶: Set the task state

inginious_container_api.feedback.set_tag(tag, value)[source]¶: Set the tag ‘tag’ to the value True or False. :param value: should be a boolean :param tag: should be the id of the tag. Can not starts with ‘*auto-tag-’

inginious_container_api.feedback.tag(value)[source]¶: Add a tag with generated id. :param value: everything working with the str() function

Running subcontainers¶

inginious_container_api.run_student.run_student(cmd, container=None, time_limit=0, hard_time_limit=0, memory_limit=0, share_network=False, working_dir=None, stdin=None, stdout=None, stderr=None, signal_handler_callback=None)[source]¶

Run a command inside a student container :param cmd: command to be ran (as a string, with parameters) :param container: container to use. Must be present in the current agent. By default it is None, meaning the current

container type will be used.

Parameters:

time_limit – time limit in seconds. By default it is 0, which means that it will be the same as the current container (NB: it does not count in the “host” container timeout!)
hard_time_limit – hard time limit. By default it is 0, which means that it will be the same as the current container (NB: it does count in the “host” container hard timeout!)
memory_limit – memory limit in megabytes. By default it is 0, which means that it will be the same as the current container (NB: it does not count in the “host” container memory limit!)
share_network – share the network with the host container if True. Default is False.
working_dir – The working directory for the distant command. By default, it is os.getcwd().
stdin – File descriptor for stdin. Can be None, in which case a file descriptor is open to /dev/null.
stdout – File descriptor for stdout. Can be None, in which case a file descriptor is open to /dev/null.
stderr – File descriptor for stderr. Can be None, in which case a file descriptor is open to /dev/null.
signal_handler_callback – If not None, run will call this callback with a function as single argument. this function can itself be called with a signal value that will immediately be sent to the remote process. See the run_student script command for an example, or the hack_signals function below.

Returns:

the return value of the calling process. There are special values: - 252 means that the command was killed due to an out-of-memory - 253 means that the command timed out - 254 means that an error occurred while running the proxy

inginious_container_api.run_student.run_student_simple(cmd, cmd_input=None, container=None, time_limit=0, hard_time_limit=0, memory_limit=0, share_network=False, working_dir=None, stdout_err_fuse=False, text='utf-8')[source]¶

A simpler version of run, which takes an input string and return the output of the command. This disallows interactive processes.

Parameters:

cmd – cmd to be run.
cmd_input – input of the command. Can be a string or a bytes object, or None.
container – container to use. Must be present in the current agent. By default it is None, meaning the current container type will be used.
time_limit – time limit in seconds. By default it is 0, which means that it will be the same as the current container (NB: it does not count in the “host” container timeout!)
hard_time_limit – hard time limit. By default it is 0, which means that it will be the same as the current container (NB: it does count in the “host” container hard timeout!)
memory_limit – memory limit in megabytes. By default it is 0, which means that it will be the same as the current container (NB: it does not count in the “host” container memory limit!)
share_network – share the network with the host container if True. Default is False.
working_dir – The working directory for the distant command. By default, it is os.getcwd().
stdout_err_fuse – Weither to fuse stdout and stderr (i.e. make them use the same file descriptor)
text – By default, run_simple assumes that stdout/stderr will be encoded in UTF-8. Putting another encoding will make the streams encoded using this encoding. text=False indicates that the streams should be opened in binary mode. In this case, run_simple returns streams in the form of binary, unencoded, strings.

Returns:

The output of the command, as a tuple of objects (stdout, stderr, retval). If stdout_err_fuse is True, the output is in the form (stdout, retval) is returned. The type of the returned strings (stdout, stderr) is dependent of the text arg.

RST helpers¶

inginious_container_api.rst.get_admonition(cssclass, title, text)[source]¶: Generates rst admonition block given a bootstrap alert css class, title, and text

inginious_container_api.rst.get_codeblock(language, text)[source]¶: Generates rst codeblock for given text and language

inginious_container_api.rst.get_imageblock(filename, format='')[source]¶: Generates rst raw block for given image filename and format

inginious_container_api.rst.indent_block(amount, text, indent_char='\t')[source]¶: Indent (or de-indent) text

Language management¶

inginious_container_api.lang.get_lang_dir_path()[source]¶

inginious_container_api.lang.init()[source]¶: Install gettext with the default parameters

Translating tasks¶

INGInious provides support for translating tasks in the language of your student. Most users will want to use the feedback-msg-tpl command, which is the most straighforward way to do translation.

Another possible method is to use gettext. This is by default supported in Python script and feedback templates, but you can use gettext in any language.

Four steps are necessary to translate a task:

Mark strings as translatable
Extract strings to create a .po file
Translate the extracted strings
Compile the translated strings into a .mo file

Marking strings in Python scripts¶

Simply import inginious.lang and run the command inginious.lang.init(). This will install the function _() that can be used to mark strings as translatable.

print(_("Hello"))

Marking strings in feedback templates¶

The function _() in always available in feedback templates:

{{ _("Hello") }}

Extracting strings¶

Now you need to extract the strings, for this we use babel. If it’s not already done, install babel:

pip3 install babel

Create a file named “mapping.babel”, which contains the babel mapping <http://babel.pocoo.org/en/latest/messages.html#extraction-method-mapping-and-configuration>. Here is an example of mapping that will extract both strings marked in Python and in feedback (Jinja2) templates:

[python: **.py]
[jinja2: **.tpl]
encoding = utf-8

If you use other languages, you may want to add the needed options in this file. Refer to the babel documentation.

Now, simply run the following command, which will creates a messages.pot file in your current directory:

pybabel extract -o messages.pot -F mapping.babel .

This messages.pot contains a very simple representation of the strings to translate. Here is an example of what you should obtain after running the command:

#: test.tpl:1
msgid "Hello"
msgstr ""

This will be the basis for all your translations. Let us put the file in the right place, by creating the correct directory structure:

pybabel init -i messages.pot -d lang -l fr_FR

Replace fr_FR by the language your are translating to. This will create a file lang/fr_FR/LC_MESSAGES/message.po.

Translating strings¶

In this file, simply change the msgstr entries with the translation of the immediately above msgid. For example, to translate the previous exemple: in french

(fr_FR):

#: test.tpl:1
msgid "Hello"
msgstr "Bonjour"

Compile your .po file into a .mo¶

The final step is to compile your text-based .po file into a binary .mo file, which ensures that translation occurs smoothly.

pybabel compile -d lang

Which will update all your translations.

I need more help with gettext¶

Gettext is a widely used tool; you will find a lot of software-independent help on the net :-)

Creating a new container image¶

Creating the Dockerfile¶

Container images can be viewed as small operating systems with specific software and configuration. The main force of the container images is that they are very simple to create and modify, using Dockerfiles.

Here is an example of Dockerfile:

# DOCKER-VERSION 1.1.0

# Inherit from the default container, which have all the needed script to launch tasks
FROM    ingi/inginious-c-base

# Set the environment name for tasks
LABEL   org.inginious.grading.name="php"

# Add php
RUN     yum -y install php-cli

# Add the php interpreter as run file interpreter (to allow run.php files)
RUN echo 'run_types["php"] = "/bin/php"' >> /usr/lib/python3.6/site-packages/inginious_container_api/run_types.py

As easily seen, this Dockerfile creates a new container image that can launch PHP scripts. The syntax of these Dockerfiles is extensively described on the website of Docker, but we will detail here the most important things to know.

Each Dockerfile used on INGInious should begin with `FROM inginious/ingi-c-base` and `LABEL org.inginious.grading.name="some_name"`. The first line indicates that you take as base for your new image the default image provided with INGInious. This default image is itself based on CentOS 7, and uses yum (rpm) as package manager. It is already provided with Python and basic commands, and with all the files needed by INGInious to work. The second line is used to indicate the environment name (here, `some_name`) that will be used for tasks.

The line `RUN yum -y install php-cli` indicates to Docker that it must run the command `yum -y install php-cli` inside the image. The `yum` command is the equivalent of `apt-get` (that is the package manager for Debian, Ubuntu and derivates), but for Linux distributions that derivates from Fedora, like CentOS. This will install the package `php-cli`. Creating new containers mainly consists on adding new packages to the default container, so it is probable that your Dockerfile will contain mostly this type of lines.

Here is a little more advanced Dockerfile, that is used to provide Mozart/Oz in INGInious:

FROM    ingi/inginious-c-base
LABEL   org.inginious.grading.name="oz"

ADD mozart2.rpm /mozart.rpm
RUN yum -y install emacs tcl tk
RUN rpm -ivh /mozart.rpm
RUN rm /mozart.rpm

Again, it inherits from `ingi/inginious-c-base` and the environment name is set to `oz`. Then, it uses the command `ADD`, that takes a file in the current directory of the Dockerfile (here, `mozart2.rpm`) and copy it inside the container image, here at the path `/mozart.rpm`. It then uses three `RUN` commands to install the dependencies of Mozart, then install Mozart itself, and then removing the now uneeded rpm.

Dockerfiles can do many more things, read the documentation on the Docker website to know more about the possibilities.

“Compiling” the Dockerfile¶

Once you have Docker up and running, it is very simple to create a container image from a Dockerfile:

$ cd /path/to/your/dockerfile
$ docker build -t my_container_image ./

Docker will then launch a container and run the Dockerfile on it, then will save the state of the disk, that is, in fact, the container image. INGInious will automatically detect the environment based on the labels you’ve set in the Dockerfile. Therefore, the tag `my_container_image` can be set to any value. As a convention, we adopted `inginious-c-XXX`.

For the new environment to be available, you have to restart INGInious (or, at least, the Docker agent if you are running INGInious components on separate machines).

You can also enter directly in the container image to test it in the command line:

$ docker run -t -i --rm my_container_image /bin/bash

Course administration¶

As a course administrator, you can simply access its management page by clicking on “Course administration” in the main course page.

Students submissions¶

Statistics over students submissions are largely available in INGinious, and all the files related to them are stored and can be downloaded.

General overview¶

The administration page gives you several global list views :

All the tasks of a course, with the number of students who viewed the at least one time, who tried and the number of tries, as well as the number of students who succeded the task. This view is the first displayed when you click on “Manage” to enter the administration.
All the students/groups of a course, with the number of tasks tried and done, as well as its global progression for students. This view can be accessed by switching to “Students”/”Groups” in the main administration page.
All the students/groups who tried a given task, if they succeded it, and the number of submissions they did. You can show these information by clicking “View results” on the main administration page or by clicking “Statistics” on the task page.
All the tasks tried by a given student/group, if (s)he/they succeded it and the number of submissions (s)he/they did. These information can be displayed by clicking “View” in the student/group list of a course.
All the submissions made by a student/group for a given tasks, with date of submission and the global result. Submissions can be displayed by clicking “View submissions” in tasks lists.

All the tables can be downloaded in CSV format to make some further treatment in your favourite spreadsheet editor.

More information about groups possibilities can be found below.

Downloading submissions¶

Student submissions can be downloaded from the Download submissions and statistics pages or the submission inspection page. You are able to only download the set of evaluation submissions (according to the task parameters) or all the submissions.

Submissions are downloadable as gzip tarball (.tgz) files. You may need some third-party software if your operating system does not support this format natively. The files contain, for each submissions, a test file with extension test containing the all the submission metadata and the associated archive folder containing all the files that have been exported (via the /archive folder inside the container) (See Run file).

Replaying submissions¶

Student submissions can be replayed either from the Replay submissions and statistics pages or the submission inspection page. Different replay scheme are available:

As replacement of the current student submission result. This is the default scheme for the Replay submissions page. When replayed, submission input are put back in the grading queue. When the job is completed, the newly computed result will replace the old one. This is useful if you want to change the grading scripts during or after the assignment period and want all students to be graded the same way. You can replay only the evaluation submission or all submissions. However, please note that if replayed, the best submission can be replaced by an older best submission.
As a personal copy. This mode is only available from the submission inspection page and copy the student input to generate a new personal copy. This is useful for debugging if a problem occur with a specific student submission. Submission copy is also available with SSH debug mode.

Warning

This feature is currently under testing. As the same job queue is used for standard submissions and submission replay, it is recommended not to launch huge replay jobs on a very active INGInious instance.

Task edition¶

All tasks can be edited from the webapp. To access the task editor, just click on Edit task on the task page or from the main administration page.

Task problems containing boxes are not graphically editable due to their high modularity. These kinds of problem editable on-line in YAML format.

Adding/removing problems¶

Adding and removing problems are very easy in the task editor, go to the end of the page or click on the quick link “Add subproblem”. You’ll then be brought to a new form asking a problem-id (alphanumerical characters) and a problem type.

To make a more complex question with boxes, choose “custom” problem and write the YAML problem description as described in the task file format.

When editing a multiple choice problem, you’re asked if the student is shown a multiple-answers- or single-answer-problem and which of the possible choices is (are) good answer(s).

Task files¶

Task files can be created, uploaded and modified from the task edition page. Only text-base files can be edited from the webapp. Binary files can however be uploaded.

The behaviour of the Move action is Unix-like : it can be used for renaming files.

Classrooms and teams¶

Collaborative work and separate students administration are possible in INGInious. Two models are available:

Classrooms and groups : Classrooms are useful to administratively separate students following the same course. They offer separate statistics to help the teacher identify problems students may encounter in this particular context.

Submissions groups can be set in classrooms and define a set of users that will submit together. Their submissions will contain as authors all the students that were members of the group at submission time. Note that students cannot collaborate with students from another classroom. In this case, please consider using only teams, as described below.
Teams : Teams are administratively-separated submissions groups. They are internally assimilated to classrooms with a unique submission group. They offer separate statistics for each submission group.

Choice between these two models can be made in the course settings. Switching from one model to another will reinitialize the all course structure (that is, students registration also). Course structures can be backed up if necessary from the classrooms/teams administration pages.

Creation and edition¶

Classrooms and teams are created and edited from the web app in the course administration.

Classrooms and groups¶

In the classroom list view, specify a classroom description, and click on “Create new classroom”. The newly created classroom will appear in the list.

To edit a classroom, click on the quick link “Edit classroom” located on the right side of the table. You’ll be able to change the classroom description, the associated teaching staff, and to specify the (grouped) students. Assigning tutors will help them to retrieve their classroom statistics.

The student list is entirely managed by drag-and-drop. You can create a new group on the same page, set its maximum size, and drag-and-drop ungrouped students or already grouped students in the newly created group.

Teams¶

To create a new team, click on “Edit teams” simply in the team list view and press on the “New team” button. You’ll then be able to specify the team description, its maximum size, assigned tutors and students. Team edition works the same way.

The student list is entirely managed by drag-and-drop. Students can be moved from one team to another by simply moving his name to the new team.

Group/team attribution¶

If you do not really matter the way students work together, you can set empty groups or teams with maximum size and let the students choose their groups or teams themselves. Just check the option in the course settings to allow them to gather. When submissions will be retrieved, the group/team members will be displayed as the authors as with staff-defined groups or teams.

Course structure upload¶

You can generate the course classroom or team structure with an external tool and then upload it on INGInious. This is done with a YAML file, which structure for classrooms or teams are similar and described below. The course structure can be upload on the classroom or team list view in the course administration.

Classrooms YAML structure¶

-    description: Classroom 1
     tutors:
             - tutor1
             - tutor2
     students:
             - user1
             - user2
     groups:
             - size: 2
               students:
                     - user1
                     - user2
-    description: Classroom 2
     tutors:
             - tutor1
             - tutor2
     students:
             - user3
             - user4

description is a string and corresponds to your class description
tutors is a list of strings representing the usernames of the assigned classroom tutors.
students is a list of strings representing the usernames of the classroom students.
groups is a list of group structures containing the following elements :
- size: the maximum group size
- students: the list of student usernames in this group

Teams YAML structure¶

-    description: Team 1
     tutors:
             - tutor1
             - tutor2
     students:
             - user1
             - user2
-    description: Team 2
     tutors:
             - tutor1
             - tutor2
     students:
             - user3
             - user4

description is a string and corresponds to your team description
tutors is a list of strings representing the usernames of the assigned team tutors.
students is a list of strings representing the usernames of the team students.

Backup course structure¶

Course structures (classrooms or teams) can be exported for backup or manual edition via the classroom/team list page in the course administration pages. Simply click on the “Download structure” button. The download file will have the same format as described above.

Using through LTI (edX, Moodle, …)¶

INGInious implements the LTI specification in order to integrate into edX, Moodle, or any other LMS that also implements this specification. To get started, all you need is to activate te LTI mode in the course administration and define your LTI keys (consumer key and secret). You’ll then be able to use INGInious tasks as activities in your LMS.

Defining LTI keys¶

The LTI keys are defined in the course administration by first activating the LTI mode. Then, add consumer keys and secrets separated by a colon in the LTI keys field. For instance, here’s an example of a set of keys and secrets:

consumer_key_1:a_very_secret_password
consumer_key_2:wow_such_secret

This obviously defines two LTI keys, consumer_key_1 and consumer_key_2, with passwords a_very_secret_password and wow_such_secret. You also need to check the Send grades back option to actually send the scores from INGInious to your LTI.

Setting up your LMS¶

Setting up (Open) edX¶

edX provides a good tutorial on how to install LTI components.

When it asks for the LTI passport, you have to enter it in the format an_id_that_you_define:consumer_key:password. A good example, taking values from the start of this document, would be

inginious:consumer_key_1:a_very_secret_password

The launch url is, if your server is located at https://HOST:PORT/, and you want to load the task task_id from the course course_id:

https://HOST:PORT/lti/course_id/task_id

Please note that, for now, official edX needs https. You also need to set the LTI activity to accept a score back from INGInious if your have set up INGInious such that scores are sent back.

Setting up Moodle¶

Under edition mode, select add an activity, choose external tool, and confirm.

Directly click on Show more.... Fill in the activity name.

The Launch URL is, if your server is located at https://HOST:PORT/, and you want to load the task task_id from the course course_id:

https://HOST:PORT/lti/course_id/task_id

For the field Launch Container, the best value is “Embded without block”. Consumer key and Consumer secret are the LTI key you defined earlier. In the Privacy fieldset, check that accept grades from the tool is checked. Leave the other fields blank (or modify them as you want).

Save, and it should work.

Setting up other LMS¶

INGInious has only been tested with edX and Moodle, but it should work out-of-the-box with any LMS that respects LTI 1.1. You are on your own for the configuration, though; but with the LTI keys and the launch URL, it should be enough to configure anything.

Using ReStructuredText in INGInious¶

This guide gives some examples on how to use ReStructuredText in INGInious.

Links, images, tables, and other basic things¶

The RST documentation is pretty complete, and INGInious supports most of it. See below for some basic examples.

Example¶

https://raw.githubusercontent.com/UCL-INGI/INGInious/master/inginious/frontend/static/images/header.png

A link to the image

Math block:

\[α_t(i) = P(O_1, O_2, … O_t, q_t = S_i λ)\]

A sentence with inline math: $α_t(i) = P(O_1, O_2, … O_t, q_t = S_i λ)$ (you see, it’s inline). $\LaTeX$ is supported.

A	not A
False	True
True	False

Code¶

`The RST documentation is pretty complete <http://docutils.sourceforge.net/docs/ref/rst/directives.html>`_, and
INGInious supports **most** of it. See below for some basic examples.

Example
```````

.. image:: https://raw.githubusercontent.com/UCL-INGI/INGInious/master/inginious/frontend/static/images/header.png

`A link to the image <https://raw.githubusercontent.com/UCL-INGI/INGInious/master/inginious/frontend/static/images/header.png>`_

Math block:

.. math::

  α_t(i) = P(O_1, O_2, … O_t, q_t = S_i λ)

A sentence with inline math: :math:`α_t(i) = P(O_1, O_2, … O_t, q_t = S_i λ)` (you see, it's inline).
:math:`\LaTeX` is supported.


.. table::

   =====  =====
     A    not A
   =====  =====
   False  True
   True   False
   =====  =====

Admonitions - warnings, tips, …¶

The following warning/tips/… can be displayed using the code given below. INGInious is able to parse standard RST admonitions (which are displayed as bootstrap alerts).

We also introduced a new option, :title:, which allows providing a title. In case a title is given, the admonitions are displayed as bootstrap cards.

Example¶

A custom danger title

Beware killer rabbits!

A custom warning title

Water is humid!

You may need this

Take this tip with you

Some note

Write something here

Bravo!

You succeeded!

Now in blue

Some blue for you

Now in grey

Some grey for you

Some dark in the light

...

Beware killer rabbits!

Water is humid!

Take this tip with you

You succeeded!

Some blue for you

...

Code¶

.. danger::
   :title: A custom danger title

   Beware killer rabbits!

.. warning::
   :title: A custom warning title

   Water is humid!

.. tip::
   :title: You may need this

   Take this tip with you

.. note::
   :title: Some note

   Write something here

.. admonition:: success
   :title: Bravo!

   You succeeded!

.. admonition:: primary
   :title: Now in blue

   Some blue for you

.. admonition:: secondary
   :title: Now in grey

   Some grey for you

.. admonition:: dark
   :title: Some dark in the light

   ...

.. danger::
   Beware killer rabbits!

.. warning::
   Water is humid!

.. tip::
   Take this tip with you

.. admonition:: success

   You succeeded!

.. admonition:: primary

   Some blue for you

.. admonition:: dark

   ...

Hidden-until¶

hidden-until is a special directive to give feedback after a given date is reached. The recommended format to indicate the date is YYYY-MM-DD hh:mm:ss.

.. hidden-until: 2020-01-01 12:00:03

    This sentence will only be displayed after the 1st January 2020 at 12:00:03.

Developer’s documentation¶

Understand INGInious¶

INGInious is made from three different packages:

The common which contains basic blocks, like tasks and courses. Derivates from this blocks are created by the frontend and other modules. The common does not need the backend nor the frontend;
The agent, that runs jobs. It interacts directly with Docker to start new containers, and sends the grades back to the backend. A specific part of the backend is in charge of starting the agents automatically; you most of time won’t need to it manually. The agent needs to be run on the Docker host, as it interacts with other containers with Unix sockets, and must also interact with CGroups to allow a very fine management of timeouts and memory limits.
The backend, which is in charge of handling grading requests, giving the work to distant agents; the backend is made to be simple and frontend-agnostic; you can ‘easily’ replace the frontend by something else. The backend only store information about running tasks. This point is important when considering replication and horizontal scalability (see later)
The frontend which is a web interface for the backend. It provides a simple yet powerful interface for students and teachers. It is made to be “stateless”: all its state is stored in DB, allowing to replicate the frontend horizontally.

Basic architecture of INGInious¶

The following schema shows the basic architecture of INGInious:

Scalability of Docker hosts¶

In order to share the work between multiple servers, INGInious can use multiple agents, as shown in the following schema. The completely horizontal scalability is (nearly) without additional configuration, and can be made fully automatic with a bit of work.

Scalability of the INGInious frontend¶

As the backend only stores information about running submission, and the frontend is stateless, we can use the replication feature of MongoDB to scale horizontally the frontends too. The (final) schema below shows the most advanced way of configuring INGInious, with multiple frontends replicated and multiple Docker hosts.

Grading containers and student containers¶

A grading container is a container that do the grading. It typically runs a script made by a teacher or its assistants, a launch sub-containers, called student containers, that will separately jail code made by students.

A single grading container can launch more than one student container; the interaction between the two is completely secured by the agent.

Jobs¶

When you send a student’s input to the backend, it creates what we call a job. Jobs are sent to an object called the Client, which itself is a simple communication layer to a job queue that we call the Backend. The Backend itself can be used by multiple Clients, and dispatch jobs among Agents, which can be of different types (for now, we have two kinds of agents, DockerAgent and MCQAgent)

When a job is submitted, a callback must be given: it is automatically called when the task is done, asynchronously.

Submission¶

A submission is an extension of the concept of job. A submission only exists in the frontend, and it is mainly a job saved to db.

Creating plugins¶

INGInious provides a simple plugin system that allow to register some hooks to extend existing features, create new frontend pages and features, and add new authentication methods.

Hooks actually call callback functions that you indicated with the add_hook method from HookManager. Please note that all hooks may be called by another thread, so all actions done into a hook have to be thread-safe.

Tutorial¶

The following code adds a new page displaying This is a simple demo plugin on the /plugindemo location.

class DemoPage(object):
    """ A simple demo page showing how to add a new page """

    def GET(self):
        """ GET request """
        return "This is a simple demo plugin"


def init(plugin_manager, course_factory, client, plugin_config):
    """ Init the plugin """
    plugin_manager.add_page("/plugindemo", DemoPage)

The plugin is initialized by the plugin manager, which is the frontend-extended hook manager, by calling method init. This method takes four arguments:

plugin_manager which is the plugin manager singleton object. The detailed API is available at inginious.frontend.common.plugin_manager. Please note that PluginManager inherits from inginious.common.hook_manager module.
course_factory which is the course factory singleton object, giving you abstraction to the tasks folder. The detailed API is available at inginious.common.course_factory module.
client which is the INGInious client singleton object, giving you access to the backend features, as launching a new job. The detailed API is available at inginious.client.client module.
plugin_config which is a dictionary containing the plugin configuration fields set in your configuration.yaml file. For instance, configuration:
```
plugins:
    - plugin_module: inginious.frontend.plugins.demo
      param1: "value1"
```
will generate the following plugin_config dictionary :
```
{"plugin_module": "inginious.frontend.plugins.demo", "param1": "value1"}
```

The remaining INGInious classes can be used from your plugins using correct imports. The init method gives you access to the different singletons used by INGInious which are instantiated at boot time. For instance, LTIPage class can be used as base for a new LTI page.

The plugin_module configuration parameter corresponds to the Python package in which the init method is found. A demonstration plugin is found in the inginious.frontend.plugins.demo. You do not need to include your plugin in the INGInious sources. As long as your plugin is found in the Python path, it will remain usable by INGInious.

List of hooks¶

You may be interested to generate some actions useful for your plugins before or after some INGInious events. You would therefore need to add a hook method. This can be done using the add_hook method of package inginious.frontend.common.plugin_manager. For instance, the following plugin :

import logging

def submission_done(submission, archive, newsub):
    logging.getLogger("inginious.frontend.plugins.demo").info("Submission " + str(submission['_id']) + " done.")

def init(plugin_manager, course_factory, client, plugin_config):
    """ Init the plugin """
    plugin_manager.add_hook("submission_done", submission_done)

will log each submission id that has been returning from the backend.

Each hook available in INGInious is described here, starting with its name and parameters. Please refer to the complete inginious.frontend.common package documentation for more information on the data returned by those hooks.

css

Returns : List of path to CSS files.

Used to add CSS files in the header. Should return the path to a CSS file (relative to the root of INGInious).

course_admin_menu (course)

course : inginious.frontend.common.courses.FrontendCourse

Returns : Tuple (link, name) or None.

Used to add links to the administration menu. This hook should return a tuple (link,name) where link is the relative link from the index of the course administration. You can also return None.

main_menu (template_helper)

template_helper : inginious.frontend.common.template_helper.TemplateHelper

Returns : HTML or None.

Allows to add HTML to the menu displayed on the main (course list) page. template_helper is an object of type TemplateHelper, that can be useful to render templates.

course_menu (course, template_helper)

course : inginious.frontend.common.courses.FrontendCourse

template_helper : inginious.frontend.common.template_helper.TemplateHelper

Returns : HTML or None.

Allows to add HTML to the menu displayed on the course page. Course is the course object related to the page. template_helper is an object of type TemplateHelper, that can be useful to render templates.

task_menu (course, task, template_helper)

course : inginious.frontend.common.courses.FrontendCourse

task : inginious.frontend.common.tasks.FrontendTask

template_helper : inginious.frontend.common.template_helper.TemplateHelper

Returns: HTML or None.

Allows to add HTML to the menu displayed on the course page. course is the course object related to the page. task is the task object related to the page. template_helper is an object of type TemplateHelper, that can be useful to render templates.

welcome_text (template_helper)

template_helper : inginious.frontend.common.template_helper.TemplateHelper

Returns : HTML or None.

Allows to add HTML to the login/welcome page. template_helper is an object of type TemplateHelper, that can be useful to render templates.

javascript_header

Returns : List of path to Javascript files.

Used to add Javascript files in the header. Should return the path to a Javascript file (relative to the root of INGInious).

javascript_footer

Returns : List of path to Javascript files.

Used to add Javascript files in the footer. Should return the path to a Javascript file (relative to the root of INGInious).

course_accessibility (course, default)

Returns: inginious.frontend.accessible_time.AccessibleTime