inginious.common package¶

Common package: basic library for INGInious, needed by all its components.

Subpackages¶

Submodules¶

inginious.common.asyncio_utils module¶

Utilities for asyncio

class inginious.common.asyncio_utils.AsyncIteratorWrapper(obj)[source]¶

Bases: object

A wrapper that converts old-style-generators to async generators using run_in_executor

class inginious.common.asyncio_utils.AsyncProxy(module, loop=None, executor=None)[source]¶

Bases: Generic[inginious.common.asyncio_utils.T]

An asyncio proxy for modules and classes

property sync: inginious.common.asyncio_utils.T¶

Return the original sync module/class

Return type: TypeVar(T)

inginious.common.asyncio_utils.create_safe_task(loop, logger, coroutine)[source]¶: Calls loop.create_task with a safe (== with logged exception) coroutine

inginious.common.base module¶

Basic dependencies for every modules that uses INGInious

inginious.common.base.dict_from_prefix(prefix, dictionary)[source]¶

>>> from collections import OrderedDict
>>> od = OrderedDict()
>>> od["problem[q0][a]"]=1
>>> od["problem[q0][b][c]"]=2
>>> od["problem[q1][first]"]=1
>>> od["problem[q1][second]"]=2
>>> AdminCourseEditTask.dict_from_prefix("problem",od)
OrderedDict([('q0', OrderedDict([('a', 1), ('b', OrderedDict([('c', 2)]))])), ('q1', OrderedDict([('first', 1), ('second', 2)]))])

inginious.common.base.directory_compare_from_hash(from_directory, to_directory)[source]¶

Parameters

from_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash
to_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash

Returns

a tuple containing two list: the files that should be uploaded to “to_directory” and the files that should be removed from “to_directory”

inginious.common.base.directory_content_with_hash(directory)[source]¶

Parameters: directory – directory in which the function list the files
Returns: dict in the form {file: (hash of the file, stat of the file)}

inginious.common.base.get_json_or_yaml(file_path, content)[source]¶: Generate JSON or YAML depending on the file extension.

inginious.common.base.hash_file(fileobj)[source]¶

Parameters: fileobj – a file object
Returns: a hash of the file content

inginious.common.base.id_checker(id_to_test)[source]¶: Checks if a id is correct

inginious.common.base.id_checker_tests(id_to_test)[source]¶: Checks if a id is correct

inginious.common.base.load_json_or_yaml(file_path)[source]¶: Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.loads_json_or_yaml(file_path, content)[source]¶: Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.write_json_or_yaml(file_path, content)[source]¶: Write JSON or YAML depending on the file extension.

inginious.common.custom_yaml module¶

A custom YAML based on PyYAML, that provides Ordered Dicts

inginious.common.custom_yaml.dump(data, stream=None, **kwds)[source]¶

Serialize a Python object into a YAML stream. If stream is None, return the produced string instead. Dict keys are produced in the order in which they appear in OrderedDicts.

Safe version.

If objects are not “conventional” objects, they will be dumped converted to string with the str() function. They will then not be recovered when loading with the load() function.

inginious.common.custom_yaml.load(stream)[source]¶

Parse the first YAML document in a stream and produce the corresponding Python object. Use OrderedDicts to produce dicts.

Safe version.

inginious.common.exceptions module¶

Some type of exceptions used by parts of INGInious

exception inginious.common.exceptions.CourseAlreadyExistsException[source]¶: Bases: Exception

exception inginious.common.exceptions.CourseNotFoundException[source]¶: Bases: Exception

exception inginious.common.exceptions.CourseUnreadableException[source]¶: Bases: Exception

exception inginious.common.exceptions.ImportCourseException[source]¶: Bases: Exception

exception inginious.common.exceptions.InvalidNameException[source]¶: Bases: Exception

exception inginious.common.exceptions.TaskAlreadyExistsException[source]¶: Bases: Exception

exception inginious.common.exceptions.TaskNotFoundException[source]¶: Bases: Exception

exception inginious.common.exceptions.TaskReaderNotFoundException[source]¶: Bases: Exception

exception inginious.common.exceptions.TaskUnreadableException[source]¶: Bases: Exception

inginious.common.log module¶

Some common functions for logging

class inginious.common.log.CustomLogMiddleware(app, logger)[source]¶

Bases: object

WSGI middleware for logging the status in webpy

log(status, environ)[source]¶

inginious.common.log.get_course_logger(coursename)[source]¶

Parameters: coursename – the course id
Returns: a logger object associated to a specific course

inginious.common.log.init_logging(log_level=10)[source]¶: Init logging :param log_level: An integer representing the log level or a string representing one

inginious.common.messages module¶

class inginious.common.messages.AgentHello(friendly_name, available_job_slots, available_environments, ssh_allowed)[source]¶

Bases: object

Let the agent say hello and announce which environments it has available

available_environments: Dict[str, Dict[str, Dict[str, Any]]]¶

available_job_slots: int¶

friendly_name: str¶

ssh_allowed: bool¶

class inginious.common.messages.AgentJobDone(job_id, result, grade, problems, tests, custom, state, archive, stdout, stderr)[source]¶

Bases: object

Gives the result of a job.

archive: Optional[bytes]¶

custom: Dict[str, Any]¶

grade: float¶

job_id: str¶

problems: Dict[str, Tuple[str, str]]¶

result: Tuple[str, str]¶

state: str¶

stderr: Optional[str]¶

stdout: Optional[str]¶

tests: Dict[str, Any]¶

class inginious.common.messages.AgentJobSSHDebug(job_id, host, port, user, password)[source]¶

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

host: str¶

job_id: str¶

password: str¶

port: int¶

user: str¶

class inginious.common.messages.AgentJobStarted(job_id)[source]¶

Bases: object

Indicates to the backend that a job started

job_id: str¶

class inginious.common.messages.BackendGetQueue(jobs_running, jobs_waiting)[source]¶

Bases: object

Send the status of the job queue to the client. Attributes:

jobs_running : a list of tuples in the form (job_id, is_current_client_job, info, launcher, started_at, max_tuime) where:
- job_id is a job id. It may be from another client.
- is_current_client_job is a boolean indicating if the client that asked the request has started the job
- agent_name is the agent name
- info is “courseid/taskid”
- launcher is the name of the launcher, which may be anything
- started_at the time (in seconds since UNIX epoch) at which the job started
- max_time the maximum time that can be used, or -1 if no timeout is set
jobs_waiting : a list of tuples in the form (job_id, is_current_client_job, info, launcher, max_time) where:
- job_id is a job id. It may be from another client.
- is_current_client_job is a boolean indicating if the client that asked the request has started the job
- info is “courseid/taskid”
- launcher is the name of the launcher, which may be anything
- max_time the maximum time that can be used, or -1 if no timeout is set

jobs_running: List[Tuple[str, bool, str, str, str, int, int]]¶

jobs_waiting: List[Tuple[str, bool, str, str, int]]¶

class inginious.common.messages.BackendJobDone(job_id, result, grade, problems, tests, custom, state, archive, stdout, stderr)[source]¶

Bases: object

Gives the result of a job.

archive: Optional[bytes]¶

custom: Dict[str, Any]¶

grade: float¶

job_id: str¶

problems: Dict[str, Tuple[str, str]]¶

result: Tuple[str, str]¶

state: str¶

stderr: Optional[str]¶

stdout: Optional[str]¶

tests: Dict[str, Any]¶

class inginious.common.messages.BackendJobSSHDebug(job_id, host, port, user, password)[source]¶

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

host: str¶

job_id: str¶

password: str¶

port: int¶

user: str¶

class inginious.common.messages.BackendJobStarted(job_id)[source]¶

Bases: object

Indicates to the backend that a job started

job_id: str¶

class inginious.common.messages.BackendKillJob(job_id, state)[source]¶

Bases: object

Kills a running job.

job_id: str¶

state: str¶

class inginious.common.messages.BackendNewJob(job_id, course_id, task_id, task_problems, inputdata, environment_type, environment, environment_parameters, debug)[source]¶

Bases: object

Creates a new job

course_id: str¶

debug: Union[str, bool]¶

environment: str¶

environment_parameters: Dict[str, Any]¶

environment_type: str¶

inputdata: Dict[str, Any]¶

job_id: str¶

task_id: str¶

task_problems: Dict[str, Any]¶

class inginious.common.messages.BackendUpdateEnvironments(available_environments)[source]¶

Bases: object

Update the information about the environments on the client, from the informations retrieved from the agents

available_environments: Dict[str, List[str]]¶

class inginious.common.messages.ClientGetQueue[source]¶

Bases: object

Ask the backend to send the status of its job queue

class inginious.common.messages.ClientHello(name)[source]¶

Bases: object

Let the client say hello to the backend (and thus register to some events)

name: str¶

class inginious.common.messages.ClientKillJob(job_id)[source]¶

Bases: object

Kills a running job.

job_id: str¶

class inginious.common.messages.ClientNewJob(job_id, priority, course_id, task_id, task_problems, inputdata, environment_type, environment, environment_parameters, debug, launcher)[source]¶

Bases: object

Creates a new job

course_id: str¶

debug: Union[str, bool]¶

environment: str¶

environment_parameters: Dict[str, Any]¶

environment_type: str¶

inputdata: Dict[str, Any]¶

job_id: str¶

launcher: str¶

priority: int¶

task_id: str¶

task_problems: Dict[str, Any]¶

class inginious.common.messages.Ping[source]¶

Bases: object

Ping message

class inginious.common.messages.Pong[source]¶

Bases: object

Pong message

class inginious.common.messages.Unknown[source]¶

Bases: object

Unknown message. Sent by a server that do not know a specific client; probably because the server restarted

class inginious.common.messages.ZMQUtils[source]¶

Bases: object

Utilities that do serializing/unserializing of messages (whose metaclass is MessageMeta)

async classmethod recv(socket, skip_first=False)[source]¶

async classmethod recv_with_addr(socket)[source]¶

async classmethod send(socket, obj, send_white=False)[source]¶

async classmethod send_with_addr(socket, addr, obj)[source]¶

inginious.common.messages.cls(tcls)¶: Register a new type of message

inginious.common.messages.dump(msg)[source]¶

Return type: bytes
Returns: a bytestring containing a black-box representation of the message, that can be loaded using messages.load.

inginious.common.messages.load(bmessage)[source]¶: From a bytestring given by a (distant) call to dump(), retrieve the original message :type bmessage: bytes :param bmessage: bytestring given by a dump() call on a message :rtype: Any :return: the original message

inginious.common.messages.register_message(tcls)[source]¶: Register a new type of message

inginious.common.messages.run_tests()[source]¶

inginious.common.tasks_problems module¶

Tasks’ problems

class inginious.common.tasks_problems.CodeProblem(problemid, content, translations, taskfs)[source]¶

Bases: inginious.common.tasks_problems.Problem

Code problem

check_answer(_, __)[source]¶: Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]¶: Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]¶: Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]¶: Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]¶: Indicates if problem input type

classmethod parse_problem(problem_content)[source]¶

class inginious.common.tasks_problems.CodeSingleLineProblem(problemid, content, translations, taskfs)[source]¶

Bases: inginious.common.tasks_problems.CodeProblem

Code problem with a single line of input

classmethod get_type()[source]¶: Returns the type of the problem

class inginious.common.tasks_problems.FileProblem(problemid, content, translations, taskfs)[source]¶

Bases: inginious.common.tasks_problems.Problem

File upload Problem

check_answer(_, __)[source]¶: Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]¶: Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]¶: Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]¶: Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]¶: Indicates if problem input type

classmethod parse_problem(problem_content)[source]¶

class inginious.common.tasks_problems.MatchProblem(problemid, content, translations, taskfs)[source]¶

Bases: inginious.common.tasks_problems.Problem

Display an input box and check that the content is correct

check_answer(task_input, language)[source]¶: Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]¶: Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]¶: Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]¶: Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]¶: Indicates if problem input type

classmethod parse_problem(problem_content)[source]¶

class inginious.common.tasks_problems.MultipleChoiceProblem(problemid, content, translations, taskfs)[source]¶

Bases: inginious.common.tasks_problems.Problem

Multiple choice problems

allow_multiple()[source]¶: Returns true if this multiple choice problem allows checking multiple answers

check_answer(task_input, language)[source]¶: Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

get_choice_with_index(index)[source]¶: Return the choice with index=index

classmethod get_text_fields()[source]¶: Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]¶: Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]¶: Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]¶: Indicates if problem input type

classmethod parse_problem(problem_content)[source]¶

class inginious.common.tasks_problems.Problem(problemid, content, translations, taskfs)[source]¶

Bases: object

Basic problem

abstract check_answer(task_input, language)[source]¶: Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

get_id()[source]¶: Get the id of this problem

get_name(language=None)[source]¶: Get the name of this problem

get_original_content()[source]¶: Get a dict fully describing this sub-problem

abstract classmethod get_text_fields()[source]¶: Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

get_translation_obj(language=None)[source]¶

abstract classmethod get_type()[source]¶: Returns the type of the problem

gettext(language, *args, **kwargs)[source]¶

abstract input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]¶: Check if an input for this problem is consistent. Return true if this is case, false else

abstract input_type()[source]¶: Indicates if problem input type

classmethod parse_problem(problem_content)[source]¶