inginious.common package

Common package: basic library for INGInious, needed by all its components.

Submodules

inginious.common.asyncio_utils module

Utilities for asyncio

class inginious.common.asyncio_utils.AsyncIteratorWrapper(obj)[source]

Bases: object

A wrapper that converts old-style-generators to async generators using run_in_executor

class inginious.common.asyncio_utils.AsyncProxy(module: T, loop=None, executor=None)[source]

Bases: Generic[inginious.common.asyncio_utils.T]

An asyncio proxy for modules and classes

property sync

Return the original sync module/class

inginious.common.asyncio_utils.create_safe_task(loop, logger, coroutine)[source]

Calls loop.create_task with a safe (== with logged exception) coroutine

inginious.common.base module

Basic dependencies for every modules that uses INGInious

inginious.common.base.dict_from_prefix(prefix, dictionary)[source]
>>> from collections import OrderedDict
>>> od = OrderedDict()
>>> od["problem[q0][a]"]=1
>>> od["problem[q0][b][c]"]=2
>>> od["problem[q1][first]"]=1
>>> od["problem[q1][second]"]=2
>>> AdminCourseEditTask.dict_from_prefix("problem",od)
OrderedDict([('q0', OrderedDict([('a', 1), ('b', OrderedDict([('c', 2)]))])), ('q1', OrderedDict([('first', 1), ('second', 2)]))])
inginious.common.base.directory_compare_from_hash(from_directory, to_directory)[source]
Parameters
  • from_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash

  • to_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash

Returns

a tuple containing two list: the files that should be uploaded to “to_directory” and the files that should be removed from “to_directory”

inginious.common.base.directory_content_with_hash(directory)[source]
Parameters

directory – directory in which the function list the files

Returns

dict in the form {file: (hash of the file, stat of the file)}

inginious.common.base.get_json_or_yaml(file_path, content)[source]

Generate JSON or YAML depending on the file extension.

inginious.common.base.hash_file(fileobj)[source]
Parameters

fileobj – a file object

Returns

a hash of the file content

inginious.common.base.id_checker(id_to_test)[source]

Checks if a id is correct

inginious.common.base.id_checker_tests(id_to_test)[source]

Checks if a id is correct

inginious.common.base.load_json_or_yaml(file_path)[source]

Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.loads_json_or_yaml(file_path, content)[source]

Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.write_json_or_yaml(file_path, content)[source]

Write JSON or YAML depending on the file extension.

inginious.common.custom_yaml module

A custom YAML based on PyYAML, that provides Ordered Dicts

inginious.common.custom_yaml.dump(data, stream=None, **kwds)[source]

Serialize a Python object into a YAML stream. If stream is None, return the produced string instead. Dict keys are produced in the order in which they appear in OrderedDicts.

Safe version.

If objects are not “conventional” objects, they will be dumped converted to string with the str() function. They will then not be recovered when loading with the load() function.

inginious.common.custom_yaml.load(stream)[source]

Parse the first YAML document in a stream and produce the corresponding Python object. Use OrderedDicts to produce dicts.

Safe version.

inginious.common.exceptions module

Some type of exceptions used by parts of INGInious

exception inginious.common.exceptions.CourseAlreadyExistsException[source]

Bases: Exception

exception inginious.common.exceptions.CourseNotFoundException[source]

Bases: Exception

exception inginious.common.exceptions.CourseUnreadableException[source]

Bases: Exception

exception inginious.common.exceptions.ImportCourseException[source]

Bases: Exception

exception inginious.common.exceptions.InvalidNameException[source]

Bases: Exception

exception inginious.common.exceptions.TaskAlreadyExistsException[source]

Bases: Exception

exception inginious.common.exceptions.TaskNotFoundException[source]

Bases: Exception

exception inginious.common.exceptions.TaskReaderNotFoundException[source]

Bases: Exception

exception inginious.common.exceptions.TaskUnreadableException[source]

Bases: Exception

inginious.common.log module

Some common functions for logging

class inginious.common.log.CustomLogMiddleware(app, logger)[source]

Bases: object

WSGI middleware for logging the status in webpy

log(status, environ)[source]
inginious.common.log.get_course_logger(coursename)[source]
Parameters

coursename – the course id

Returns

a logger object associated to a specific course

inginious.common.log.init_logging(log_level=10)[source]

Init logging :param log_level: An integer representing the log level or a string representing one

inginious.common.messages module

class inginious.common.messages.AgentHello(friendly_name: str, available_job_slots: int, available_environments: Dict[str, Dict[str, Dict[str, Any]]])[source]

Bases: object

Let the agent say hello and announce which environments it has available

available_environments: Dict[str, Dict[str, Dict[str, Any]]]
available_job_slots: int
friendly_name: str
class inginious.common.messages.AgentJobDone(job_id: str, result: Tuple[str, str], grade: float, problems: Dict[str, Tuple[str, str]], tests: Dict[str, Any], custom: Dict[str, Any], state: str, archive: Optional[bytes], stdout: Optional[str], stderr: Optional[str])[source]

Bases: object

Gives the result of a job.

archive: Optional[bytes]
custom: Dict[str, Any]
grade: float
job_id: str
problems: Dict[str, Tuple[str, str]]
result: Tuple[str, str]
state: str
stderr: Optional[str]
stdout: Optional[str]
tests: Dict[str, Any]
class inginious.common.messages.AgentJobSSHDebug(job_id: str, host: str, port: int, user: str, password: str)[source]

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

host: str
job_id: str
password: str
port: int
user: str
class inginious.common.messages.AgentJobStarted(job_id: str)[source]

Bases: object

Indicates to the backend that a job started

job_id: str
class inginious.common.messages.BackendGetQueue(jobs_running: List[Tuple[str, bool, str, str, str, int, int]], jobs_waiting: List[Tuple[str, bool, str, str, int]])[source]

Bases: object

Send the status of the job queue to the client. Attributes:

  • jobs_running : a list of tuples in the form (job_id, is_current_client_job, info, launcher, started_at, max_tuime) where:

    • job_id is a job id. It may be from another client.

    • is_current_client_job is a boolean indicating if the client that asked the request has started the job

    • agent_name is the agent name

    • info is “courseid/taskid”

    • launcher is the name of the launcher, which may be anything

    • started_at the time (in seconds since UNIX epoch) at which the job started

    • max_time the maximum time that can be used, or -1 if no timeout is set

  • jobs_waiting : a list of tuples in the form (job_id, is_current_client_job, info, launcher, max_time) where:

    • job_id is a job id. It may be from another client.

    • is_current_client_job is a boolean indicating if the client that asked the request has started the job

    • info is “courseid/taskid”

    • launcher is the name of the launcher, which may be anything

    • max_time the maximum time that can be used, or -1 if no timeout is set

jobs_running: List[Tuple[str, bool, str, str, str, int, int]]
jobs_waiting: List[Tuple[str, bool, str, str, int]]
class inginious.common.messages.BackendJobDone(job_id: str, result: Tuple[str, str], grade: float, problems: Dict[str, Tuple[str, str]], tests: Dict[str, Any], custom: Dict[str, Any], state: str, archive: Optional[bytes], stdout: Optional[str], stderr: Optional[str])[source]

Bases: object

Gives the result of a job.

archive: Optional[bytes]
custom: Dict[str, Any]
grade: float
job_id: str
problems: Dict[str, Tuple[str, str]]
result: Tuple[str, str]
state: str
stderr: Optional[str]
stdout: Optional[str]
tests: Dict[str, Any]
class inginious.common.messages.BackendJobSSHDebug(job_id: str, host: str, port: int, user: str, password: str)[source]

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

host: str
job_id: str
password: str
port: int
user: str
class inginious.common.messages.BackendJobStarted(job_id: str)[source]

Bases: object

Indicates to the backend that a job started

job_id: str
class inginious.common.messages.BackendKillJob(job_id: str)[source]

Bases: object

Kills a running job.

job_id: str
class inginious.common.messages.BackendNewJob(job_id: str, course_id: str, task_id: str, task_problems: Dict[str, Any], inputdata: Dict[str, Any], environment_type: str, environment: str, environment_parameters: Dict[str, Any], debug: Union[str, bool])[source]

Bases: object

Creates a new job

course_id: str
debug: Union[str, bool]
environment: str
environment_parameters: Dict[str, Any]
environment_type: str
inputdata: Dict[str, Any]
job_id: str
task_id: str
task_problems: Dict[str, Any]
class inginious.common.messages.BackendUpdateEnvironments(available_environments: Dict[str, List[str]])[source]

Bases: object

Update the information about the environments on the client, from the informations retrieved from the agents

available_environments: Dict[str, List[str]]
class inginious.common.messages.ClientGetQueue[source]

Bases: object

Ask the backend to send the status of its job queue

class inginious.common.messages.ClientHello(name: str)[source]

Bases: object

Let the client say hello to the backend (and thus register to some events)

name: str
class inginious.common.messages.ClientKillJob(job_id: str)[source]

Bases: object

Kills a running job.

job_id: str
class inginious.common.messages.ClientNewJob(job_id: str, priority: int, course_id: str, task_id: str, task_problems: Dict[str, Any], inputdata: Dict[str, Any], environment_type: str, environment: str, environment_parameters: Dict[str, Any], debug: Union[str, bool], launcher: str)[source]

Bases: object

Creates a new job

course_id: str
debug: Union[str, bool]
environment: str
environment_parameters: Dict[str, Any]
environment_type: str
inputdata: Dict[str, Any]
job_id: str
launcher: str
priority: int
task_id: str
task_problems: Dict[str, Any]
class inginious.common.messages.Ping[source]

Bases: object

Ping message

class inginious.common.messages.Pong[source]

Bases: object

Pong message

class inginious.common.messages.Unknown[source]

Bases: object

Unknown message. Sent by a server that do not know a specific client; probably because the server restarted

class inginious.common.messages.ZMQUtils[source]

Bases: object

Utilities that do serializing/unserializing of messages (whose metaclass is MessageMeta)

async classmethod recv(socket, skip_first=False)[source]
async classmethod recv_with_addr(socket)[source]
async classmethod send(socket, obj, send_white=False)[source]
async classmethod send_with_addr(socket, addr: bytes, obj)[source]
inginious.common.messages.cls(tcls: Type[Any])

Register a new type of message

inginious.common.messages.dump(msg: Any)bytes[source]
Returns

a bytestring containing a black-box representation of the message, that can be loaded using messages.load.

inginious.common.messages.load(bmessage: bytes)Any[source]

From a bytestring given by a (distant) call to dump(), retrieve the original message :param bmessage: bytestring given by a dump() call on a message :return: the original message

inginious.common.messages.register_message(tcls: Type[Any])[source]

Register a new type of message

inginious.common.messages.run_tests()[source]

inginious.common.tasks_problems module

Tasks’ problems

class inginious.common.tasks_problems.CodeProblem(problemid, content, translations, taskfs)[source]

Bases: inginious.common.tasks_problems.Problem

Code problem

check_answer(_, __)[source]

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]

Indicates if problem input type

classmethod parse_problem(problem_content)[source]
class inginious.common.tasks_problems.CodeSingleLineProblem(problemid, content, translations, taskfs)[source]

Bases: inginious.common.tasks_problems.CodeProblem

Code problem with a single line of input

classmethod get_type()[source]

Returns the type of the problem

class inginious.common.tasks_problems.FileProblem(problemid, content, translations, taskfs)[source]

Bases: inginious.common.tasks_problems.Problem

File upload Problem

check_answer(_, __)[source]

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]

Indicates if problem input type

classmethod parse_problem(problem_content)[source]
class inginious.common.tasks_problems.MatchProblem(problemid, content, translations, taskfs)[source]

Bases: inginious.common.tasks_problems.Problem

Display an input box and check that the content is correct

check_answer(task_input, language)[source]

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

classmethod get_text_fields()[source]

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]

Indicates if problem input type

classmethod parse_problem(problem_content)[source]
class inginious.common.tasks_problems.MultipleChoiceProblem(problemid, content, translations, taskfs)[source]

Bases: inginious.common.tasks_problems.Problem

Multiple choice problems

allow_multiple()[source]

Returns true if this multiple choice problem allows checking multiple answers

check_answer(task_input, language)[source]

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

get_choice_with_index(index)[source]

Return the choice with index=index

classmethod get_text_fields()[source]

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()[source]

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()[source]

Indicates if problem input type

classmethod parse_problem(problem_content)[source]
class inginious.common.tasks_problems.Problem(problemid, content, translations, taskfs)[source]

Bases: object

Basic problem

abstract check_answer(task_input, language)[source]

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ. the fifth is the problem state (a string send in the input at next submission).

get_id()[source]

Get the id of this problem

get_name(language=None)[source]

Get the name of this problem

get_original_content()[source]

Get a dict fully describing this sub-problem

abstract classmethod get_text_fields()[source]

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

get_translation_obj(language=None)[source]
abstract classmethod get_type()[source]

Returns the type of the problem

gettext(language, *args, **kwargs)[source]
abstract input_is_consistent(task_input, default_allowed_extension, default_max_size)[source]

Check if an input for this problem is consistent. Return true if this is case, false else

abstract input_type()[source]

Indicates if problem input type

classmethod parse_problem(problem_content)[source]