inginious.common package

Common package: basic library for INGInious, needed by all its components.

Subpackages

• inginious.common.task_file_readers package
  • Submodules
  • inginious.common.task_file_readers.abstract_reader module
  • inginious.common.task_file_readers.yaml_reader module
• inginious.common.tests package
  • Submodules
  • inginious.common.tests.TestBase module
  • inginious.common.tests.TestCourse module
  • inginious.common.tests.TestCustomYaml module
  • inginious.common.tests.TestHookManager module
  • inginious.common.tests.TestTask module

Submodules

inginious.common.asyncio_utils module

Utilities for asyncio

class inginious.common.asyncio_utils.AsyncIteratorWrapper(obj)

Bases: object

A wrapper that converts old-style-generators to async generators using run_in_executor

class inginious.common.asyncio_utils.AsyncProxy(module, loop=None, executor=None)

Bases: object

An asyncio proxy for modules and classes

sync

Return the original sync module/class

inginious.common.base module

Basic dependencies for every modules that uses INGInious

inginious.common.base.dict_from_prefix(prefix, dictionary)

>>> from collections import OrderedDict
>>> od = OrderedDict()
>>> od["problem[q0][a]"]=1
>>> od["problem[q0][b][c]"]=2
>>> od["problem[q1][first]"]=1
>>> od["problem[q1][second]"]=2
>>> AdminCourseEditTask.dict_from_prefix("problem",od)
OrderedDict([('q0', OrderedDict([('a', 1), ('b', OrderedDict([('c', 2)]))])), ('q1', OrderedDict([('first', 1), ('second', 2)]))])

inginious.common.base.directory_compare_from_hash(from_directory, to_directory)

Parameters:

• from_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash
• to_directory – dict in the form {file: (hash of the file, stat of the file)} from directory_content_with_hash

Returns:

a tuple containing two list: the files that should be uploaded to “to_directory” and the files that should be removed from “to_directory”

inginious.common.base.directory_content_with_hash(directory)

Parameters:directory – directory in which the function list the files Returns:dict in the form {file: (hash of the file, stat of the file)}

inginious.common.base.get_json_or_yaml(file_path, content)

Generate JSON or YAML depending on the file extension.

inginious.common.base.hash_file(fileobj)

Parameters:fileobj – a file object Returns:a hash of the file content

inginious.common.base.id_checker(id_to_test)

Checks if a id is correct

inginious.common.base.id_checker_tests(id_to_test)

Checks if a id is correct

inginious.common.base.load_json_or_yaml(file_path)

Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.loads_json_or_yaml(file_path, content)

Load JSON or YAML depending on the file extension. Returns a dict

inginious.common.base.write_json_or_yaml(file_path, content)

Write JSON or YAML depending on the file extension.

inginious.common.course_factory module

Factory for loading courses from disk

class inginious.common.course_factory.CourseFactory(filesystem: inginious.common.filesystems.provider.FileSystemProvider, task_factory, hook_manager, course_class=<class 'inginious.common.courses.Course'>)

Bases: object

Load courses from disk

create_course(courseid, init_content)

Parameters:

• courseid – the course id of the course
• init_content – initial descriptor content

:raise InvalidNameException or CourseAlreadyExistsException Create a new course folder and set initial descriptor content, folder can already exist

delete_course(courseid)

Parameters:courseid – the course id of the course

:raise InvalidNameException or CourseNotFoundException Erase the content of the course folder

get_all_courses()

Returns:a table containing courseid=>Course pairs

get_course(courseid)

Parameters:courseid – the course id of the course

:raise InvalidNameException, CourseNotFoundException, CourseUnreadableException :return: an object representing the course, of the type given in the constructor

get_course_descriptor_content(courseid)

Parameters:courseid – the course id of the course

:raise InvalidNameException, CourseNotFoundException, CourseUnreadableException :return: the content of the dict that describes the course

get_course_fs(courseid)

Parameters:courseid – Returns:a FileSystemProvider pointing to the directory of the course

get_task(courseid, taskid)

Shorthand for CourseFactory.get_course(courseid).get_task(taskid) :param courseid: the course id of the course :param taskid: the task id of the task :raise InvalidNameException, CourseNotFoundException, CourseUnreadableException, TaskNotFoundException, TaskUnreadableException :return: an object representing the task, of the type given in the constructor

get_task_factory()

Returns:the associated task factory

update_course_descriptor_content(courseid, content)

Updates the content of the dict that describes the course :param courseid: the course id of the course :param content: the new dict that replaces the old content :raise InvalidNameException, CourseNotFoundException

inginious.common.course_factory.create_factories(fs_provider, task_problem_types, hook_manager=None, course_class=<class 'inginious.common.courses.Course'>, task_class=<class 'inginious.common.tasks.Task'>)

Shorthand for creating Factories :param fs_provider: A FileSystemProvider leading to the courses :param hook_manager: an Hook Manager instance. If None, a new Hook Manager is created :param course_class: :param task_class: :return: a tuple with two objects: the first being of type CourseFactory, the second of type TaskFactory

inginious.common.courses module

Contains the class Course and utility functions

class inginious.common.courses.Course(courseid, content_description, course_fs, task_factory, hook_manager)

Bases: object

Represents a course

get_descriptor()

Get (a copy) the description of the course

get_fs()

Returns a FileSystemProvider which points to the folder of this course

get_id()

Return the _id of this course

get_task(taskid)

Returns a Task object

get_tasks()

Get all tasks in this course

get_translation_obj(language)

gettext(language, *args, **kwargs)

inginious.common.custom_yaml module

A custom YAML based on PyYAML, that provides Ordered Dicts

inginious.common.custom_yaml.dump(data, stream=None, **kwds)

Serialize a Python object into a YAML stream. If stream is None, return the produced string instead. Dict keys are produced in the order in which they appear in OrderedDicts.

Safe version.

If objects are not “conventional” objects, they will be dumped converted to string with the str() function. They will then not be recovered when loading with the load() function.

inginious.common.custom_yaml.load(stream)

Parse the first YAML document in a stream and produce the corresponding Python object. Use OrderedDicts to produce dicts.

Safe version.

inginious.common.exceptions module

Some type of exceptions used by parts of INGInious

exception inginious.common.exceptions.CourseAlreadyExistsException

Bases: Exception

exception inginious.common.exceptions.CourseNotFoundException

Bases: Exception

exception inginious.common.exceptions.CourseUnreadableException

Bases: Exception

exception inginious.common.exceptions.InvalidNameException

Bases: Exception

exception inginious.common.exceptions.TaskNotFoundException

Bases: Exception

exception inginious.common.exceptions.TaskReaderNotFoundException

Bases: Exception

exception inginious.common.exceptions.TaskUnreadableException

Bases: Exception

inginious.common.hook_manager module

Hook Manager

class inginious.common.hook_manager.HookManager

Bases: object

Registers an manages hooks. Hooks are callback functions called when the inginious.backend does a specific action.

add_hook(name, callback, prio=0)

Add a new hook that can be called with the call_hook function. prio is the priority. Higher priority hooks are called before lower priority ones. This function does not enforce a particular order between hooks with the same priorities.

call_hook(name, **kwargs)

Call all hooks registered with this name. Returns a list of the returns values of the hooks (in the order the hooks were added)

call_hook_recursive(name, **kwargs)

Call all hooks registered with this name. Each hook receives as arguments the return value of the previous hook call, or the initial params for the first hook. As such, each hook must return a dictionary with the received (eventually modified) args. Returns the modified args.

inginious.common.log module

Some common functions for logging

class inginious.common.log.CustomLogMiddleware(app, logger)

Bases: object

WSGI middleware for logging the status in webpy

log(status, environ)

inginious.common.log.get_course_logger(coursename)

Parameters:coursename – the course id Returns:a logger object associated to a specific course

inginious.common.log.init_logging(log_level=10)

Init logging :param log_level: An integer representing the log level or a string representing one

inginious.common.message_meta module

class inginious.common.message_meta.MessageMeta(name, bases, attrs, msgtype)

Bases: type

A MetaClass for messages

Provides message checking on both side of the communication.

Each class depending from this MetaClass MUST have a __init__ function that takes only arguments that are type-hinted, and that ONLY assign the argument to self, under the SAME name.

Moreover, the class should define a argument msgtype for the metaclass, that gives the name of the message when parsed

Example:

class SendNumberToContainer(metaclass=MessageMeta, msgtype=”send_nbr_container”):

def __init__(self, container_id: str, a_number: int):

self.container_id = container_id self.a_number = a_number

DEBUG = True

classmethod load(bmessage)

From a bytestring given by a (distant) call to Message.dump(), retrieve the original message :param bmessage: bytestring given by a .dump() call on a message :return: the original message

class inginious.common.message_meta.ZMQUtils

Bases: object

Utilities that do serializing/unserializing of messages (whose metaclass is MessageMeta)

classmethod recv(socket, skip_first=False)

classmethod recv_with_addr(socket)

classmethod send(socket, obj, send_white=False)

classmethod send_with_addr(socket, addr: bytes, obj)

inginious.common.message_meta.run_tests()

inginious.common.messages module

class inginious.common.messages.AgentHello(*args, **kwargs)

Bases: object

Let the agent say hello and announce which containers it has available

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.AgentJobDone(*args, **kwargs)

Bases: object

Gives the result of a job. A->B.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.AgentJobSSHDebug(*args, **kwargs)

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.AgentJobStarted(*args, **kwargs)

Bases: object

Indicates to the backend that a job started A->B.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendGetQueue(*args, **kwargs)

Bases: object

Send the status of the job queue to the client

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendJobDone(*args, **kwargs)

Bases: object

Gives the result of a job.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendJobSSHDebug(*args, **kwargs)

Bases: object

Gives the necessary info to SSH into a job running in ssh debug mode

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendJobStarted(*args, **kwargs)

Bases: object

Indicates to the backend that a job started

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendKillJob(*args, **kwargs)

Bases: object

Kills a running job. B->A.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendNewJob(*args, **kwargs)

Bases: object

Creates a new job B->A.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.BackendUpdateContainers(*args, **kwargs)

Bases: object

Update the information about the containers on the client, from the informations retrieved from the agents

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.ClientGetQueue(*args, **kwargs)

Bases: object

Ask the backend to send the status of its job queue

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.ClientHello(*args, **kwargs)

Bases: object

Let the client say hello to the backend (and thus register to some events)

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.ClientKillJob(*args, **kwargs)

Bases: object

Kills a running job. B->A.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.ClientNewJob(*args, **kwargs)

Bases: object

Creates a new job B->A.

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.Ping(*args, **kwargs)

Bases: object

Ping message

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.Pong(*args, **kwargs)

Bases: object

Pong message

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

class inginious.common.messages.Unknown(*args, **kwargs)

Bases: object

Unknown message. Sent by a server that do not know a specific client; probably because the server restarted

dump()

Returns:a bytestring containing a black-box representation of the message, that can be loaded using MessageMeta.load.

inginious.common.task_factory module

Factory for loading tasks from disk

class inginious.common.task_factory.TaskFactory(filesystem: inginious.common.filesystems.provider.FileSystemProvider, hook_manager, task_problem_types, task_class=<class 'inginious.common.tasks.Task'>)

Bases: object

Load courses from disk

add_custom_task_file_manager(task_file_manager)

Add a custom task file manager

add_problem_type(problem_type)

Parameters:problem_type – Problem class

delete_all_possible_task_files(courseid, taskid)

Deletes all possibles task files in directory, to allow to change the format

delete_task(courseid, taskid)

Parameters:

• courseid – the course id of the course
• taskid – the task id of the task

:raise InvalidNameException or CourseNotFoundException Erase the content of the task folder

get_all_tasks(course)

Returns:a table containing taskid=>Task pairs

get_available_task_file_extensions()

Get a list of all the extensions possible for task descriptors

get_problem_types()

Returns the supported problem types by this task factory

get_readable_tasks(course)

Returns the list of all available tasks in a course

get_task(course, taskid)

Parameters:

• course – a Course object
• taskid – the task id of the task

:raise InvalidNameException, TaskNotFoundException, TaskUnreadableException :return: an object representing the task, of the type given in the constructor

get_task_descriptor_content(courseid, taskid)

Parameters:

• courseid – the course id of the course
• taskid – the task id of the task

:raise InvalidNameException, TaskNotFoundException, TaskUnreadableException :return: the content of the task descriptor, as a dict

get_task_descriptor_extension(courseid, taskid)

Parameters:

• courseid – the course id of the course
• taskid – the task id of the task

:raise InvalidNameException, TaskNotFoundException :return: the current extension of the task descriptor

get_task_fs(courseid, taskid)

Parameters:

• courseid – the course id of the course
• taskid – the task id of the task

:raise InvalidNameException :return: A FileSystemProvider to the folder containing the task files

update_cache_for_course(courseid)

Clean/update the cache of all the tasks for a given course (id) :param courseid:

update_task_descriptor_content(courseid, taskid, content, force_extension=None)

Update the task descriptor with the dict in content :param courseid: the course id of the course :param taskid: the task id of the task :param content: the content to put in the task file :param force_extension: If None, save it the same format. Else, save with the given extension :raise InvalidNameException, TaskNotFoundException, TaskUnreadableException

inginious.common.tasks module

Task

class inginious.common.tasks.Task(course, taskid, content, task_fs, translations_fs, hook_manager, task_problem_types)

Bases: object

Contains the data for a task

allow_network_access_grading()

Return True if the grading container should have access to the network

check_answer(task_input, language)

Verify the answers in task_input. Returns six values 1st: True the input is currently valid. (may become invalid after running the code), False else 2nd: True if the input needs to be run in the VM, False else 3rd: Main message, as a list (that can be join with

or <br/> for example)

4th: Problem specific message, as a dictionnary (tuple of result/text) 5th: Number of subproblems that (already) contain errors. <= Number of subproblems 6th: Number of errors in MCQ problems. Not linked to the number of subproblems

get_course()

Return the course that contains this task

get_course_id()

Return the courseid of the course that contains this task

get_custom_run_cmd()

Returns a string containing the custom run command to be run inside the container, instead of the default run file. Returns None if no custom command has been set

get_environment()

Returns the environment in which the agent have to launch this task

get_fs()

Returns a FileSystemProvider which points to the folder of this task

get_hook()

Returns the hook manager parameter for this task

get_id()

Get the id of this task

get_limits()

Return the limits of this task

get_order()

Get the position of this task in the course

get_problems()

Get problems contained in this task

get_response_type()

Returns the method used to parse the output of the task: HTML or rst

get_translation_fs()

Return the translation_fs parameter for this task

get_translation_obj(language)

gettext(language, *args, **kwargs)

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for a task is consistent. Return true if this is case, false else

inginious.common.tasks_code_boxes module

inginious.common.tasks_problems module

Tasks’ problems

class inginious.common.tasks_problems.CodeProblem(task, problemid, content)

Bases: inginious.common.tasks_problems.Problem

Code problem

check_answer(_, __)

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ.

classmethod get_text_fields()

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()

Indicates if problem input type

classmethod parse_problem(problem_content)

class inginious.common.tasks_problems.CodeSingleLineProblem(task, problemid, content)

Bases: inginious.common.tasks_problems.CodeProblem

Code problem with a single line of input

classmethod get_type()

Returns the type of the problem

class inginious.common.tasks_problems.FileProblem(task, problemid, content)

Bases: inginious.common.tasks_problems.Problem

File upload Problem

check_answer(_, __)

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ.

classmethod get_text_fields()

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()

Indicates if problem input type

classmethod parse_problem(problem_content)

class inginious.common.tasks_problems.MatchProblem(task, problemid, content)

Bases: inginious.common.tasks_problems.Problem

Display an input box and check that the content is correct

check_answer(task_input, language)

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ.

classmethod get_text_fields()

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()

Indicates if problem input type

classmethod parse_problem(problem_content)

class inginious.common.tasks_problems.MultipleChoiceProblem(task, problemid, content)

Bases: inginious.common.tasks_problems.Problem

Multiple choice problems

allow_multiple()

Returns true if this multiple choice problem allows checking multiple answers

check_answer(task_input, language)

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ.

get_choice_with_index(index)

Return the choice with index=index

classmethod get_text_fields()

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

classmethod get_type()

Returns the type of the problem

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()

Indicates if problem input type

classmethod parse_problem(problem_content)

class inginious.common.tasks_problems.Problem(task, problemid, content)

Bases: object

Basic problem

check_answer(task_input, language)

Check the answer. Returns four values: the first is either True, False or None, indicating respectively that the answer is valid, invalid, or need to be sent to VM the second is the error message assigned to the task, if any (unused for now) the third is the error message assigned to this problem, if any the fourth is the number of errors in MCQ; should be zero when not a MCQ.

get_id()

Get the id of this problem

get_name(language=None)

Get the name of this problem

get_original_content()

Get a dict fully describing this sub-problem

get_task()

Get the task containing this problem

classmethod get_text_fields()

Returns a dict whose keys are the keys of content dict and val is True if value of content[key] is human-readable text

get_translation_obj(language=None)

classmethod get_type()

Returns the type of the problem

gettext(language, *args, **kwargs)

input_is_consistent(task_input, default_allowed_extension, default_max_size)

Check if an input for this problem is consistent. Return true if this is case, false else

input_type()

Indicates if problem input type

classmethod parse_problem(problem_content)