Add LITMUS^RT core implementation

This patch adds the core of LITMUS^RT:

 - library functionality (heaps, rt_domain, prioritization, etc.)
 - budget enforcement logic
 - job management
 - system call backends
 - virtual devices (control page, etc.)
 - scheduler plugin API (and dummy plugin)

This code compiles, but is not yet integrated with the rest of Linux.

Squashed changes:

LITMUS^RT Core: add get_current_budget() system call

Allow userspace to figure out the used-up and remaining budget
of a task.

Adds deadline field to control page and updates it when setting up jobs for release.

Adds control page deadline offset

ftdev: respect O_NONBLOCK flag in ftdev_read()

Don't block if userspace wants to go on doing something else.

Export job release time and job sequence number in ctrl page

Add alternate complete_job() default implementation

Let jobs sleep like regular Linux tasks by suspending and waking them
with a one-shot timer. Plugins can opt into using this implementation
instead of the classic complete_job() implementation (or custom
implementations).

Fix RCU locking in sys_get_rt_task_param()

sys_get_rt_task_param() is rarely used and apparently attracted some
bitrot.

Free before setting NULL to prevent memory leak

Add hrtimer_start_on() support

This patch replaces the previous implementation of hrtimer_start_on() by
now using smp_call_function_single_async() to arm hrtimers on remote
CPUs.

Expose LITMUS^RT system calls via control page ioctl()

Rationale: make LITMUS^RT ops available in a way that does not create
merge conflicts each time we rebase LITMUS^RT on top of a new kernel
version. This also helps with portability to different architectures,
as we no longer need to patch each architecture's syscall table.

Pick non-zero syscall ID start range

To avoid interfering with Linux's magic reserved IOCTL numbers

Don't preempt before time check in sleep_until_next_release()

Avoid preempting jobs that are about to go to sleep soon anyway.

LITMUS^RT proc: fix wrong memset()

TRACE(): add TRACE_WARN_ON() helper

Useful to replace BUG_ON() and WARN_ON() with a non-fatal
TRACE()-based equivalent.

Add void* plugin_state pointer to task_struct

LITMUS^RT: split task admission into two functions

Plugin interface: add fork_task() callback

LITMUS^RT: Enable plugins to permit RT tasks to fork

one-shot complete_job(): set completed flag

This could race with a SIGSTOP or some other forced suspension, but
we'll let plugins handle this, should they actually care.

FP: add list-based ready queue

LITMUS^RT core: add should_wait_for_stack() callback

Allow plugins to give up when waiting for a stack to become available.

LITMUS^RT core: add next_became_invalid() callback

LITMUS^RT core: add post-migration validation callback

LITMUS^RT core: be more careful when pull-migrating tasks

Close more race windows and give plugins a chance to validate
tasks after they have been migrated.

Add KConfig options for timer latency warnings

Add reservation creation API to plugin interface & syscalls

LITMUS^RT syscall: expose sys_reservation_create() via ioctl()

Add reservation configuration types to rt_param.h

Add basic generic reservation-based scheduling infrastructure

Switch to aligned quanta by default.

For first-time users, aligned quanta is likely what's expected.

LITMUS^RT core: keep track of time of last suspension

This information is needed to insert ST_COMPLETION records for
sporadic tasks.

add fields for clock_nanosleep() support

Need to communicate the intended wake-up time to the plugin wake-up handler.

LITMUS^RT core: add generic handler for sporadic job arrivals

In particular, check if a job arrival is triggered from a
clock_nanosleep() call.

add litmus->task_change_params() callback to plugin interface

Will be used by adaptive C-EDF.

Call litmus->task_change_params() from sys_set_rt_task_param()

Move trace point definition to litmus/litmus.c

If !CONFIG_SCHED_TASK_TRACE, but CONFIG_SCHED_LITMUS_TRACEPOINT, then
we still need to define the tracepoint structures.

This patch should be integrated with the earlier sched_task_trace.c
patches during one of the next major rebasing efforts.

LITMUS^RT scheduling class: mark enqueued task as present

Remove unistd_*.h

rebase fix: update to new hrtimer API

The new API is actually nicer and cleaner.

rebase fix: call lockdep_unpin_lock(&rq->lock, cookie)
The LITMUS^RT scheduling class should also do the LOCKDEP dance.

LITMUS^RT core: break out non-preemptive flag defs
Not every file including litmus.h needs to know this.

LITMUS^RT core: don't include debug_trace.h in litmus.h
Including debug_trace.h introduces the TRACE() macro, which causes
symbol clashes in some (rather obscure) drivers.

LITMUS^RT core: add litmus_preemption_in_progress flags
Used to communicate that a preemption is in progress. Set by the
scheduler; read by the plugins.

LITMUS^RT core: revise is_current_running() macro

1 files changed, 16 insertions, 1 deletions

diff --git a/litmus/trace.c b/litmus/trace.c
index 7d698aac257d..0724e31a49ff 100644
--- a/litmus/trace.c
+++ b/litmus/trace.c
@@ -254,6 +254,19 @@ feather_callback void save_cpu_timestamp_irq(unsigned long event,
                             0, RECORD_LOCAL_TIMESTAMP);
 }
 
+feather_callback void save_cpu_task_latency(unsigned long event,
+                                            unsigned long when_ptr)
+{
+        lt_t now = litmus_clock();
+        lt_t *when = (lt_t*) when_ptr;
+        lt_t delta = now - *when;
+
+        write_cpu_timestamp(event, TSK_RT,
+                            0,
+                            0, LOCAL_IRQ_COUNT, 0,
+                            delta, DO_NOT_RECORD_TIMESTAMP);
+}
+
 /* Record to remote trace buffer */
 feather_callback void msg_sent_to(unsigned long event, unsigned long to)
 {
@@ -422,11 +435,13 @@ static int alloc_timestamp_buffer(struct ftdev* ftdev, unsigned int idx)
 
 static void free_timestamp_buffer(struct ftdev* ftdev, unsigned int idx)
 {
+        struct ft_buffer* tmp = ftdev->minor[idx].buf;
+        smp_rmb();
         ftdev->minor[idx].buf = NULL;
         /* Make sure all cores have actually seen buf == NULL before
          * yanking out the mappings from underneath them. */
         smp_wmb();
-        free_ft_buffer(ftdev->minor[idx].buf);
+        free_ft_buffer(tmp);
 }
 
 static ssize_t write_timestamp_from_user(struct ft_buffer* buf, size_t len,