computer-science/coursepages/ostep/Scheduling-xv6-lottery.md

all thanks to palladian

General Tips

• Read chapter 9 in the OSTEP book and watch the video for discussion 5. Lottery ticket schedulers aren't discussed in the lectures, so you really do have to read the book for this one.

• In general, you can't use C standard library functions inside the kernel, because the kernel has to initialize before it can execute library binaries.

• The xv6 kernel has a "kernel version" of printf; it takes an additional integer argument that tells it whether to print to stdout or stderr. Note that it can only handle basic format strings like "%d" and not more complex ones like "%6.3g"; you can deal with this by manually adding spaces instead. It also has another similar function, cprintf.

• If you do want to use other library functions that aren't available inside the kernel (pseudo random number generators), you can see how those functions are implemented in P.J. Plauger's book, The Standard C Library, and then implement them yourself.

  Implementation

  • You'll have to modify the same files you did in Project 1b in order to add the two new system calls.
  • In order to understand how processes are created, remember that they start in the EMBRYO state before they become RUNNABLE--you'll have to find where that happens.
  • System calls always have argument type void, so take a look at how system calls like kill and read manage to work around that limitation and get arguments (like integers and pointers) from user space. You might have to back a few steps in the chain that executes them.
  • Make sure you're including types.h and defs.h wherever you need to access code from other parts of the kernel.
  • In order to create the xv6 command ps, look at how cat, ls, and ln are implemented. Make sure to modify the Makefile to include the source code for your ps command.

Spoilers below!

Solution walk through

• Start from a fresh copy of the xv6 source code.

• argint and argptr are important functions. So syscalls take no arguments, but in reality, in user code you want to pass arguments to them.

• So the way you do that is the kernel will call the syscall, say, sys_kill() with no arguments, then sys_kill will use argint() to get the arguments from the call stack, then pass that to a function kill(int pid).

• So you can see there's a bunch of extern int sys_whatever function declarations below that; that means that these functions are defined in another file and should be pulled in from there as function pointers.

• And these sys_whatever functions are basically just wrappers for the real syscall, which doesn't have the sys_ at the beginning. So you need to add sys_settickets and sys_getpinfo to that list of function declarations.

• Then there's an array of function pointers; it's using this old-school C way of initializing arrays where you can do int arr[] = { [0] 5, [1] 7}.

• And the names inside the square brackets SYS_fork, etc. are defined as preprocessor macros in another header file syscall.h.

• So you need to add two more entries in the array with function pointers to sys_settickets and sys_getpinfo, and then you need to define SYS_settickets and SYS_getpinfo in the relevant header file.

• So then all these sys_ wrapper functions are defined in sysproc.c.

• So there, you need to create int sys_settickets(void) and int sys_getpinfo(void).

• The real settickets function will need an int argument, so you need to use argint there to grab it from the call stack and pass it to settickets; similarly, getpinfo will need a pointer, so you'll use argptr.

• Also, there's an extra condition in the if statement for sys_settickets; that's because you're not allowed to use a number of tickets below 1.

• So then there's some assembly code that needs to run for each of the system calls; luckily, it's just a pre-written macro, so you don't have to write any assembly. that's in usys.S.

• So you just add two lines at the bottom to create macros for SYSCALL(settickets) and SYSCALL(getpinfo)

• Last part for the syscalls: you need to declare them in a header file for user code to be able to call them. that's in user.h.

• So struct pstat will be properly defined in pstat.h, but you need to declare it in user.h as well so that user code doesn't complain when it sees it.

• Basically, any user code that uses syscalls or C (really, xv6) standard library functions will have to include user.h.

• So, so far, that's everything for the two system calls as far as the OS is concerned; now we just have to actually implement them with the regular functions settickets and getpinfo, then implement the scheduler and the ps program.

• pstat.h is not for the scheduler, but for the ps program, which will work somewhat like the Linux ps. pstat.h is just to define the struct pstat, but there's no .c file to go with it.

• So the scheduler will work by assigning 1 ticket by default to each process when it's created; then processes can set their own tickets using the settickets system call.

• so first we need to make sure each process tracks its own tickets, then we need to assign a default of 1 ticket when creating them, then we need to write settickets.

• the first part is in proc.h: processes are represented as a struct proc, so we add a new member for int tickets.

• the int ticks member is for ps; I'll come back to that.

• One other thing to note in proc.h is the enum procstate: you can see all the possible process states there. EMBRYO means it's in the process of creation; so what i did was grep for EMBRYO to find where the process was created in order to set the default tickets to 1. Turns out it's in proc.c.

• Inside proc.c, there's a function allocproc, which initializes a process.

• There's a process table called ptable, and allocproc looks through it to find an unused process.

• Then when it does find it, it goes to create it; i added p->tickets = 1; there.

• okay so the next change is to fit one of the requirements: child processes need to inherit the number of tickets from their parent process.

• So child processes are created with fork, which is in the same file.

• In fork, curproc is the current process, and np is the new process.

• So i set np->tickets = curproc->tickets.

• So the scheduler needs to generate a pseudo random number, then it should iterate through the process table with a counter initialized to 0, adding the number of tickets for each process to the counter. once the counter is greater than the pseudo random number, it stops and runs that process.

• So I ended up looking in P.J. Plauger's The Standard C Library, which is just a big book of all the source code for the C library with commentary. It's pretty good; I don't know if it's still written that way though because the book is from the 80s.

• So i just implemented C's rand and srand functions. srand sets a random seed (not so random, as you'll see later), then rand turns it into a pseudo random integer.

• There's a bunch of type magic going on there between changes back and forth from integers to unsigned integers; that's to avoid signed integer overflow, which causes undefined behavior. unsigned integer overflow is okay though.

• I only made one change to make it faster, which was to write & 32767 instead of % 32768.

• So you'll see the "random" seed i used: the number of ticks, which i think counts the number of timer interrupts so far.

• Which is totally not random at all, since the first time this program gets run, it'll be 0, then 1, then 2, etc.

• So there's some lines about counting ticks; that was for ps, not the scheduler.

• The main change to make it a lottery scheduler is the counter variable.

• And adding a for loop to count the total number of tickets that have been distributed.

• So then at the very bottom of this file is the implementation of settickets and getpinfo.

• So after initializing counter and totaltickets, there's for loop that counts the total numbers of tickets that have gone out to processes.

• Then we get the winning ticket.

• Let's discuss the original source code first. So first you acquire the lock. You'll release it at the very end. But in between, you have a for loop that iterates over all the processes in ptable.

• Specifically, it iterates over only the processes in RUNNABLE state; if a process isn't RUNNABLE, it just continues on to the next one. (This is for the round-robin scheduling mechanism that's already in the code.)

• So now it's gonna switch to the very first RUNNABLE process it finds. Like, switching to executing it.

• So first, c represents the current CPU. so it sets the current CPU to run the process it found with c->proc = p;.

• Then it calls this function, switchuvm(p), which sets up the virtual memory address space for p. Then it sets the process's state to RUNNING.

• And then swtch is where the magic happens: that one swaps out the register contents of the OS and scheduler content with the saved-in-memory register contents of the process p.

• So as soon as swtch executes, the CPU will continue executing instructions, but now they're the process's instructions. So this scheduler function just hangs there.

• Eventually, when a timer interrupt goes off, the processor will use another swtch call but with the arguments reversed to swap the scheduler's register contents from memory into the CPU's registers and save the process's register contents. At which point execution will continue at this exact point.

• So now switchkvm will set up the kernel's virtual memory address space.

• These 5 lines are the context switch:

  c->proc = p;
  switchuvm(p);
  p->state = RUNNING;
  
  swtch(&(c->scheduler), p->context);
  switchkvm();
  

• So then we go on to the next iteration of the inner for loop, which finds the next RUNNABLE process and repeats.

• Only once we've executed all the RUNNABLE processes do we exit the inner for loop and release the lock.

• Original source code is structured like this (this is pseudocode):

  while (1) {
    iterate over processes:
      if not runnable:
        continue
      run it
  

• New code is structured like this (this is pseudocode):

  while (1) {
    count the total tickets allotted to all processes // one for loop here
    get the winning ticket number
    iterate over processes: // another for loop here
      if not runnable:
        continue
      add its tickets to counter
      if counter <= winning ticket number:
        continue
      run it
  

• We ignore the tickets of non-RUNNABLE processes.

• So the tickets aren't numbered; each process just has a set amount of tickets, and we just count up until we've passed n tickets, where n is the winner.

• For example if proc A has 5 tickets and proc B has 7, proc C has 2. if the winning number is 3, then A would run; if it's 8, then B would run; if it's 12, then C would run.

• A winner in 0-4 would be A, 5-11 would be B, and 12-13 would be C.

• So settickets is pretty basic: you just acquire a lock, set the tickets for the process, release the lock.

• For getpinfo basically it works like this:

• p is a pointer a struct pstat, as defined in pstat.h. each of its members is an array, with one entry per process.

• Check for a null pointer.

• Iterate over the process table and set proc_i to the i-th process.

• Set the i-th entry of each member of p to the value for this process.

• One last bookkeeping piece: we need to add declarations for struct pstat and the settickets and getpinfo system calls in defs.h.

• And then the last file is ps.c, which implements the ps program, similar to Linux's ps. it just calls getpinfo to fill a struct pstat, then prints out the info for each process in use.

• And then you just modify the Makefile to include ps.c in the compilation, and we're done!

• Oh and this is why we needed the ticks in the scheduler: ps will print out how long each process has run.

• So it needs to time the number of ticks that it actually executed.

• FINALLY run make qemu in the /src directory to make sure it's all working.