1We're adding the plan 9 file system to akaros.We're bringing in the
   2name space interface, including the ability to do binds, union mounts,
   3and the like. We will extend it to support things we might need,
   4in particular mmap.  We will use it to add things to Akaros we 
   5need, such as virtio drivers, ethernet drivers, and TCP.
   7By bringing this model in, we can make the Akaros interface more powerful,
   8yet simpler. We can remove a number of system calls right away 
   9and yet still have the functions they provide, for example. 
  11This is not a from scratch effort but a code translation. The Plan 9 code
  12deals with very subtle situations and has been hardened over time. No need
  13to relearn what they learned from scratch.
  15Currently we have a lot of the code in and are testing a first device -- 
  16the regression device from NxM.
  20The biggest issue so far is implementing the Plan 9 error handling.
  21In Plan 9, errors are managed via a longjmp-like mechanism. For example,
  22in a call chain such as:
  24 b()
  25  c()
  26   d()
  28It is possible for 'd' to invoke an error that returns to 'a' directly. 
  29This model has many advantages over the standard model, which looks like
  31a{ b();}
  32 b{ if c() return OK; else return -1;}
  33  c{ if d() return OK; else return -1;}
  34   d{ if something return OK; else return -1;}
  36In Plan 9 it can look like this:
  38a{ b();}
  39 b{ c(); something else();}
  40  c{ d(); other thing();}
  41   d{ if something return OK; else error("bad");}
  43Note that the intermediate functions can be written as though nothing
  44went wrong, since anything that goes wrong gets bounced to the first level
  45error handler. 
  47The handling is implemented via something called waserror().
  48In a() we do this:
  51   if (waserror())  { handle error; poperror();}
  52   b();
  53   poperror();
  56and in d we might have this:
  58   do something; 
  59   if (bad) error("bad");
  60   return 0;
  63What if there's more than one error? There can be multiple invocations
  64of waserror in one function:
  66   if (waserror()){ printf("b failed"); poperror(); return -2;}
  67   b();
  68   if (waserror()) { printf("z failed"); nexterror(); }
  69   z();
  70   poperror();
  71   poperror();
  74Every waserror needs a matching poperror or nexterror in the same function as
  75the waserror, covering every exit from the function.  nexterror is like
  76to poperror() then error("str"), but without resetting errstr.
  78Note that the error could have been anywhere in the call chain;
  79we don't care. From the point of view of a(), something failed, and we only
  80know about b() or z(), so we blame them. We also show in this example
  81nexterror(). Nexterror() pops back to the next error in the error stack,
  82which might be in this function or somewhere up the call chain.
  84How do we find out the ultimate blame? Recall that error takes a string,
  85and that can be anything. We can tell from that. 
  87Where does the string in error() go?
  88In Plan 9, it goes into the proc struct; in Akaros,
  89it's copied to a user-mode buffer via set_errstr(). 
  91waserror()/error()/nexterror() manipulate an error call stack, similar to
  92the function call stack. In Plan 9, this stack is maintained in the proc
  93struct. This is cheap in Plan 9, since the compiler has caller-save, and
  94hence each stack entry is a stack and a PC. In a callee-save world, the
  95stack entries are much, much larger; so large that maintaining the stack
  96in the proc struct is impractical.
  98Hence, we've had to make changes that add a bit of inconvenience but
  99leave the code mostly intact. The error code in Akaros is tested in every
 100circumstance at this point due to all the bugs we had in our port
 101of the Plan 9 file system code.
 103So, we'll go from the easiest case to the hardest.
 105Case 1: You're a leaf function that does not use error(). No change.
 107Case 2: You're a leaf function that needs error().  Just call error().
 109Case 3: You're an intermediate function that calls functions that use error(),
 110even though you do not.  No change.
 112Those are in some sense the easier cases. Now it starts to get a
 113bit harder.
 115Case 4: you're a leaf or intermediate function that uses waserror().  You need
 116to use a macro which creates an array of errbuf structs as automatics. The
 117waserror() usage does not change.  The macro is ERRSTACK(x), where x is the
 118number of calls to waserror() in your function. See kern/sys/chan.c.  Every call
 119to waserror needs to have a matching poperror.  You cannot call nexterror or
 120poperror unless you are in the same scope as an ERRSTACK that had a waserror
 123Case 5: you're a root node, i.e. you're the start of a chain of calls
 124via syscall that must do the "root" errbuf setup, so that all error
 125calls eventually return to you. In this case, you need to start the error
 126stack.  This uses the same macro as case 4 (ERRSTACK(x)), for now.
 128Finally, if, in a waserror, you are finished and want to pop out to the
 129next error in the chain, either in the same function or up the call stack,
 130just call nexterror().
 132This can be handy for debugging: in any function that supports error(), i.e.
 133called from a function that called waserror(), simply call error and it bails
 134you out of wherever you are, doing appropriate cleanup on the way out.  No need
 135to add return value processing to intermediate functions; if you try this out
 136you will likely find it is extremely handy. I really miss it on Linux. It made a
 137lot of the port debugging to Akaros a lot easier.
 139There is error checking in error()/waserror()/nexterror().
 140If you overflow or underflow the error stack the kernel will panic.
 141The implementation is in kern/src/error.c, the macros are in
 144Giant warning: if you access any automatic (local) variables inside the waserror
 145block that may have been modified after you started the waserror, those
 146variables need to be volatile.  waserror() is implemented with setjmp and
 147according to one of the man pages:
 149        automatic variables  are  unspecified  after  a  call  to longjmp() if
 150        they meet all the following criteria:
 151        - they are local to the function that made the corresponding setjmp(3)
 152          call;
 153        - their  values  are  changed  between  the  calls  to  setjmp(3)  and
 154          longjmp(); and
 155        - they are not declared as volatile.
 157We could ask the compiler to tell us which variables are potentially clobbered
 158with -Wclobbered, however it is a noisy warning.  It will warn even if the
 159variables are not used in the error case.  That may be because the compiler has
 160a hard time deciding whether a variable is used or not, since we often longjmp
 161from within an error handler, though on gcc 4.9.2, even if we return
 162immediately, we still get a warning.  On a related note, it's not always
 163possible to tell which case is the error handling case - consider the "discard"
 164pattern for waserror.
 166No amount of "returns_twice" or register clobbering or "memory" clobbering is
 167enough.  Think about what happens when the variable is changed after the setjmp
 168call, i.e. farther down in the function.  It's may be in a register, then we
 169call some other function, and that longjmps.  That register value is gone (it
 170might be somewhere else on the stack).  The compiler needs to know when it makes
 171that write that it needs to go onto the stack storage of the automatic variable.
 172That's 'volatile.'
 174The best we can hope for is the compiler to know what variables could be written
 175from one side of the setjmp and used on the other.  Perhaps that will show up,
 176and then we can turn on -Wclobbered.  Until then, we have to be vigilant, or use
 177different patterns for waserror.  Note there are a bunch of bugs with
 178-Wclobbered detection, e.g. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65041.
 180For info on the format of Ms (which are part of the kernel interface):
 181        http://9p.cat-v.org/documentation/rfc/