Have switch_{to,back} use an opaque uintptr_t