Why fibers¶
Fibers was created because I wanted to implement an API which looked pretty much like a thread but with greenlet, but the API offered by greenlet got in the way, so I decided to try and extract the minimum ammount of funcionality I needed and make the API I wanted to use.
Note the use of the first person singular here, this was created for myself, I’m sharing it because it may also help others. If this helped you, let me know, I’d like to hear your use case!
Early binding¶
This was the first thing I wanted to change. In greenlet, the target function can
be specified in __init__
but it can be changed later (before the greenlet is
switched to for the first time) and arguments (and keyword arguments) must be passed
to switch
. I wanted to ‘bind’ the callable and arguments as early as possible
(in __init__
, actually) pretty much like threads do.
With greenlet
def run(*args, **kwargs):
print args
print kwargs
g = greenlet(run)
g.switch(1, 2, 3, foo='bar')
With fibers
def run(*args, **kwargs):
print args
print kwargs
f = Fiber(target=run, args=(1, 2, 3), kwargs={'foo':'bar'})
f.switch()
This is a bigger change than it seems, because it also means that switch
only
gets one argument, which is what the called will get. In greenlet the caller may
get a tuple, a dictionary or a tuple containing another tuple and a dictionary,
depending on the values passed to switch
.
Graceful failures¶
Greenlet tends to be quite graceful with failures, which may lead to unexpected behavior.
With greenlet
def run():
return 42
g = greenlet(run)
res = g.switch()
# res is 42
res = g.switch()
# res is the empty tuple
With fibers
def run():
return 42
f = Fiber(run)
res = f.switch()
# res is 42
res = f.switch()
# raises fiibers.error exception because f has ended
Garbage collection magic¶
This is a tricky one. When greenlets are garbage collected, they are switched into
(if they where running) and GreenletExit
exception is raised in them. This means
that if the code caught this exception, it could resurrect the object.
In fibers, on the contrary, this doesn’t happen and if a fiber is gc’d while alive, it’s not switched into and no exception is raised in it.
This means that the programmer needs to take care and throw
into the fibers which
he/she wishes to kill. It may look like a burden at a first sight, but I believe
it’s for the best.
Stack slicing implementation¶
I’ll be honest: I’m not smart enough to write the assembly code required to perform the actual stack slicing for the different platforms. That’s why I stand on the shoulders of giants, in this case, I leveraged the tiny stacklet library hidden inside PyPy, which is used to implement continuations, as well as the greenlet module (in PyPy, that is).
Stacklet implements save, restore and stack switch operations in assembly, which compilers won’t touch, so there should be no issues regardless of the optimizations used and the ‘smartness’ of the compiler. The greenlet implementation, on the other hand, only implements the stack switch operation in assembly, and workarounds have been needed every now and then, as compilers have become ‘smarter’.
On a personal note, using a library such as stacklet makes the code look simpler, since fibers is a wrapper over it with a cherry on top, and all the scary bits are hidden inside the library itself.