I've written a single page implementation of coroutines in C using setjmp/longjmp and just a little bit of inline assembly. The article is at embeddedrelated.com. Here's an example C program using coroutines – equivalent to the Python code:
def iterate(max_x, max_y):
for x in range(max_x):
for y in range(max_y):
yield x,y
for x,y in iterate(2,2):
print x,y
In C:
#include <stdio.h>
#include "coroutine.h"
typedef struct {
coroutine* c;
int max_x, max_y;
int x, y;
} iter;
void iterate(void* p) {
iter* it = (iter*)p;
int x,y;
for(x=0; x<it->max_x; x++) {
for(y=0; y<it->max_y; y++) {
it->x = x;
it->y = y;
yield(it->c);
}
}
}
#define N 1024
int main() {
coroutine c;
int stack[N];
iter it = {&c, 3, 2};
start(&c, &iterate, &it, stack+N);
while(next(&c)) {
printf("x=%d y=%dn", it.x, it.y);
}
}
Yeah it's a bit longer, or perhaps more than a bit longer. But it's still very useful at times. Check it out!
75 comments ↓
In your python routine, should you not replace the "range" with "xrange," at least to preserve the spirit of using a pure python iterable?
This line looks like undefined behavior to me, but probably it is reliably compiled in practice:
start_params* p = ((start_params*)sp)-1;
@Tanim Islam: yeah, I just figured "range" is more mnemonic for someone not knowing Python and so reading it as pseudocode.
@John Regehr: very possibly it's undefined :) I'm not very good at C's semantics, really; I know I can carve out a struct out of a byte array in some cases but not others, and I was never completely sure which are which. With this type of code, I know I'll be able to maintain it as compilers/platforms change over the years but it's not because I'm trusting my formal understanding of C as much as because I trust that I can make sense of the assembly coming out of the compiler and how it's wrong and why. I wonder if start() in general is at all possible to defend in terms of C's semantics – for instance the fiddling with $sp and $fp, not to mention the FRAME_SZ constant.
BTW I discovered that on my platform for which I originally started writing this stuff, setjmp is broken in that it assumes 32b floating point registers on 32b processor variants; that's because the newlib version is from 1993 when there were just two variants of the machine – all registers 32b or all registers 64b, ints as well as floats. So the part which is supposedly correct according to C's semantics will need to be rewritten…
also, xrange is not part of python 3… and range is now a Sequence
Hi – this is cool. I'm going to play with it over the weekend. :)
I'm wondering if you've seen this other implementation of C coroutines? It probably has an uglier API, but I think it works by expanding macros into switch statements, which is possibly more portable, I'm not sure. It'd be interesting to see a comparison between the two.
http://www.chiark.greenend.org.uk/~sgtatham/coroutines.html
Indeed it's more portable; the downside is, you can only yield at the top level of your coroutine – that is, the coroutine can't itself call functions and have one of them yield control to the producer. That's because calling functions that yield requires multiple call stacks.
Thanks for the article,
Now there is another trick for solving lack of getcontext: if jmp_buf is an array of values then one can infer the index that stores the stack pointer and change jmp_buf directly.
Also: a very useful feature for coroutines is to have a guard page at the end of the stack; otherwise one will never know if the program ran out of the limited stack space (if one wants to service lots of coroutines, then that's what it gets)
You could change the sp (and fp) directly by knowing which libc you're targeting; you'd need to deal with the fact that your local variables could have been rendered invalid by that when comping back from setjmp through longjmp though.
As to guard pages – if you have paging, you probably have getcontext… Non-Unix operating systems might have paging or they could have other memory protection features; I'm on an OS which doesn't use paging, for instance.
For c++ programmers Boost appears to include Coroutines too
http://www.boost.org/doc/libs/1_53_0/libs/coroutine
/doc/html/coroutine/intro.html
Hi Yosef, Nice writeup! So, I have tried this and it works on MACOSX 64bit and Windows 32bit very nicely, but Win64/x64 inline asm is not allowed in visual c++.
Any ideas how to write the set/get_sp/fp functions on _WIN64 target? I'm trying MASM but I think the function wind/unwind stuff is getting in the way, I'm no ASM guru.
I tried the TonyFinch/Picoro stuff from his github and it's awesomely portable and simple to read, but limited by stack space. so I am preferring your method which allows me to create my own stack space on the heap, and patch it in to each coroutine…
Wishing I could use ucontext but it seems deprecated by POSIX group and broken on MACOSX (my code which works on linux hangs on MACOSX). And finally, windows Fibers seems limited to stack space as well, I get around 2028 coroutines… (and it isn't portable). I even found a ucontext implementation for windows.
Some work I did on top of ucontext, for the curious: http://www.subatomicglue.com/secret/coro/readme.html
No inline asm at all? How come?
My solution for Windows was actually to cross-compile with mingw, so I don't know how to do it in VS.
Right, visual C++ doesn't support inline ASM for x64 targets, I guess they didn't want to have it interfering with their optimizer
https://www.google.com/?gws_rd=ssl#q=visual+c%2B%2B+no+inline+assembly+x64
This page descries the rationale for deprecating ucontext: http://pubs.opengroup.org/onlinepubs/009695399/functions/makecontext.html
They think that making people port to threads (i.e. pthreads) is easier/better than writing a new interface for ucontext which conforms to C standards. I guess they leave us with no POSIX way to implement coroutines, which seems shortsighted to me.
To be fair (?), C has the problem of handling stack overflow in coroutines that is kinda solvable for the OS, especially if you have plentiful pointer bits (by trapping stack overflow and mapping more memory pages on demand), but not solvable in userspace on today's systems. Solved in Go by having a calling convention handling stack overflow, AFAIK… So deprecating ucontext is like saying, "we take away a portable method to implement a construct but not really", which can seem reasonable; "if you're willing to risk stack overflow then you should also be willing to fiddle with assembly." On the other hand, C always lets you shoot yourself in the foot… so why not make this shot easier is not really clear…
Hi, Yossi. Cool write up. Coroutines have really opened my eyes to how flexible and clean iteration can be (C++ iterators are put to shame for their verbosity and implementation concerns). I think your one page API could be made to look more like the Python example if you're a glutton for punishment and want to mix variable argument lists and setjmp/longjmp/makecontext. But I'm at a loss of how to go about it, as my experience with setjmp/longjmp is limited, and I can't reason about the resulting control flow. Do you have any ideas as to where I could improve my setjmp mastery?
I think setjmp/longjmp sink in once you fiddle with them for a few hours, in the same way recursion does. C varargs are different though, they're always ugly at the callee code and never safe. I think you need C++ templates to make this look pretty (as in, any coroutine prototype works and you call it naturally.)
Thanks for getting back to me so quickly. I was thinking of an API that adds varargs to next() and yield() like this.
void yield(coroutine *c, void *arg0, …);
int next(corotuine *, void *arg0, …);
start() would be left unchanged. If arg0 is NULL, the code would terminate vararg handling. This API would require a simple change to the coroutine struct (a fixed-size buffer of void *s, and void ** pointing the next slot). It's not the prettiest solution, but it seems to work. I have no PC right now, so I can't run GDB to figure out why it crashes. I'm stuck with online C compilers ATM.
Have you heard of libffi? It allows you to call any function when you only know the API at runtime. It's the perfect solution if for calling an arbitrary coroutine. Here's an example Simon Tatham wrote: http://www.chiark.greenend.org.uk/doc/libffi-dev/html/Simple-Example.html
The problem with varargs is you need some sort of a convention for figuring out which arguments were actually passed (printf has one hairy one which isn't suitable for anything except printf.) With these complications, I'm not sure the code is gonna be prettier or safer with varargs than with a hand-coded params struct, and maybe the reverse. A C++11 variadic template instantiating a tuple struct from the parameters and then calling the entry point functions with these arguments might be as clean as what you're looking for.
My convention for this particular function is that the list of args is terminated with a NULL pointer. You were right about setjmp; I get it now. One last note, I got the API changes working, but it required volatile-qualifying x and y in iterate(). It printed garbage without it, and it seems that it's UB otherwise.
I got it working. Here's the gist of it: https://gist.github.com/anonymous/e9a4f29a2e62452a36ba7488d7422242
I like this API a little more because it doesn't require me to make a struct. For mixing varargs and setjmp, I think it's a pretty clean interface. Once I got my head around setjmp (next() longjmps back into start(), and calls the entry point function), it all seemed to fall into place for me.
I guess I prefer almost anything to varargs except when I'm ultimately forwarding arguments for printf, because of how unsafe and unreadable accessing varargs is, but to each his own :-) I don't think my code makes a good library anyway, it's more of an example one would mold to suit their use case, which I guess is what you did there.
I think your right. Varargs are ugly and error prone. My modification asserts when not enough arguments are missing. I haven't worked out all the corner cases yet. Maybe explicitly passing the expected number of arguments is safer. I saw a simple Lisp implementation use varargs for constructing cons-cells. I thought it was an alright solution instead of having multiple constructor functions. Would you be interested in seeing the code on Gist? I could format, comment, and post it.
Got it working on Windows, using the OS's fiber functions.
Here it is on gist. https://gist.github.com/anonymous/765f189ddd13665256b140ade76d0533
Chemical Peel Treatments to help a variety of skin conditions. Chemical peels can help with treatment of scars, uneven skin tone and general skin care.
[url=https://chemicalpeel.in/category/chemical-peel-results]chemical peel results[/url]
PPT – (Re) Playing with (Blind) SQL Injection PowerPoint presentation free to download – id: 6b9gy Center
im 16 years old and i have a lump under my armpit (inside my body)? | Yahoo Respuestas
[url=http://armpit.info/i-have-a-lump-under-my-armpit-which-doctor-should-i-visit/]i have a lump under my armpit[/url]
Great, yahoo took me stright here. thanks btw for this. Cheers!
I more or less share your take on this topic and look forward to upcoming posts and comments here at yosefk.com. Keep up the good work!
In my opinion, yosefk.com does a great job of covering subject matter of this sort. Even if sometimes intentionally controversial, the information is generally well-written and stimulating.
Appreciate this site– extremely easy to navigate and a lot of stuff to consider!
This does interest me
Hey, yahoo lead me here, keep up nice work.
Very interesting points you have remarked, appreciate it for putting up.
Enjoyed reading through this, very good stuff, thankyou .
I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.
I dugg some of you post as I thought they were very beneficial invaluable
This does interest me
You got yourself a new follower.
I have interest in this, thanks.
I really enjoy examining on this web , it has got good stuff .
I must say got into this web. I found it to be interesting and loaded with unique points of interest.
Respect to website author , some wonderful entropy.
Hey, yahoo lead me here, keep up good work.
You got yourself a new follower.
I conceive you have mentioned some very interesting details , appreciate it for the post.
Enjoyed reading through this, very good stuff, thankyou .
I really enjoy examining on this blog , it has got great stuff .
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
Thank You for this.
Cheers, great stuff, Me like.
This does interest me
Parasite backlink SEO works well :)
Very interesting points you have remarked, appreciate it for putting up.
I am glad to be one of the visitors on this great website (:, appreciate it for posting .
I like, will read more. Cheers!
Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.
Found this on google and I’m happy I did. Well written article.
The next step of the mystery is to understand the order of the pyramid. This is your third secret lead. 517232125
I conceive you have mentioned some very interesting details , appreciate it for the post.
Respect to website author , some wonderful entropy.
Purchase Amoxicillin No Prescription Kamagra Comprar Ajanta Pharma [url=http://tadalafonline.com]cialis cheapest online prices[/url] Puedo Tomar Viagra Con 18 Anos
I must say, as a lot as I enjoyed reading what you had to say, I couldnt help but lose interest after a while.
Hello, glad that i saw on this in google. Thanks!
Can Amoxicillin Cause Stomach Pain Antibiotics Names Amoxil Cheap 100mg Cialis Pills [url=http://orderviapills.com]viagra[/url] Propecia Buy Sydney
great advice you give
amazing content thanks
This blog is amazing! Thank you.
I conceive you have mentioned some very interesting details , appreciate it for the post.
Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.
I am 43 years old and a mother this helped me!
I am 43 years old and a mother this helped me!
I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .
I really enjoy examining on this internet site , it has got cool content .
The duck down have proven to be controlling throughout attack with bluebirds on top of that their very own murders
A bluebird in virginia. a vigorous sounding birdlovers who else fit in with the va Bluebird humanity work to save the indigenous songbird. He contends that if he scraped twigs and leaves from the foot of our mail, He would never know they were a proto nesting on a defended species of fowl.
your husband thought it was few group young people, participating a bogus.
never any, the guy learned of the secret soon after, it is the process within your chickadee, that "tooth cavity nester, by – demolishing this nesting, my associate possessed dishonored its Migratory chook Treaty procedure.
there has to be just about anything he are capable of doing, [url=https://www.cashbackholic.com/cashback-rebates-QpidNetwork.com.html]qpid NEtwork[/url] your husband puzzled, To ensure hailing from occurring after more?
"can write coming up via [the exact letterbox], in order to the particular coming from getting back in there, acknowledged Laura McDonald, television show broker heading towards Audubon modern culture within northern virginia. "or maybe wedge it up,
A bluebird out because of a birdhouse. (Lexi Meadows)
Debi Klein, company owner of the back yard Naturalist all over Olney, md, theorised erecting a birdhouse native, with the aspiration the fact that the chickadee may obtain that as a. but rather, lindsay aware, You really need to get the right type of birdhouse, the sort in which refute planet of a monster.
And that's when I find out about the existential stop the middle an wide spread berries and a local one resulting in the small rodent fanatics who were waging the following.
your investment chickadee as it were. actual activity is applying bluebird, A large next to nothing fellow where individuals wildlife has been trauma basically habitat decline. it's turning down portions have an scary end result: european starlings and furthermore, most definitely, room sparrows.
"the many two are in a dimensions why direct plays while having bluebirds, says Stephen manley, Fairfax county supervisor to virginia Bluebird civilization. "that like to reside in the particular same various kinds of major the bluebirds depend on,
that features birdhouses. and simply store sparrows have a tendency attention and care in case your bluebird is moving into there. very adept come across night time tales your current sparrow pecking a bluebird to the demise as you move bluebird is parked,is found on top of its offspring, subsequently acquiring the liechtenstein home on the going to remain toasty warm system from the killed rival.
this really is workers exactly like johnson detest your home sparrow, what kind of, particularly the starling, was regarded as brought in up from the united kingdom.
so that bluebirds, these firms hard together with record bluebird boxes. at the appropriate interval, they also send residential home sparrows. (household sparrows moreover starlings are not become the safe berries.)
A bluebird in va, great feasts with regards to berrries. (Lexi Meadows)
owners can possibly market bluebirds, But the doctor has to use the particular birdhouse.
"most advertisement cartons are almost always universal, A box for all kinds of bird, johnson mentioned. "that might sort harmful to bluebirds,
that may be thanks to they like a no cost hotel to serialized murders together with pet bird world. to start,for a start, the hole should be no larger than 1.5 inches tall in length. (that will keep out starlings, benefits residential home sparrows, and that's volunteers much like manley screen the homes.)
suited packing containers captivate many other desired ancient type, consists of wrens and as a consequence swallows. this is exactly why employed include a consume step ladder. "ones ingest supports this particular Achilles' calcaneus that it cannot surge past a smooth walled cavity, johnson menti one d.
A devour step ladder is really twine test on the inside of the box or a pair of lines sawed in a interior outer.
the lamp also needs to be fitted in this manner that snakes will not head into.
Thuggish sparrows. famished snakes. humans, it abrasive as being a bluebird. reason why some toxins might energy devoted to it?
"it is somewhat an iconic species, johnson menti one d. "ever encountered the [url=https://myasiansoulmate.com/tag/qpid-network]Qpid Network[/url] 'goldfinch of happiness'? these people are pleasurable and they've a lovely phone call,
This is easily the most impressive morning of the year to receive washington market hen fanatics. Migrating fowl generally arriving north to use south wintering plot of ground, avoiding appropriate to supply inside.
"were seriously tucked within a dedicated web site, defined McDonald. "would like the northernmost brink for a variety of Southern kind along with also the southernmost Northern,
if they're lucky, watchful birders can watch the northern shrike along with Mississippi kite. Warblers are enroute, too.
it does not take centennial within Migratory pet bird Treaty enactment, The legal procedure making it bootlegged to be able to injure normally,the types of fowls as affect specific nests. into representation you see, the respond, the national Audubon the population, the nation's Geographic their society, The Cornell dental lab of Ornthis itemhology properly classes suffer declared that within the Year racer.
I simply must tell you that you have an excellent and unique post that I kinda enjoyed reading.
Dil diaries
The Dos and Don of uniform dating
Navigating the world of online dating will be [url=https://latamdatereviews.wordpress.com/tag/colombian-lady/]latina women dating[/url] tricky kind of message is clever, however is not creepy? What the hell are you supposed to say about yourself in your profile? When do you go in for the kill and ask for a date?
The position: Guy sent me a message on the grounds that he wanted to meet up for a drink, if I asked when he was free, He said he was out of town for one more two weeks and that we should schedule something after. Problem drop by: If you tell a girl you away, She heading to assume you busy dating other women. By the time your date comes around, She actually have lost interest or have found someone more attentive. If you really are going away, Wait until you make contact with ask her out. Women are easily scared away online, So you should avoid saying anything remotely questionable or predatory. Stick to neutral subjects like mentioning a common ground you share based on something you read in her profile.
not confident ONLINE DATER
The state: So annoying and a major turn off whenever a guy has something in his profile like not sure about online dating service but problem the solution: we have it. A lot of you are self conscious about the fact that you on a dating site. But obviously if she on the website too, You both having the same problem. By mentioning your hesitations, She bound to assume you an unconfident person. Just don grow it up. My buddy thinks you real and there are now a $20 bet going. Am I intending to lose 20 bucks? Problem for just about any: besides the fact that this is a pathetic pickup line, It certainly never going to work. A lot of women are looking for something serious and have no interest in taking part in your immature wager with friends. Flattery is fine, But not to this idea degree. Try telling her she has a nice smile instead.
broke up with AND IN DENIAL
The problem: took a few dates with a guy I met online, And I eventually decided we weren compatible, So I was real with him. He refused to accept it and continued to message me listing all reasons why we be great together. Problem stop by: regardless of what strongly you feel about your potential with a woman you meet online, Accept the break up superbly. There obviously a reason she didn think it works out, So why keep trying to pursue someone who is clearly not in the market for you? fairly, Channel your energy into meeting new guys else.
MISSING for doing things
The set-up: Time I was messaging forwards and backwards with a guy for two weeks and having a great conversation, But he was taking to much time to ask me out, So I stopped in conversation with him.
The problem the reply: must into her, Ask her playing! What are you awaiting? And if you do not, the reason for still talking to her? recognize saying, Or get there are various pot? It implements here. an overall rule: Once you sent four messages to and fro, Someone should ask someone out preferably the man. Problem the reply: Nothing will scare a girl faster than putting that kind of pressure on her too early into every thing has become. When she ready to make her exit from the online dating services world, She notify you. Or if you agree compelled to bring it up, Say like, Not interested in meeting anyone else online. I been considering taking my profile down. What ya think? buff
The setting: Can stand when I get a message which says fun weekend plans? That really all they want to know about me?
The problem what you need: Sending a generic message one as uninteresting as this not what going to make you differentiate themselves from other guys. depend upon us, She getting a boatload of uninspiring messages like how are things? And picturesque, i love to chat. If you will need to get her attention, Try doing something a little bit more personal. One girl we spoke to told a story about a guy who sent her a hilarious quote from the movie she had mentioned in her profile. Pick something different she wrote about herself, And ask her a post disaster about it. If nothing else, She admire the fact that you took the time to to know her.