DmitryOlshansky · January 15, 2018 16:33
diff --git a/Topics b/Topics
 Library topics

 1. Multi-threading the scheduler.

    - have a thread pool with thread per core (+affinity to avoid overheads)

    - balance each spawned fiber to e.g. least-loaded thread of the pool

    - provide a hint to spawn on the same thread as parent (?)

    - migrating fibers across threads is out of question
    
    - API is an open question.

 2. Handle files
    Files are not like sockets and are not supported by native event-loop of Linux (epoll).
    Options:
        - use async I/O (AIO) for files 
        - insulate using a dedicated I/O thread pool
        - at least cleanly bypass them completely

 3. Handle process await functions wait/wait4
    Will enable use to not block fibers that wait for a process - std.process as litmus test.
    This can be done via signalfd.

 4. Handle select/poll family of functions
    Some (even blocking) libraries will use thouse for checking 
    if a socket can be read for some timeout of time and then check if they should terminate (or some such).

    Will enable libcurl and std.net.curl.

 5. Check what syscall Thread.sleep and friends do, intercept in our scheduler.
    Some libraries use empty select with timeout for this as well.

 6. Get rid of GC in init/finalize part
    Some C code calls syscalls during at exit, notably close.
    Need to gracefully handle that, GC memory is all gone at this point.

 7. Handle futex calls (fast user-space mutex) 

 8. Messaging between fibers.
    See if std.concurrency works with our scheduler, how to integrate it to multiplexor (see below).

 9. Multiplexing
    There should be a way to do one of many I/O operations, whichever is ready first (read or write/etc.).
    Timers and messages from other Fibers as a bonus.
    

 Integration topics

 1. Remove obstacles on D side - trim down initialization and move things to be lazily inited instead,
    potentially make it part of Dmain.

 2. `-betterC` compatibility
    Will need to migrate fibers to be stand-alone thing.

 3. Foreign API
    Plain C interface in addition to D. Example of runnning a C program on top of D fibers w/o D runtime at all.

 4. Combining with dynamic language Interpretter
    See if we can show-case the same aproach for a popular dynamic languge (Python/Lua?)
    with interpreter per fiber.


 Demo topics (this is just an example, pick whatever tech is more sexy)

 1. Showcase memcached C client working with our fiber scheduler

 2. ditto Redis

 3. ditto MongoDB

 4. Simple TCP reverse-proxy / balancer, compare with the likes of HAProxy.

 ....

 In summary 2 ways:
 - get any off the shelf C library work (blocking interface, which most are).
 - make simple useful network app / benchmark against state of the art
	Library topics

	1. Multi-threading the scheduler.

	- have a thread pool with thread per core (+affinity to avoid overheads)

	- balance each spawned fiber to e.g. least-loaded thread of the pool

	- provide a hint to spawn on the same thread as parent (?)

	- migrating fibers across threads is out of question

	- API is an open question.

	2. Handle files
	Files are not like sockets and are not supported by native event-loop of Linux (epoll).
	Options:
	- use async I/O (AIO) for files
	- insulate using a dedicated I/O thread pool
	- at least cleanly bypass them completely

	3. Handle process await functions wait/wait4
	Will enable use to not block fibers that wait for a process - std.process as litmus test.
	This can be done via signalfd.

	4. Handle select/poll family of functions
	Some (even blocking) libraries will use thouse for checking
	if a socket can be read for some timeout of time and then check if they should terminate (or some such).

	Will enable libcurl and std.net.curl.

	5. Check what syscall Thread.sleep and friends do, intercept in our scheduler.
	Some libraries use empty select with timeout for this as well.

	6. Get rid of GC in init/finalize part
	Some C code calls syscalls during at exit, notably close.
	Need to gracefully handle that, GC memory is all gone at this point.

	7. Handle futex calls (fast user-space mutex)

	8. Messaging between fibers.
	See if std.concurrency works with our scheduler, how to integrate it to multiplexor (see below).

	9. Multiplexing
	There should be a way to do one of many I/O operations, whichever is ready first (read or write/etc.).
	Timers and messages from other Fibers as a bonus.


	Integration topics

	1. Remove obstacles on D side - trim down initialization and move things to be lazily inited instead,
	potentially make it part of Dmain.

	2. `-betterC` compatibility
	Will need to migrate fibers to be stand-alone thing.

	3. Foreign API
	Plain C interface in addition to D. Example of runnning a C program on top of D fibers w/o D runtime at all.

	4. Combining with dynamic language Interpretter
	See if we can show-case the same aproach for a popular dynamic languge (Python/Lua?)
	with interpreter per fiber.


	Demo topics (this is just an example, pick whatever tech is more sexy)

	1. Showcase memcached C client working with our fiber scheduler

	2. ditto Redis

	3. ditto MongoDB

	4. Simple TCP reverse-proxy / balancer, compare with the likes of HAProxy.

	....

	In summary 2 ways:
	- get any off the shelf C library work (blocking interface, which most are).
	- make simple useful network app / benchmark against state of the art
No results found