Home   Archive   Permalink

Multithreading and Task in Rebol - Just say No

In discussing potential bounty ideas, @GregP mentioned:
> - support of "do task" (multi threading)
I will very strongly say DON'T try to attack multithreading with Rebol. That is a tar pit.
If your goal is to speed up your code on algorithms using multiple cores...then you are misguided in using an interpreter to be solving a problem where speed is essential.
If your goal is concurrency, then start multiple processes and message between them. If those processes are on the same machine, the code pages for the Rebol executable will only be loaded once (on all modern OSes). Multiple processes that are using messaging also lets you have processes talking across the network.
When people suggest the internals of Rebol3 have been prepped for multitasking, here is what they mean. The state of the interpreter has been wrapped up within the code with (relatively) few global variables. It would be possible for you to get a console variant which let you say "spawn new state"...and within the same process...some list of active sessions could add a new interpreter. You could switch back and forth between them.
But I find countless instances where if two such interpreters were running simultaneously on threads within the same process, they would stomp on each other and crash. And none of the Rebol data structures have any locking logistics. It's even rather common to take things that are conceptually read-only, modify them temporarily, and then put them back how they were ("for efficiency").
Could Rebol have multiple threads within the same process running concurrently? Yes. Would it crash and burn and take a long time to fix? Yes. Would it require a heck of a lot of work in grafting on a multithreaded worldview and locking primitives after-the-fact, likely producing a hackneyed variant of mutexes and semaphores, which most of the world has moved past in client code? Yes.
This isn't to say that the work done in whittling out global state was in vain. It's good style to reduce global variables; the interpreter code is easier to follow and modify that way. But Rebol is best served by having people go back to the messaging roots and exchange the "REN" data. If you focus on good solutions for that, you can message over to Red and back, not to mention other languages.
But it's time to kill TASK!, reclaim the type, and walk away from the idea that Rebol3 will be multi-threaded. Note that every web browser tab you load in Chrome is a process. It's time to think inter-process and make that whole experience smooth...not dig a hole based around threads. If Rebol is to use threads it should do so internally to the interpreter, not as a user-facing feature.
Red's strategies for concurrency are different and borrow from more modern languages. And it's worth it as it's making compiled code, so if performance is your goal in the first place then you might just achieve it.

posted by:   Hostile Fork       29-Sep-2014/7:56:17-7:00

I agree.

posted by:   Nick       29-Sep-2014/13:22:58-7:00

Well said.

posted by:   Patrick Stewart       29-Sep-2014/14:27:34-7:00

I have used multiple Rebol processes communicating via TCP to efficiently use multi-processor systems with great success. I have no desire for multi-threaded Rebol functionality.

posted by:   Bo       29-Sep-2014/14:52:28-7:00


posted by:   Gregg Irwin       29-Sep-2014/15:01:43-7:00

Well put Brian. Cost of multithread is too high and there are other priorities listed above. And who is going to need multithreading (YAGNI?) that bad?
And Patrick, I am not calling your bluff ;-) no need to do that. By now you know much better how to spend some of your money on Rebol and or Red, without throwing it away.

posted by:   Arnold       29-Sep-2014/15:14:03-7:00

Indeed, I just keep reading the feedback and it is intense, This community was a damn sleeping giant with much to say. I am trying to go over all the information put out there and making some notes for myself to guide my thoughts.

posted by:   Patrick Stewart       29-Sep-2014/17:08:06-7:00

I can't speak for anyone else, but I can say that I am deeply interested and invested in Rebol and Red. However, like you, Patrick, I also have a business and it takes all my attention to keep it afloat.
Other than an occasional vacation, I usually work 7 days a week which leaves little time to devote to the internals of Rebol and Red. Also, I am not a C-level developer which makes it hard to contribute to Rebol's development. However, if I were to contribute to the codebase, it would be nice to know which repository to use for best results.

posted by:   Bo       29-Sep-2014/17:21:58-7:00

Multithreading and Task in Rebol - Just say No
I fully agree it is really easy and easier to rely on a distributed implementation with REBOL.
We had Rugby (from Maarten) as a request broker and even the excellent Uniserve + Task-Master (from Doc).
But if we want some additional processes to work on the window screen, the only way is by using threads.
Now I fully understand this might be costly and that it doesn't fill the community expectations.

posted by:   GregP       30-Sep-2014/3:31:16-7:00

It is not that we do not want it at all @GregP. I would really like Rebol to have them, but I rather have Rebol3 back reanimated and usable like R2 was. The multithread will have to wait a little.

posted by:   Arnold       30-Sep-2014/10:32:17-7:00

"Red's strategies for concurrency are different and borrow from more modern languages. And it's worth it as it's making compiled code, so if performance is your goal in the first place then you might just achieve it. "
+1 for Red, it really shows that Red will be a better language. Generally speaking everybody needs better performance.
for example, if performance was not important, we would still be using 286 processor speed today, every iteration of processor design comes with faster and more powerful processor.
or we would still be travelling by train, instead of flying.
or we would still be happy waiting for 1 hour to see a program move from one screen to another screen.
I can't wait for RED to reach version 1.0

posted by:   hey teacher leave the kids alone!       3-Oct-2014/22:43:12-7:00