Stakeholders Answer: C89, C99...?
So...I have a forthcoming fairly 'large' announcement (as announcements go). But, let's not worry about that bit yet...
...instead, front-running that announcement--just as a check on the pulse of who is still reading these channels--I will ask a question about Rebol's implementation language, and who cares, and why.
Historically: prior to source release, people believed that Rebol was implemented in ANSI-C, a.k.a. C89:
http://en.wikipedia.org/wiki/ANSI_C
Not at all true. It was quickly determined after source release that Rebol was *not* ANSI C. This was evident to me at first glance even by appearance of C++ style comments ('// comment text'). Puzzled I asked Carl about it:
http://www.rebol.com/cgi-bin/blog.r?view=0515#comments
His answer to my query was:
'the compiler will be the subject of another blog. In the past it's been determined by practicality: what actually works over the widest range. (I can usually port REBOL3/core to any target machine in 5 minutes.)
This hasn't been easy, and it's why there are various C restrictions. I agree... many of the C++isms are nice, but can dramatically reduce portability to older boxes. It's a balancing act.'
If you want to know how *not* ANSI-C Rebol is, use the compiler flags '--std=c89 --pedantic'. Get back to me if-and-when you get the patches to fix that are accepted.
So from the horse's mouth: the coding standards were guided pragmatically. In the pre-standardization era, that's all you had. (And even once the standards were written that said nothing about which compilers bothered to *implement* the standard...)
...but as I often ask: might one move from pragmatic to 'formal', and benefit instead of lose by doing so?
Let us imagine for a moment that there was a community-agreed-upon and cohered Rebol source base, that was agreed to be C (and not C++). In the present world, what should guide the choice of 'which C'? If Rebol never actually was a C89 standard codebase, would it be worth it to bump up the timeline and be able to *honestly* say it is a valid C99 codebase? (There is a --pedantic check to confirm standards compliance, wouldn't it be nice to pass that check? Rebol today can't pass that for C89 by a long shot.)
The thing I'd observe is this: C99 is a standard of the C language released a year after C++'s first standard (C++98). It was designed by the committees largely as a *subset*. Even old computer systems now tend to have a C++98 to C translator. It seems that Rebol being able to rigorously claim to be a valid C99 codebase is a better stake in the ground than the somewhat hand-wavy 'it's a mix... of... some things that work... and some don't?'
And I must point out the great work being done lately by @MarkI and @kealist on getting TCC--the tiny C++ compiler--to build Rebol. TCC supports much of C99, and weighs in at only about 150K. There seem to be huge opportunities in linking TCC directly into Rebol itselfsuch that you can build the whole system with a rebol.exe... a kind of 'Full Stack' Rebol. Optimistically I'd guess it can be done in a way that could cut out as much as 50K off that dependency...so the Rebol that could build itself would be just 100K larger. (Theory so far--not practice--but I am usually not so far off in my guessing. However my plan would make Rebol-TCC's command line processing and guts completely in service of Rebol, effectively a fork.)
My question for the stakeholders is: If Rebol became C99, who is this hurting exactly? And why? How are you hurt? C99 is a much improved version of the C standard, and old in its own right (there's a C11 and more...). While people might be skeptical of me as a C++11 thru C++14 advocate... there is no comparison in the levels of complexity between C++ and C. C99 is just cleaner C89, and even TCC can do much of it.
How far in the past should stakeholders be concerned about, and why? Do people have opinions? If it's about an old computer, are we willing to dig up the old computer and see if the solution works?
I am leaning toward saying that Rebol should move from a 'no standard...really :-/' codebase to a C99 subset supported by TCC, and that TCC become the new 'lowest common denominator compiler'. Because it's open source and can be built into Rebol itself. This would generate a 'Full Stack Rebol' that could not only build itself, but generate extensions and in general start to rival Red.
(Note: in 'rivaling' Red I'm not saying the goal is to distract or deflect attention, but rather to raise the bar; I think I've been clear in saying that what's good for Red is good for Rebol and vice-versa. This is not zero-sum.)
posted by: Fork 13-Apr-2015/4:10:29-7:00
I like clean and standardized as long as it doesn't add much weight to the final executable.
posted by: Bo 13-Apr-2015/23:52:37-7:00
I'd like a clean, clear C implementation as well.
So ... you saying, or at least implying, that C irregularities are at the heart of at least some of R3's development irregularities? That's quite a skeleton in the closet. Yeah, that would explain a bit.
I've been itching to ask a different but, in light of what I think you are saying, possibly related question: how hard would it be to add Unix shell out capability to R3 (to support full interaction with other Unix utilities)?
Better yet a full Unix shell with Rebol at its core? Now THAT would be fun!
posted by: stever 14-Apr-2015/0:43:12-7:00
My gut reaction would be, that I want Rebol to work on as many systems as possible.
At the end of the day, though, I personally probably only need Linux, Windows and Mac.
So a clean codebase which makes it easier to contribute and has less bugs is much more valuable than for it to be compilable on AIX. At least to me. And I guess on modern systems this would even work better.
Regarding TCC it sounds like fun, but having rebol completely compilable by rebol itself does not sound too useful to me.
There is another edge though. Rebol as an all in one toolbox is great, but I think there is a scaling problem: too many new technologies, which are mostly available as C-libraries.
So, if TCC would help to have first class C interfacing, maybe even embeddable C-libraries, which can be compiled on the target system - now _that_ would be great.
posted by: ingo 14-Apr-2015/1:16:10-7:00
While I do want to migrate the code base to C99, I also want it to be buildable with microsoft visual studio, as it's much easier to debug on Windows systems with MS VS.
posted by: Shixin 14-Apr-2015/11:17:58-7:00
What you describe sounds good to me. In my view the way rebol 3 is now opensourced is coming to a dead end, now the community has their hands tied to their backs, in practise the opensourcing is read-only. The way to go is to make the codebase into a standard so that TCC can be used to create a healthy competitive alternative to where Red is heading, where both party's profit.
posted by: Arnold 15-Apr-2015/3:47:59-7:00
The biggest problem with Rebol is that the developer community is nearly non-existent (that is, C developers who create Rebol tools, as opposed to users of R2/R3). Anything that makes Rebol more accessible and interesting to C developers is worth while.
posted by: Nick 15-Apr-2015/4:51:54-7:00
After much gnashing of teeth I have modified Rebol to build under both Clang and GCC under high warning level (-Wall, -Wextra) without complaint. It can do so as both standards-compliant C99 and C++89 with the "-pedantic" warning level up. In the C++89 standard there is no "long long" officially, so -Wno-long-long has to be used...and it is frequently the case that Rebol has unused parameters to functions, so I'm going with -Wno-unused-parameter for now as well. (The parameter's name could be omitted and that would stop the warning.)
Many devils in the details of getting compliance at this level. There are a lot of assumptions that programmers tend to take for granted. To pick a random one: let's take the assumption that the size of a function pointer and a void pointer are the same. This is not necessarily so: C and C++ are compiled and raw functions cannot be dynamically created after the link point. So it could be the case that if you had fewer than 255 functions in your program that they would be identified by a byte and dispatched through a table.
Whether such things are important to be concerned about or not--or being able to build at high warning levels--is up for debate. But for every warning that turns out to be one that you can rewrite the code to avoid, there's going to be one that indicates *an actual bug*. I'd also say that if the embedded market is staying on the radar that being as "modern as possible, while still being old-school" is of benefit.
Replies to the above:
@Bo - There should be no added weight for standards compliance. If anything the code can be optimized more aggressively without failure.
@stever - I will announce at some point some analysis and modifications related to "irregularities" in terms of bugs that have historically been hard to trace, and how I've reworked the memory model to address it. Regarding the integration of a shell with Rebol, you might be interested in the relevant part of the Ren Garden video: https://youtu.be/0exDvv5WEv4?t=539
@ingo - I am all for building on AIX, Amiga, or any old (or new/basic/embedded platform). But as @earl likes to say "someone needs to be active in keeping that build working". I'm all for doing something just for the art or retro puzzling fun of it. But it is actively harmful to handcuff the codebase to compiler features for the sake of an abstract compilation target that *doesn't even work*. The idea of Rebol building itself is a side-benefit of the greater idea of bringing it closer to full stack by being able to compile its own extensions and bindings dynamically as Red would.
@Shixin - C99 isn't "fully" implemented anywhere so whatever we wind up using will be some kind of subset, and I imagine Visual Studio should be able to build whatever that subset is. Somewhere between building with TCC, C++89 GCC/Clang, and C99 GCC/Clang it should work.
@Arnold/@Nick - Rallying people would have been easier with an official delegation. That day has yet to come. However, I think the truth of the matter is few people really feel like C programming in the first place or know where to begin. (Especially on an older codebase which seemed to have spiralled out of control during expansion...and did not have developer documentation and was left on the factory floor.) It has been easier to ruminate on user-space design questions and wait around for Red. I was motivated by Ren Garden and the Lest conversion of my website to become more aggressive about the reliability and control of Rebol, and it will require strong support to succeed as a group effort.
posted by: Fork 25-Apr-2015/0:28:03-7:00
Fork,
I'm still willing to pay for a browser build, with a GUI implementation. Do you have any interest in working on this? I expect that your Core build would likely make it through Emscripten more easily, and there are a couple guys capable of porting the renderer to WebGL. I'd love to be able to run all our GUI apps in the browser - and I believe that would attract more developers to Rebol.
posted by: Nick 25-Apr-2015/11:01:45-7:00
@Nick emscripten capability has existed for some time but is just not publicly deployed. If you want to spur the release I will suggest you speak with Andreas.
There are two key things I am personally interested in with emscripten:
1. Empowering a "Try Rebol" that holds state in the browser and gives an interactive tutorial. Currently Kaj's Try Rebol discards the state on each evaluation and uses his server to run the request (manage infinite loops, timeout, security, etc.) On the plus side of Kaj's tool, you don't have to download a huge pile of JavaScript. On the downside, it isn't a REPL which pushes the responsibility into the computational power of the client's browser, where it can hold state between evaluations. Contrast with http://repl.it
2. Being the engine behind a sort of parse front end that anyone can use, presenting PARSE only and giving hopefully some good highlighting and feedback. It could be a swiss-army-knife page anyone could visit and use right away. Think of it as Rebol's answer to http://regexpal.com/
However, I'm not personally that interested in R3-GUI development. I will say that recent learning has put me in more of a position to critique/improve it, and my announcement will say more about that.
What I'll say at this moment is I'm not sure that WebGL is the right technical target for compositing in the browser. It seems to be overly focused on 3D features, has no text rendering, and some of the demos I tried to check on the status just crashed my Chromebook. Thus continuing the push to make the compositor not matter--and using something more native in the browser (canvas? ordinary divs/spans and CSS?) might be the preferred solution.
Also: I've been seeing the building-in of GOB! as a case of baking in an arbitrary extension to the core, when it represents one of many extension needs...and shouldn't be in the console executable at all. So what is probably needed is more good bridges... to C/C++/JavaScript for extensions, and to break out graphics a bit more cleanly. RenCpp paused on this question of plugging in new types vs. just functions, and I'm trying to think of a bigger picture answer.
I cannot say offhand how compelled today's web developers would be by Rebol GUI dialect experience as people might have been in the past. Right now is a really strong focus on "metal" and control. Standing out from the crowd has a huge importance, and everyone wants to distinguish themselves; so they need to be able to set the font and shadows just-so. People are willing to pay for this with arbitrary amounts of investment...the "80s cash register" aesthetic just because it's "easier" won't win if it doesn't feel native/original.
Understanding the psychology of today's generation of programmers (who've grown up with the web and StackOverflow/Reddit/Hackernews and JavaScript/NodeJS/Python/Ruby) requires spending some time with them and listening to what's important in their world. It's different. With the SO talk, I'm reminded of the "kids react to" videos...
https://www.youtube.com/watch?v=bDOZbvE01Fk
Being able to win minds in the anti-complexity movement is a hard one. Right now I think PARSE is a better bet than R3-GUI and VID, and making dialecting work as advertised is important as well. Delivering on claimed features like the ability to author one's own loop constructs must happen... if you write a loop wrapper and RETURN returns from the wrapper vs. acting like you'd expect, that's a problem. These line items must be fixed, no excuses.
posted by: Fork 26-Apr-2015/14:51:10-7:00
Fork, your perspective is well thought out and well supported by your own experience and knowledge, as always. My perspective is much simpler, but I think still valid. I still see Rebol as the best tool for people who aren't full time developers. It has been for me, and I think there's a large crowd of potential users in the same boat as me, who would really benefit from a having it available, especially with GUI, in the browser (whatever our community seems to think, the browser is a ubiquitous modern computing platform which is not going away soon). Unfortunately, everyone's time must be spent on their own priorities, and there just aren't enough people here to implement everything. I'll keep working to get the GUI port done (whether it's done with Canvas or WebGL I have no preference - there are just a few guys already looking at WebGL).
posted by: Nick 29-Apr-2015/12:00:38-7:00
So ? We reached another stand-off? Or the direction is set now?
We put Fork in charge of a new effort?
We get Carl to hand over the key to the rebol/rebol github treasure chest? We get a semi-formal commit committee?
posted by: Arnold 1-May-2015/15:31:08-7:00
Any word from Carl about handing over the key to rebol/rebol on Github?
posted by: Arnold 5-May-2015/4:48:17-7:00
@iArnold - Carl has acknowledged that the lack of bugfixes or changes of rebol/rebol is a problem. He has been on the brink of giving his endorsement to the idea of delegation, but never reached the point of explicitly saying to do it. Exactly why that has been the case is not understood completely.
When I'm ready to announce + publish what I'm doing, I will. There will be questions to answer about it at that time, and who will join on board with it (besides Dr. Rebmu).
posted by: Fork 6-May-2015/11:06:15-7:00
I'm pretty likely to "join on board with it."
Really looking forward to seeing Rebol with some fixes to the appliances, a yard clean-up and a new paint job!
Gentrification and urban renewal has to start somewhere ... which requires a "someone"!
posted by: stever 7-May-2015/10:40:59-7:00
@stever Thanks for the likely vote of confidence--and yes it does take a "someone". I hope (hope!) to have a good formal announcement soon.
In the absence of that announcement--I'm bad at keeping secrets (or really don't want to, once I actually have decided on something). So I will say that for those who followed RenCpp, the angle I have been looking at isn't to make a Rebol.exe or Saphir.exe or Atronobol.exe competitor. Rather, I'm constructing a "RenC" library of Rebol interpreter services, which is powerful/useful/debugged and can be embedded in your executable of choice.
Rebol did have the idea of an "API" before. But that was based on a pre-open source era, where the goal was to leak as little of the implementation as possible. Yet the outcome of this was an anemic interface, and you do not need to look very far to see how anemic it was:
https://github.com/rebol/rebol/blob/25033f897b2bd466068d7663563cd3ff64740b94/src/core/a-lib.c#L334
It was basically "string of code in, string of value out". So if you're wondering why the console experience in Rebol is so mediocre, it's because that's kind of all it could do.
The mimic type hierarchies are unsustainable and fragile, trying to force every type through an abstract void* version of itself for the sake of not revealing the internals:
https://github.com/rebol/rebol/blob/25033f897b2bd466068d7663563cd3ff64740b94/src/core/f-extension.c#L75
RenCpp and Ren Garden could never have been built on this API. So they weren't. They co-opted the interpreter source itself as a library. (Again--it is important to note that this is based on a very different mindset than the closed-source Rebol was willing to enable.)
Under this model, the current console would just be a demo program for RenC...the way Ren Garden is a demo for RenCpp. You could have a version that uses only standard IO, or a version that uses libreadline, or anything. It should be easy to write. Once you *really* expose the whole of the functionality, then you have some hope of writing a console that can act as a debugger or do other things.
I will mention that I do not think RenC will have a compiler "switch" to interface with C via Rebol or Red. The C++ trickery that RenCpp uses to do this is simply not available. If Red wants a C interface, it will not come through this plan (any time soon). But RenCpp is light-years more usable anyway, so there's no reason to complain about that.
Stay tuned...
posted by: Fork 14-May-2015/1:02:24-7:00
@Fork
I think I need a hands-on experience in order to relate well enough to what you are trying to describe to answer your questions. And I suspect that is what most of the community needs. Saying that we can experience RenC if we are willing to download and install an OS in order to download/compile/install the demo (no matter if I have the count of steps exactly right) is mostly guaranteed to keep the number of informed votes to an absolute minimum. With actual informed (experienced) votes at a minimum, hallucinations and disinterest are likely to rule the conversation. Probably not what any of us want.
I suggest you compile a version for (ugh) Windows and make an easy download and install. I say Windows not because that is my preference but because that is the easiest common denominator. At least all of us can probably reasonably borrow a Windows system connected to the internet for long enough to try the RenCpp/Ren Garden thing you keep talking about.
And, no, watching a video really isn't much of a substitute, even if it is better than nothing.
I think you will find the community response much more significant and, certainly, much more reliable and constructive once everyone has a chance to try and test what you've got running so far.
posted by: stever 28-May-2015/1:51:44-7:00
|