Comments on: Basic Contexts of the R3 System

Carl Sassenrath, CTO
REBOL Technologies
23-Sep-2010 18:49 GMT

Article #0334
Main page || Index || Prior Article [0333] || Next Article [0335] || 12 Comments || Send feedback

Updated: 25-Sep-2010 - Name change: "lib" replaces "exports"

With all the recent discussion about system and user contexts, module imports and exports, and the various fields of the system object, you might think that R3 has gotten complicated. It hasn't.

To understand what we mean by contexts just think of objects. An object encapsulates names (words) and their values. The contexts used in the system are essentially the same. They hold the names and values of the system as well as those of your program and any modules you've loaded. (Keep in mind that REBOL does not require object classes, so you can have a unique single object. That's what we use here.)

Here's a diagram of R3's main contexts:

The boxes are:

  • Root context - a minimal hidden context that is used by the memory manager for garbage collection.
  • System object - holds the primary references to all other values related to the system, including run-time options, catalogs, lists, objects, functions, and much more.
  • Lib context - holds all publicly exported functions (and data values too). Includes both the native and mezzanine functions of the system, as well as the exported functions and data of any modules that your program has imported. (Previously called "exports".)
  • Sys context - where private system functions are kept (as of release A108.) Some of these are used just for boot; others are special intrinsic mezzanine functions used by the system. For example, when you make a module, the module datatype creation code is a sys function, because it's called internally by the module! datatype.
  • User context - home for your program, all of its functions and data, plus any imported functions and data. (You can think of these as the "globals" of your program, although as you can see, they're not really global. They are local to your program.) If your program uses any of the values from the exports context, you will find them here as well.

The difference between system and sys should be made clear. The system object holds special values, such as settings or options, that are used by your program. The sys context is a hidden run-time library used by the R3 system and is not directly accessed by most programs.

I should also note that when you import modules into your program, each module will have it's own unique context for storing its own public and private functions and data.

To see the words of your user context, you can type:

probe words-of system/contexts/user

Or, to see the words of the lib context:

probe words-of lib

So, the system design is smart and simple. Of course, you don't need to know any of this to write programs, but if you want to dive deeper, these are the basics.



24-Sep-2010 7:20:38
Where is system/contexts/system? It is superfluous? Or is it what you refer to as Sys context?
Brian Hawley
24-Sep-2010 12:53:21
The old system/contexts/system object will no longer be externally referenced like that (it will be the "Sys context" mentioned above). The parts of it that need to be exported will be put in system/contexts/exports, as if they are module exports. For various reasons the Sys context will likely not be an actual module, just an object.
24-Sep-2010 13:24:52
I seem to understand that system/intrinsic will also bemoved to the Sys context. Would that mean we can no longer do source system/intrinsic/make-module and the like? This would diminish the self-documentation properties of R3 somewhat.
Carl Sassenrath
24-Sep-2010 18:50:04
Brian: If for some reason we find it worthwhile, we might be able to wrap sys as a module after the boot. We'll want to determine if that adds any benefit. Maybe not. Difficult to say right now.

Meijeru: It's all still there! Just in a different location, hence, a different access path. Advanced users will still need access to the sys context, but it's an explicit sys/whatever reference, not part of default binding.

So, a progress report:

  • I'm reworking the entire boot and its context. I've stripped all non-essential mezz (even all the to-* helpers, so we can drop most of them during this rework.)
  • The main purpose of the boot code is to build a viable runtime context. It's almost there.
  • On closer inspection, guess what? With the layers stripped away, it's easy to recognize that the exports context is really a form of runtime lib. Since most programmers recognize this concept, we will want to go along with that idea; therefore, 'exports is going to be called to just 'lib.

That gives us sys, lib, and user as the primary contexts.

So, it's shaping up. The goal over the next couple days is to get it running well enough to make the next boot phase work. I'll be adding a micro-loader, and that will give Brian (and any other R3 gurus) the ability to build and test the remaining lower-level boot code without requiring it within the exe. That helps the development cycle move faster.

After that, the main challenge will become how we weigh what's in and what's not. I think that will be the most difficult part of this process.

Brian Hawley
25-Sep-2010 14:50:22
Every imported module can be upgraded, so we may not want to make a module out of sys, to prevent it being overriden. But since overrides have to be done explicitly anyways, it might be safe.

I will make an export-words function (haven't settled on a name yet, might be import/words) in the module-tools mixin module that will be able to add words to the lib context without having to put them in a module. This will allow you to have a premade mixin that contains a few global exports, or to add words to the global lib context that are bound to a regular object context.

Here is some context for that last paragraph: On top of the boot code is the module system, and that is also currently being rewritten (likely done by this weekend). In addition to regular modules that export to the lib context for everyone to use, R3's module system will support "mixin" modules that have to be explicitly imported in order for your module or script to see their exported words. The existing R3 module system supports mixins; the new one supports them better. Most mixins will be used for expanded APIs and other advanced stuff. If you don't need that stuff, you will be able to ignore the concept altogether.

The new module system is built around low-level accessor functions that will be collected in a premade mixin called module-tools. The globally exported load, save, import and intern functions, and do and make-module intrinsics will be simpler wrappers around those accessors. Unless you are doing advanced script and module manipulation, you won't need to import module-tools, just use the globals.

Here's another progress report:

  • Currently working on import, the last function left to write. The accessors it uses have been written already, and are just being tweaked accordingly.
  • Guru-level design docs here, being updated as we go.
  • Must rename references to the export context to lib.
  • Must write down the already-designed bootstrap process for building and importing the module-tools mixin.

The new module system is not required for the boot code rewrite or core context reorg, or even for the function split. However, it will make it easier to organize the modules that we end up with, and to make the APIs even cleaner.

For regular R3 users the new module system changes nothing that you need worry about. Regular modules work the same as before.

Brian Hawley
27-Sep-2010 16:38:57
It occurs to me that it might be more appropriate to put the module-tools stuff in the sys context. Once you have that sys shortcut, rigging up a mixin for them may be unnecessary. You choose.
29-Sep-2010 4:02:42
Yes, all this stuff is for Gurus.

But as some of us will probably want to become one, or almost know things better, be sure to well document and explain all this boot, contexts, modules, mixins, exports...

Actually, I'm lost in all that, like a meat ball in a spaghetti plate !

Maxim Olivier-Adlhoch
29-Sep-2010 11:35:13
sorry for cross-posting, but this is too big an issue for me to let it go unnoticed by Carl.

the use of the word 'LIB is plain wrong. saving 4 characters to make it a reserved word is not a good idea.

I did a proper post in the other blog entry related to this:

Brian Hawley
30-Sep-2010 3:54:45
Maxim, your assessment of the use of the word lib is wrong. It is not reserved, you can override it with your local redefinition of lib at any time. And it doesn't refer to a context that is primarily meant to store exports, or which will necessarily have any exports in it at all. The new system model is different.

I am also sorry for crossposting this information, but this misconception could not be left unanswered. Now back to the show :)

Charlie Bonano
26-Oct-2010 19:30:57
The diagram writer confuses all.

If "[T]he system object holds special values, such as settings or options, that are used by your program," then arrow should lead from the user context and point to the system object, that is, the head of the arrow pointing at the system object.

The diagram has it backward, exactly.

Also, why does the diagram show the system object pointing to the lib context? No relationship exists there.

However, the user context should point to the lib context since sometimes, a user program takes (imports) functions and data from there. Also, it is from where a user context gets all publicly exported functions and data values.

Also, should not the sys context point to the root context? No sense can be made to have the root context pointing at the system object.

Please rework this diagram and the subsequent explanation. All too often, when persons are too close to their work, they become afraid to acknowledge they do not know exactly what they're talking about because of peer pressure.

Maxim Olivier-Adlhoch
29-Oct-2010 0:00:20
Charlie, actually IMHO you haven't exactly grasped what is going on.

the system object actually has a member called contexts which stores a reference to the lib sys and user context.

>> words-of system/contexts
== [root sys lib user]

Scripts are run in the user context, but have access to the more protected and internal contexts through the use of the system object (which is bound within the user context as the word 'SYSTEM), as the above code example demonstrates.

Note that the root context displayed in system/context is set to none after the core has initialized, to prevent malicious hacking of the deep internals of the language.

Also note that in REBOL the difference between contexts and objects is purely functional in nature, they actually are the same thing.

>> type? system/contexts
== object!
>> type? system/contexts/lib
== object!

You can understand a context as being the source of binding for other REBOL data.

So why is this a big deal? it means that the functions defined in sys might not be the same ones in lib... more importantly functions bound in the lib context will be using words from that context.

Carl Sassenrath
31-Oct-2010 12:26:54
Charlie, good questions, and Maxim has provided good answers.

Note that is a diagram of the design hierarchy. It is not a reference path diagram used for coding.

I can assure you that the diagram is correct.

In REBOL, as Maxim points out, when we say "context" we mean referential closures, (objects) not execution environments (stacks) -- as you may be used to thinking from other languages. This is the "RE" part of "REBOL".

So indeed, as you point out, your program code is bound to the user context. From there you can access via the system object various other contexts. For example:

probe system/contexts/sys

But, that's only a standard convention! Your user code can set system to some other value of your own making, and that's perfectly valid, and you can create your own objects that refer to the system object or any of its contexts.

In fact, we do some of that in user context: the words sys and lib are handy shortcuts to those other system contexts.

probe sys

Inverting the diagram, as you suggest, would be meaningless, because in REBOL any context can refer to anything, including other contexts.

This is also important... because the user context within sub-tasks each have their own clone of the system object, with various sub-objects replaced to reference the task-relative environment.

Post a Comment:

You can post a comment here. Keep it on-topic.


Blog id:



 Note: HTML tags allowed for: b i u li ol ul font span div a p br pre tt blockquote

This is a technical blog related to the above topic. We reserve the right to remove comments that are off-topic, irrelevant links, advertisements, spams, personal attacks, politics, religion, etc.

Updated 26-Mar-2017 - Edit - Copyright REBOL Technologies -