REBOL 3.0

Comments on: Introducing the Concept of a Current Module

Carl Sassenrath, CTO
REBOL Technologies
24-May-2006 23:53 GMT

Article #0029
Main page || Index || Prior Article [0028] || Next Article [0030] || 18 Comments || Send feedback

Let me just say this outright: my Modules document is not very good. Sorry, but it's just not very clear. It is quite old, and as soon as I get a better doc written, I promise I will delete that old one.

So, to make it clear: Modules provide a way to extend the REBOL environment. That's their purpose.

Of course, that statement is not indended to mean that everything about modules is simple. It is not. But, to understand and to use modules must be simple. Anyone and everyone should be able to do it. (Otherwise, I would not be doing it, right? You know me.)

So, this means a few things:

1. You need to be able to create a module just as simply as loading a script that looks like:

REBOL [title: "My module" type: module]
...

and alternately you can use this method at any time in your code:

my-mod: make module! [
    [title: "My module"]
    a: 10
    func1: func [...] [...]
]

Note: that means you can create multiple modules within a single script, if that's what you need to do.

2. You should be able to access the exported values of a module easily. How easily? Well... automatically.

For instance, if your module is:

REBOL [title: "My Math" type: module export: [buy sell]]
...
buy: func [value price] [...]
sell: func [value price] [...]

then later, you run a script that does this:

buy "google" $200
sell "microsoft" $10

It should work as you would expect. It must be just that easy.

3. There are many extra options that are the other 20%. Yes, they are useful, but most of the time you don't care.

Now, that's where it gets a bit more complicated. Let me give you a good example:

Let's say a section of code runs an external script using the do function. To what module do the words of that script get bound? This is a non-trivial question. I mention it go get you thinking. Deeply, I hope.

Why does it matter? Here's why: if you are using a function of a REBOL module (call it Mod-A) that looks like:

do-it: func [file] [
    ...
    do file
    ...
]

are the variables of the newly evaluated file bound to Mod-A or are they bound to the module that called do-it in the first place?

Over the years, I've found that when these questions come up, there is some kind of implicit action taking place. There is an assumption going on. In this case it is what module do the new words get bound to? We have two choices:

  1. The module where the do was evaluated.

  2. The original module where the do-it was evaluated.

In low-power static languages like C and most others, this would be an easy choice. But, in the dynamic evaluation of REBOL, it's not quite as easy to decide. Both choices are valid.

Like in other areas or REBOL, the better design choice must come from some kind of common sense rule. Let it be this:

  • It is best to avoid the accidental auto-expansion of modules.

Such results would be problematic because arbitrary new functions could be added to any of the existing standard modules of the system. That wouldn't be a good thing.

Therefore, according to this rule, the do above would bind new words to the current active module. So there is the implicit action: the concept of a current module.

When your user script gets evaluated, REBOL will launch it within its own unique "user module". And, when any additional scripts are loaded and evaluated, by default they are bound to the same user module unless they are explicitly directed elsewhere.

Now, what if you want a section of code to refer to an explicit module? What if you do a load or a do and you want a specific module to be affected? That's the purpose of the with function:

Just as you can say:

with object [... expressions ...]

Here object is any valid object, and the block is evaluated within the context of an object, you can also say:

with module [... expressions ...]

Where module is any existing module. This line specifies that the expressions are bound to the context of a given module.

You can even write:

with module [do %script.r]

to force the module to accommodate the new script's bindings.

In addition, to help make this action easier, you can also use any word with a context to provide the context itself. That way you don't have to figure out how to obtain the reference to the context. For example:

with 'word [do %script.r]

where word is bound within the target context.

I think those of you who are expert REBOLers see that with is more than just another name for bind. A hidden variable is being set that indicates the module, the "current environment" for the block. And, there are some deeper implications that go along with it (such as what happens when an error occurs that throws us out of the with block.)

So, there you are. Simplicity with just a pinch of spice to create the very best of flavor. Now, if only I could cook everything that well.

Please post your comments. I am open to your ideas. Thanks.

18 Comments

Comments:

Petr Krenzelok
24-May-2006 22:48
:) Hi,

semi reboller's opinion - before I got myself reading till 'with section, I was thinking of following refinement:

do/with

as we will have 'with itself, not sure such refinement is needed, otoh it would not hurt :)

Also - will there still be possibility to access exposed words functions by path?

module/buy "Microsoft" $0

As for the correct binding, I am definitely for local module binding. IMO user should know, what he is doing, and if he is working in terms of module, he should not expect do will bind to upper module context. That would auto-expand "current module" to upper context and would be imo wrong and unsecure.

"A hidden variable is being set" - really? Do we have variables? :)

-pekr-

Brian Hawley
24-May-2006 23:11
Will there be a way to set a module as unchangeable, nonexpandable? To close a module, as it were? That way, you wouldn't be able to make further changes to a module from the outside, or from unchecked code.

A limit on WITH, at least for security purposes, would be nice. Otherwise there would be no security advantages to the modular code over the global namespace.

I'm cool with the current module concept though - your other module writings implied such an idea just to make them work. Nice to hear it said.

Allen Kamp
25-May-2006 3:19
:) Could a default module be set for a script as a bit of a safety catch? e.g if I haven't specified otherwise bind it to module 'scatchpad instead of 'current.
Goldevil
25-May-2006 6:16
The module concept becomes more clear in my mind, now.

Just a question about error context. When an error occurs and is trapped with 'try, what informations are avalaible at his level ? It depends if 'try is inside or outside the module and the spec of the module.

When error trapping it could be nice to be able to built a path to the error generator code.

The error occurs at ... Myscript / moduleA / moduleB / objectC / functionD

But, if functionD (or ObjectC or moduleB) is not public, is this informationmust be still avalaible ?

Michael Berg
25-May-2006 8:49
As Petr (if I understood him right), I would also think that it would be more natural and secure to bind the words of a script after some 'do in a particular module to that module and not the "current module" (exept if "current" means the module the do appears in). After all, a user shouldn't have to know what some exported function is doing, except for the case that a functions purpose is to create some new words in the "current module" (by loading another script). In my view every other behavior should be a special case, which in most cirumstances might even pose security problems, like redirecting newly bound words to another context. Also how can a function which "do"ed (did) a script rely on exported words from that script, if these words might get bound to some other (user defineable) place? Only way would be that they are "globally" accessable. But maybe is missunderstood things.
Ingo
25-May-2006 10:16
Given:

REBOL [title: "Module A" type: module] f: func [][print "Module A"]

REBOL [title: "Module B" type: module] f: func [][print "Module B"]

REBOL [title: "My Test"] do %mdolue-a.r do %mdolue-b.r

f

Which f get's called here? And how do I use the other one?

Volker
25-May-2006 10:41
Good semantics IMHO. Regarding where to bind i don't know. I prefer to the caller, but to the current module looks more obvious. To others: The point here is that modules can do user-code. Think of forall scripts[do scripts/1] Now 'forall comes from a module. Where should stuff be bound? I think in the callers context and not in 'foralls context. And specially build-markup "the value is " should use users context. Although that could be done with with current-module [..]

Also, if i 'do in the current module, i 'do usually a block, and that is already bound. so do[this] binds in the current module and do "this" in the users. Except of 'build-markup..

Hmm, maybe strings could be "bound" too, bind uses the module where they stem from? Blocks too, so with do [load file] the data is bound to the module of [load file] ? As Carl sayed, its deep.

Maybe, to keep things simple, how about forbidding 'do etc completely inside modules, forcing a module-reference? do/user, do/current, do/with module ?

Volker
25-May-2006 10:44
And HaloScan should learn how to deal with sourcecode. to write things like   build-markup "the value is <%value%>" (hope that works)
maximo
25-May-2006 12:11
I'd say keep things private and bind locally within the modules by default.

The primary use of modules is to add security. If we want stuff going back into our context, use objects!.

but I'd also add a mechanism to the modules themselves, so that one can be told to target another module (which can be the script context).

ex:

a-module/target: b-module a-module/do [f: func [print "a"]]

would actually add the 'f word :) to 'b-module

this way we don't have to annoyingly wrap our code in 'do or 'with. also, our current (script) module could change targets and work within a module.

actually a 'with word could temporarily set a module's target and place it back to what it was afterwards.

Pier Johnson
25-May-2006 12:11
:) Carl writes, "Let's say ... code runs an external script using ... do." And then Carl asks where should notation occur, i.e., using what source should interpreter's reader bind names to corresponding values for later denotational processing.

Conceptually, a module is external. This makes a module very similar to a server in REBOL/Services or a middleware transaction processing server stuck between a "data store persistence engine" (a database server) and a web server (a fancy file server, perhaps with string generation functioning).

Conceptually, the REBOL interpreter is much like an OS kernel -- an interrupt controller combined with memory management for processes. Just as an OS kernel uses a process box for switching between running programs by saving data about data currently in memory being used by running program, REBOL uses pages where it writes name chains to final values. This is the proverbial "binding" of pages to a book (aka context) and enables the proverbial "denotation".

Since there should be ZERO surprises, the binding all words should exist where there definition occurs. In other words, the process box that holds a module (conceputally a server) should be the book where REBOL binds its pages (names chains to final values).

Essentially, we're talking about a simple request/reply model where a "call to a functional" existing within a module is nothing more than a request to a server to do some "black box processing" and return us some result -- a number, a time, none, error message for malformed request...

Volker
25-May-2006 17:00
An important thought IMHO: It only applies to strings/files. Blocks are already bound inside their module.

And i think strings/files are mostly done on behalf of the caller. The module is already loaded/encapped and has no need to do other files. (Except of 'build-markup). So doing strings in user-space is the better idea IMHO. About security there are two issues: - If we bind user-supplied code inside our module, the user can overwrite all our stuff. - leaking: i never use strings to set words (at least i cant remember). So i doubt i would leak something in user-space by setting it accidentally. (beginners may do, they often build strings instead of using pathes and compose)

Volker
25-May-2006 17:02
Btw, can 'with be extended with a timeout? I would like that if modules are protective enough to allow user-code on the server.
Jeff M.
25-May-2006 18:41
Here are just some enumerated thoughts I had while reading Carl's blog and the other comments made:

- There should be a "user" module. This is where new words are defined, etc. This module can be set programatically, but it should default to something known. I'd imagine that most of the time it never changes. Think of Common Lisp packages. When the system starts up, it defaults to the CL-USER package.

- It would be nice if I could force the creation of a word to a specific module. Again, drawing from Common Lisp, I can use INTERN to create a new word specifically to a module. This would allow for runtime extensibility of a pre-defined module.

- Refinement paths should always be possible, but I think a "shorthand" would be nice. Consider something like this:

using opengl [ load-identity translate 0 0 -5 begin ; ... end swap-buffers ]

- Symbol name clashing should be very difficult for the average programmer to run into (IMO). Especially since REBOL just "overwrites". This brings up that issue, too, though. Should REBOL warn if this is about to happen (yuck)?

- I'd like to be able to both extend modules and lock modules from being extended. If someone created a "math" module with vector math, it would be nice if someone else in the community could extend it with matrices as well, and all these extensions work together in the same module space. It would be nice the the order in which they are loaded doesn't matter, either.

- It would be nice if we could somehow "encrypt" modules. I personally have no use for this, but the idea of modules does bring about the evolution into selling modules, etc. REBOL is so open now (and I love it) that I'm not sure this is really possible. Something to think on?

romano
25-May-2006 20:09
About the module 'export keyword. I think that the module should not add new words to the calling default module with the 'export word. It should be the calling script which, if it needs it, can import words from the module (like in Python), else the module could overwrite important words in the calling default module and generate name clash when many modules are used. I think that the word 'export should be more like the opposite of hide: if a word is exported it can be accessed with path and imported by the calling script in its default module with an import function.
Brian Hawley
25-May-2006 23:53
Romano, that sounds good to me. The other thing I'm wondering about is the names of the modules themselves. Will the names of all loaded modules be added to the namespace of the current module automatically? If so, someone could have a module masquerade as a standard function or object, a security hole.

I would prefer to only have the names of modules that your module or script needs visible to the current scope, sort of like the USE statement at the beginning of Oberon modules (rather than like the USE statement in Delphi which imports all of the module's exports into the local namespace). Other modules, even ones that are loaded and used elsewhere, shouldn't have locally visible names. This could be done with a Needs header or also a native! equivalent. Then, you can import the exported words into your local namespace with an IMPORT function like Romano suggests, with an optional /only refinement to let you specify particular words you want to import.

If you want to avoid two-stage importing, you could make all module names visible to the IMPORT function regardless of whether the module's name is visible to the local namespace. This would have certain advantages with ease of use, but may make it more difficult to determine module dependencies, making encap more tricky.

Brian Hawley
25-May-2006 23:57
Assuming the aforementioned IMPORT function, would there be a module that all scripts and modules would import by default? Perhaps a Core module, or a product-specific default import (/Base would import Base, /Face would import Face)?
Marco
25-May-2006 23:57
:) Question :

do load my-module.r

Will load my-module.r as a module or like standard script ?

Ladislav
11-Oct-2006 3:35:15
Romano said: "I think that the module should not add new words to the calling default module with the 'export word" - I am for the friendly and simple solution Carl suggested. If you want a more complicated thing, you can always use a "two level" process using two modules instead of one.

Post a Comment:

You can post a comment here. Keep it on-topic.

Name:

Blog id:

R3-0029


Comment:


 Note: HTML tags allowed for: b i u li ol ul font span div a p br pre tt blockquote
 
 

This is a technical blog related to the above topic. We reserve the right to remove comments that are off-topic, irrelevant links, advertisements, spams, personal attacks, politics, religion, etc.

REBOL 3.0
Updated 19-Aug-2017 - Edit - Copyright REBOL Technologies - REBOL.net