P
7990

P
7891

P
7804

maybe you see extension as handy tools, but I see them in an entirely different light. It allows us to extend rebol with as close to native access as we can get. extensions enable low-level use and tweaking of REBOL. its not a thing about security and danger... if you don't want extensions, don't allow them in your secure setup. I don't want extensions to be limited because they are a security risk. by definition, they *are* a security risk. so limiting them based on security concerns is basically an oxymoron .. no?

P
7802

I specifically do not want the user of an extension to have to fiddle around with

P
7798

P
7797

P
7796

P
7779

really, all I'd like to be able to do is get similar functionality to load/transcode directly within the C API.

where things are bound (*if at all*) would be options for the mode argument. possibly there could be an additional arg (optional i.e. when non NULL) to provide a context to bind to.

binding to the extension's module sounds like a nice feature as well as normal user context support.

P
7778

P
7777

when I say NOT user lib I mean not the reference which is within the current user context for the value of the word 'LOAD.

the idea of post 7769 is to make it *easy* to integrate into and out of REBOL data for the many cases where it is directly possible.

this could be a single datatype (a tuple, a date, etc) or complete block-like datasets.

P
7776

some tools will want to be able to explicitely get access to the user context because they must not be tampered with by the user script itself.

for example a tool to list current unset user words. could be part of a debug extension. something of a get unset list ... do something ... get new unset list provide some causality on effect of user code.

P
7769

P
7768

P
7767

debugging and utilities: -- Currently, debugging and inspecting data coming from the core within extensions is very complicated to achieve because we basically have to make our own print statements which build-up the data, often requiring loops and several lines of code for even simple values (ex: a tuple!).

If we had access to the molded string of any RXIARG, (in plain, serialized or even its 'FORM variant) we could do this MUCH more easily. The mode argument above would switch between these 3 modes.

tool integration: -- Also, REBOL relies on its transparent concept of data as code, and IMHO, a big part of that is that it can be manipulated to/from strings with no fuss.

currently, we cannot serialize or load from within an extension and this limits how easily we can inter-operate with other APIs which we want to merge to rebol.

Many tools save or provide serialized datasets which are very close to rebol's native format, so they can be used as-is within rebol... but right now, we can't provide that native interface from within the extension...

The same is true on output. We can't provide Rebol data as strings to external tools, without a lot of hoops.. (ex, converting a tupple to a text string so it can be displayed in an OpenGL text polygon.)

All of that stuff is complex, and its already built-in... we need it and don't want to redo what's already there.

note that the mode argument of RL_Load_String could be used to switch string loading in different ways... 'LOAD, 'TRANSCODE, /only /next /all /unbound, etc.

P
7765

debugging and utilities: -- Currently, debugging and inspecting data coming from the core within extensions is very complicated to achieve because we basically have to make our own print statements which build-up the data, often requiring loops and several lines of code for even simple values (ex: a tuple!).

If we had access to the molded string of any RXIARG, (in plain, serialized or even its 'FORM variant) we could do this MUCH more easily. The mode argument above would switch between these 3 modes.

tool integration: -- Also, REBOL relies on its transparent concept of data as code, and IMHO, a big part of that is that it can be manipulated to/from strings with no fuss.

currently, we cannot serialize or load from within an extension and this limits how easily we can inter-operate with other APIs which we want to merge to rebol.

Many tools save or provide serialized datasets which are very close to rebol's native format, so they can be used as-is within rebol... but right now, we can't provide that native interface from within the extension...

The same is true on output. We can't provide Rebol data as strings to external tools, without a lot of hoops.. (ex, converting a tupple to a text string so it can be displayed in an OpenGL text polygon.)

All of that stuff is complex, and its already built-in... we need it and don't want to redo what's already there.

note that the mode argument of RL_Load_String could be used to switch string loading in different ways... 'LOAD, 'TRANSCODE, /only /next /all /unbound, etc.

P
7764

it would be much more practical if the RL_Words_Of_Object() function returned the number of words it returns as its first argument, just like the RL_Map_Words()... as per its spec.... The [0] value is the size.

it took me a while to realize this, especially since the documentation refers to RL_Words_Of_Object() saying: "Returns a word array similar to MAP_WORDS()"

P
7763

#define RXA_GOB(f,n) (RXA_ARG(f,n).series)

but shouldn't it be : #define RXA_GOB(f,n) (RXA_ARG(f,n).addr)

P
7762

These are easy to setup, I've added them within my includes because when using words in objects, we don't have a stack frame, but a naked RXIARG to set/get.

note that to use these with dynamic allocation just use the macro with *(arg) instead of arg... ex: RXV_OBJECT(*(arg)) = my_object;

#define RXV_INT64(a) (a.int64) #define RXV_INT32(a) (i32)(a.int64) #define RXV_DEC64(a) (a.dec64) #define RXV_LOGIC(a) (a.int32a) #define RXV_CHAR(a) (a.int32a) #define RXV_TIME(a) (a.int64) #define RXV_DATE(a) (a.int32a) #define RXV_WORD(a) (a.int32a) #define RXV_PAIR(a) (a.pair) #define RXV_TUPLE(a) (a.bytes) #define RXV_SERIES(a) (a.series) #define RXV_INDEX(a) (a.index) #define RXV_OBJECT(a) (a.addr) #define RXV_MODULE(a) (a.addr) #define RXV_HANDLE(a) (a.addr) #define RXV_IMAGE(a) (a.image)

#define RXV_GOB(a) (a.series) // gob examples used RXA_SERIES...

but shouldn't it be : #define RXV_GOB(a) (a.addr)

P
7761

Since the user context has references to the other resident contexts, this allows us to pick and set values anywhere in REBOL as if the command where a normal rebol script (with usual restrictions).

P
7760

returns 1 or 0 (if we actually shortened the series or not) ---- These would allow us to shrink a series "in-place" effectively allowing 'REMOVE or 'CLEAR like functionality from within extensions.

I would expect RL_Series(prim_obj, RXI_SER_TAIL) to be up to date after calling RL_Remove_xxx .

Right now I can't process state flags in a block and remove them... I have to actually rebuild a new block each time.

This isn't optimal and its possible a block needs to be shared.

In my opengl rendering loop, I may be manipulating hundreds of primitive state blocks 50 times a second... it REALLY adds up.

P
7759

Mainly, there seems to be a few core functionalities missing in the current Host-Lib (RL_xxxx) API.

I'll list each one as a separate post so it can become a thread.

P
7738

P
7737

P
7735

http://www.pointillistic.com/open-REBOL/moa/files/first-opengl-R3-Application-ever.png

I did a quick integration of the custom rendering gob within the latest R3, just to see how hard it would be to wrap it into a higher-level frame which I didn't control... it was a breeze.

the system is using the custom rendering API which I discuss on the rebol wiki here: http://www.rebol.net/wiki/Custom_Gobs

though the document is a flying target, it does give a idea of the scope and seriousness of this endeavor.

as I encounter issues and approach new architectural topics, I lay them down there.

P
7722

basically the idea is to allow ANY rendering package to embed itself within a standard gob with minimal fuss for the user.

imagine something like:

web-gob: make gob! [renderer: 'web-kit custom: http://www.rebol.com size: 800x500 offset: 0x0] view/as-is web-gob

and poof, there is a web browser with rebol.com within it.

I spent the better part of the day documenting the whole idea as a proposal on the doc-wiki

it has a lot of the requirements, changes and even some code examples with file names identified.

http://www.rebol.net/wiki/Custom_Gobs

don't hesitate to comment here on any issues.

P
7721

1. On GOB types: they are defined in reb-gob.h

2. On custom GOB types: no. GOBs are primitives, and the TYPE defines how they map to and from REBOL constructs. If you want to define your own graphics constructs, do not use GOBs. However, that being said, it would be possible to allow a CUSTOM type, where the content field could be whatever you want. But, there would be no map to REBOL.

3. GOBs created by the interpreter have their own allocator, so you cannot change size of a GOB. However, I suppose you can have an external GOB that defines whatever you want.

Of course, the fundamental question is what are you trying to do? What additional features do you want in a GOB?

P
7720

In the method compositor::cp_process_gobs() there is a switch statement which selects the process to use based on the GOB_TYPE ( rebol_gob->ctype ). I do not see any where else in the complete host-kit where this is Assigned.

this fact leads me to think that we cannot create our own gob types within the host kit.

am I wrong? if not, if I manually set this to another value, will it cause mayhem in the system?

I have successfully increased the size of the rebol_gob struct without any crashes in actual use, so I am thinking that with the use of another extension or some tricky host-kit alterations that I might get something working... but I fear to go to far only to discover that what I am doing is basically going to corrupt the interpreter.

P
7691

P
7690

I have a new version with a few bugs less, but I guess your version is better.

P
7685

P
7684

And... various users will want it to work with Linux, etc., which use the read-line code that I wrote earlier to handle line editing in a simple way (or whatever substitute might become available that runs on all other OSes.)

Also... I've got some C code here from R2 for it's method of doing similar input handling, I just need to publish it to the site.

Anyway, thanks for the submission... and something like this is useful to all programmers.

P
7682

P
7675

P
7674

P
7673

P
7672

P
7661

P
7660

P
7659

DECODE is actually a lower level function, probably not used much in code because it's mainly part of the LOAD function.

However, I do understand your concern about the order, and if we're going to change it, needs to be very soon. We should look around other functions and see if there are many other examples for it. I made the assumption that the codec should be the first arg, but that was just my gut intuition on it.

Let's figure it out.

P
7654

P
7654

P
7647

P
7646

P
7643

CALLBACKS HAVE SPECIAL RULES! And I'll be updating the documentation to cover them. For now, just keep things simple. Avoid callbacks to commands to callbacks to commands unless you really know what you're doing. Remember that REBOL can GC at any time, and poof goes your data... if you're not referencing it properly.

P
7640

P
7639

Perhaps post it somewhere, and I will take a quick look.

P
7638

P
7637

P
7636

P
7611

and for those who want it:

www.rebol.com/r3/downloads/libr3-a102.so

but, you'll need the posix host parts from an earlier host kit release. Merge them with the above zip host-kit sources, and the build for Linux should work.

P
7564

If you are interested in this host-kit, contact me.

P
7328

P
7320

P
7313

P
7284

P
7283

P
7282

P
7277

P
7257

We could go for more powerfull shell stuff, Like python uses in IDLE, but it has to be coded and for the factual benefits,. I'd rather have Carl concentrating on MUCH more serious stuff.

anyone else can step up to the plate though... and so far it isn't bothering us enough to do so.

I don't even miss multi-line pasting, cause i never do so... it fills up the history with many lines...

much better to: >> do read clipboard://

P
7256

P
7255

Which is not to say that I wouldn't want the option of an R2-style GUI console for REPL development, and maybe the text-mode chat. At runtime though, I would either want a real Windows console, or an embedded console, or no console at all, but not the R2-style console.

P
7254

P
7252

P
7251

P
7249

P
7248

I have found one useful trick how to set the R3 Windows console to the "QuickEdit Mode"(ie enable context menu with copy/pase functions and text selection/copy/paste using mouse buttons directly) by default when R3 is executed:

In src\os\win32\dev-stdio.c file inside function Open_IO() put right after this line:

if (!Redir_Inp) Std_Inp = GetStdHandle(STD_INPUT_HANDLE);

following call:

SetConsoleMode( Std_Inp, ENABLE_LINE_INPUT | ENABLE_PROCESSED_INPUT | ENABLE_QUICK_EDIT_MODE | ENABLE_ECHO_INPUT );

What do you think?

P
7203

To prevent multi-module linking, pass -single_module to the linker (e.g. -Wl,-single_module). Single module linking is the default since at least Xcode 3.2.

I've now tried the example code I posted in #6332 on an OSX 10.4 on PPC using Xcode 2.5, and after adding -Wl,-single_module the resulting library looks clean.

Hope that helps. I'd really love to see a OSX version accompanying A98.

P
7158

P
7156

P
7153

P
7152

P
6627

P
6355

P
6348

Of most interest would be to add callbacks. This will enable a bunch of new applications.

P
6347

P
6346

It seems that it works well enough, no?

P
6345

So, WRT R3, it is the top priority, and there are various areas of development on it. The most recent was the host-kit, and soon we'll do another build to address the new bugs in CureCode.

Mixed in with R3, we're also working on the website... for "marketing" reasons. This is long overdue, but absolutely necessary.

And finally, we want there to be a "background job" where we again release R2 updates. Personally, I'd like to see a new version each month (but, maybe I'm the only one who wants that.) The reason is simple: if we only release a new R2 every year, then there's too much to add and to test at the same time, and it becomes a very large project... often getting delayed for long periods.

Thanks again for the note.

P
6344

However, with the success you posted here, I am encouraged to give it another try... will let you know, thanks for the posting.

P
6334

My department is the last stop for escalation issues. We have the say so when it comes to what impacts production and how long we can hold up recovery just for the sake of discovering root cause. This is a major inroad to advancing the REBOL namespace. Developers escalate debugging issues directly to my group. I'm excited as you can probably tell. Hopefully, R3 is not restrictive and advances rapidly. I'm eager to see it hit the street in my domain..

P
6332

Here's a simple example.

foo.c:

extern void bar();

__attribute__((visibility("default"))) void foo() { bar(); }

bar.c:

#include <stdio.h>

void bar() { printf("bar"); }

If we build a dynamic lib from this using:

gcc -fvisibility=hidden -c -o foo.o foo.c gcc -fvisibility=hidden -c -o bar.o bar.c gcc -dynamiclib -Wl,-x -o lib.so foo.o bar.o

We get the following output from nm:

lib.so(single module): 00000fce T _foo U _printf

Hope that helps. The above used gcc version 4.0.1. I've put up the above (including a Makefile) at http://bolka.at/2009/misc/exports.tar.gz.

P
6331

extern void bar(); "))) void foo() { bar(); }bar.c: stdio.h> void bar() { printf("bar"); } build a dynamic lib from this using: dden -c -o foo.o foo.c bar.c gcc -dynamiclib -Wl,-x -o lib.so foo.o bar.o t the following output from nm: lib.so(single module): 00000fce T _foo U _printf above used gcc version 4.0.1. If anyone wants to play with above example, I've put it up at http://bolka.at/2009/misc/exports.tar.gz (including a Makefile).

P
6330

Maybe someone there will know what to do with OS X.

P
6329

P
6323

P
6322

Then, use gcc -dynamic-lib -o test.dl a.o b.o

Then use nm on test.dl -- you will see all the funcs in both a and b. Try _attributes_ and __private_extern__ and see if you can keep them from showing on nm -s and other options. If so, you've succeeded.

If so, let me know what I need. Also, I use gcc 4.01. Perhaps that's the source of my problems, but it is > 4.0.

Also, try gcc -fvisibility=hidden which in theory removes all non-exported symbols.

Let me know... and thanks!

P
6318

P
6316

P
6315

These get called internally by gcc, which passes arguments on to these.

P
6313

P
6308

P
6306

There are three choices on it: 1. find a tool that does what I need 2. make a tool that does what I need 3. join all the sources into one large .c compile

Note that gcc -fvisibility=hidden does not work, nor does __private_extern__ work either.

I've got to get on with other projects now. So, if you happen to find the solution, let me know. (PS, yes, using GCC > 4.0.)

P
6305

P
6304

P
6302

P
6297

P
6294

P
6293

The error message points to a condition that should be prevented or guarded by the REBOL code.

P
6292

Most open source packages use the GNU AutoTools for configuration. That means that the makefiles are generated, and very long and indeed unreadable. The source files are usually fairly clean, but in practice, you often end up debugging in the generated makefiles.

REBOL programs and configurations are usually much cleaner for two reasons. They usually execute their models directly instead of generating intermediate code in different languages. And mainstream programs tend to want to adapt to and use many peculiarities of host systems and support libraries. For which they need the many compile time configuration checks. REBOL and a very few other projects go in the other direction by minimising dependencies and concentrating them in a library interface.

I often wonder if I was the last one they taught in university that loose coupling between components is to be preferred over tight coupling, or if I was the only one paying attention.

P
6291

P
6290

It makes no difference on Syllable Desktop, though, so that must be an earlier error.

P
6289

for one thing, I'll be supporting my OpenGL work on OS X as soon as I can.

P
6286

P
6285

Back in April, I modified the code to get the message base in chunks to prevent this problem. I was at a different location that had long lags, and that's when I first noticed that the timeout was happening.

P
6283

After debugging the segfaults a bit, I decided that it was GCC to blame for what looks like a miscompilation. Either dropping -fpack-structs or -O1 fixed this for me, and I've since been sucessfully using my locally built R3 core for R3 chat. In fact, this reply is posted this way.

P
6281

When it comes to the host build, I want to make sure we can eliminate as many "barriers to builds" as possible. So, using standard accepted makefile format is good... as long as it's kept simple. (Many of the makefiles of primary tools are very difficult to read... so we won't be doing that.)

P
6280

I should mention that I've had -dynamic-lib built OS X libr3 and host working for several days. But, the libr3 isn't in the form I want, because it's not internally linked and resolved. Examining it with nm it looks like just a concat of .o files.

Specifically, I want all internal symbols resolved, and I only want to export the library interface.

If OS X only builds libs (dynamic or otherwise) as concatenated .o files, that's a serious breach of coding ethics! There are two reasons:

1. it means I can link against the internal interfaces - a serious short circuit in code encapsulation rules.

2. it means I can discover the entire internal structure of any product... say I want to peek inside Photoshop to see how it does something.

If I nm a lib that's been properly prepared, I should only see its API, nothing else. So far, this has not been possible on OS X.

I suppose I could easily confirm this by nm'ing some of the various apps I have on OS X and checking if I can see their internals. Let's hope not.

P
6279

P
6276

P
6275

P
6274

P
6273

P
6272

P
6271

This works great when running older programs on a newer library. However, when an executable is built on a system with a newer library and then transferred to another system with an older version of GLibC, it still just bombs out with hardly a clue.

The Fedora builds of REBOL, both 2 and 3, do run on newer Linuxes, from the last two years or so. On older distros, or conservative distros with older libraries, they bomb out with a floating point exception. So that would be a case of libm.so.6 being too old.

P
6270

Ah, fair enough if one build is meant to cover as many Linuxes as possible. However, it remains to be seen how far this will go. In any case, it's essential information for people to know on which OS version a binary was built.

P
6269

--- Note: login initiated--- Note: fetching user list pt error: cannot use subtract on word! value msg either either post-text either post-reply parse try either either forever command-loop make context do catch either either applier do try chat** Near: - 1 = length? msgs-base [ append-msgs reduce [r]] [ ... logged to error-log.txt

I'm afraid I will have to copy it again from the screen line by line.

P
6268

P
6267

At a glance, Apple seems to be using a -dynamic linker directive instead of -shared, and as others mentioned, .so is suffixed .dylib instead. Environment variables for search paths to libraries are also different, if you need those. (Syllable Desktop uses $DLL_PATH instead of $LD_LIBRARY_PATH on Linux, but I didn't need that in the "port".)

P
6264

A very old Ubuntu just says segfault when I try to start chat fresh.

Syllable Server gets a little farther and is more specific. It creates the empty database, but then segfaults on login. When I start it again with the empty database, it segfaults immediately. Here's one of them:

Script: "RebDev Command Line Interface" Version: 0.4.21 Date: 3-Dec-2009/0:00 r3[2966]: segfault at 14 ip 08049c9b sp bfe9a880 error 4 in core[8048000+6000] Segmentation fault

Syllable Desktop doesn't segfault but already errors out on creation of %base/, so maybe at the same point as the old Ubuntu, with a REBOL error that seems to come from CREATE.

Note that monolithic A96 has no problem with chat (I'm on it now).

P
6260

http://developer.apple.com/mac/library/documentation/DeveloperTools/Conceptual/DynamicLibraries/000-Introduction/Introduction.html

P
6259

here is the apple reference page on the subject... scroll down, you will see some info about using and compiling them:

http://developer.apple.com/mac/library/documentation/Porting/Conceptual/PortingUnix/compiling/compiling.html

hope this helps!

P
6257

http://bolka.at/2009/patches/posix-loading.patch

This patch, another patch to reb-ext-lib.h, and a fix to bug #0001382 allows me to load the extension compiled from the sample shown on the R3 docs page [1]:

>> append system/options/file-types [%.so extension] >> import %ext/sample.so >> add-mul 10 20 30 == 900

[1]: http://www.rebol.com/r3/docs/concepts/extensions-making.html

P
6254

P
6253

P
6252

Suggestions?

P
6251

P
6250

P
6249

P
6247

Needless to say, send me any makefile corrections you've made, and I'll integrate them into the releases. Thanks.

P
6246

In other words, OS X does not appear to have libs. It uses concat'd objs (like the old BCPL Amiga libs.) Please tell me I'm wrong, and let me know what it takes to build one.

So OS X goes on the "wait list" until we figure this out.

P
6245

P
6240

P
6239

P
6238

GCC produces one warning when compiling the host code, the same with GCC 4.1.2 on Syllable Desktop and GCC 4.3.2 on Syllable Server. So if you can fix that, it will be perfect:

src/os/host-main.c: In function 'main': src/os/host-main.c:110: warning: passing argument 2 of 'Parse_Args' from incompatible pointer type

P
6237

(It's the bucket of crabs pattern. All the crabs are angry and decide that they want out, cheering each other to climb up. But when one crab gets ahead of the others, they get angry at him and pull him back in the bucket.)

P
6236

Many tools expect that just like humans, so for example the way most Linux shells are configured, with tab completion per command, tab completion with the tar command doesn't work with a .gz file, and my build has never before come across a .gz with a tar in it.

P
6235

I tested the pre-built host version for Fedora and compiled the host versions for Fedora and Ubuntu. They all worked. This is on a highly experimental development version of Syllable Server 0.3, currently with Linux 2.6.25, GLibC 2.8 and GCC 4.3.2.

I do have some recommendations to make the process even smoother.

P
6234

We'll have to investigate whether this is an incompatibility with Desktop or a weakness in the hack with the Linux binary.

P
6233

http://syllable.cvs.sourceforge.net/viewvc/syllable/syllable/system/apps/utils/Builder/packages/Syllable/REBOL-Core--current/REBOL-Core--current.recipe?view=markup

P
6232

http://syllable.cvs.sourceforge.net/viewvc/syllable/syllable/system/apps/utils/Builder/packages/REBOL-Core--current/REBOL-Core--current.recipe?view=markup

P
6230

It's not supposed to work at all, because Syllable Desktop is our non-Linux based system (our own stack from the kernel all up to the GUI) and R3 isn't ported to it yet. However, Syllable is closely aligned to Linux, or better said GNU, using much of the same infrastructure (you could picture our own software stack as vertical, and the ported infrastructure as horizontal). Syllable uses the GNU C library, GLibC, like most Linux distros, and REBOL uses the C library to interface with the system. Now we use the GNU versioning of GLibC, which is different from the GLibC library versioning as used in Linux (for some historical reason from before my time with Linux). So I compiled the R3 host source code on Syllable Desktop and linked it with a binary R3 core library from Linux, a trick we have done before with a Linux binary codec. The result can't find the libc.so.6 and libm.so.6 libraries from GLibC, but symlinking those to Syllable's libc.so.2 and libm.so.2 makes it work!

This works because the libraries are basically the same, because the way they are used by Linux and Syllable are much the same, and because R3 is very well isolated from the host system. This is not normally the case, and to illustrate that this is a fragile hack: it works with the R3 library compiled on Ubuntu, but not with the one from Fedora. Not everything works, either, and I have to do more testing. For example, initialisation of chat doesn't work, so I couldn't proclaim total victory by posting from it. ;-) But HTTP works. I can even load the GUI demo, but it doesn't work, of course.

So, R3 still needs to be ported to Syllable Desktop for real, but I think it's interesting that it works even without porting, while other platforms require real porting work. :-) I didn't even have to make any changes in the host source code as configured for Linux. The binary library just working makes it very likely that porting the R3 library to Syllable Desktop from source will also require very little if any modification.

P
6229

P
6228

P
6227

P
6223

P
6222

P
6218

I could just jailbreak it and convert a port to it I guess... but we still need the hosts in the native cpu's format.... unless ARM cpus are compatible with x86 instructions?

P
6216

P
6212

Next, will build for Linux/Ubuntu. Let's see if there are any diffs.

P
6211

P
6210

P
6209

P
6208

P
6199

I'll be doing a lot more tests in the next few days.

P
6184

making sure the project is set to using unicode.

the first time I setup the host, all compiled without issue. then after a few tests, changes and a completelly rebuilt solution which now contains all my projects related to r3 binary stuff (r3 host, OpenGL extension, and others) the app compiled with ascii versions of Windows strings functions, complaining of type mismatch in compilation, and showing mangled text for local, version and other strings... (all OS-based).

so I did a diff between this new project file and the old one, and saw that the character set in the new one was set to multi-byte by default.

P
6183

P
6182

P
6172

P
6161

P
6160

In a castle up on the mountain, electric storm and lightning rods pointing to the sky, chains dangling...

parts sprawled around, files here and there... all linked by bits and pieces...

lever is pulled... the storm roars... the machine starts turning, things move, it groans, compiles... it pulls on its links...

... its alive!!! yes... Frankenrebol stands up stairing at me against the black background and just waits for me to give it orders...

rubbing my chin I ask myself: "shall I take over the world?"

Or should I just help the invasion .... like it says in the headers?

:-)

P
6143

P
6116

P
6104

P
6103

Because I'm not sure how much feedback to expect or immediate corrections to make, I'm only going to send it out to a few developers (as I mentioned much earlier.) I am hoping that by later this month, with enough effort, we can do a larger release, and have a mechanism in place to manage it (via Curecode or whatever.)

Also, just to note, this is the first stage of the Host-Kit, as defined on the wiki host-build page.

P
6093

P
6092

P
6091

http://code.google.com/p/fog/wiki/Benchmarks

p

P
6090

http://code.google.com/p/blitjit/

http://code.google.com/p/asmjit/

P
6089

However, that same licensing issue affects us. The best alternative to LLVM is libjit, but libjit is LGPL - this means that we can't incorporate its code into R3 itself to build a new rebcode on. We have to wrap libjit in an extension.

LLVM is BSD licensed, so we could in theory use some of its code. The problem is taking the JIT out of the rest of the 12MB binaries, and hopefully it will be small enough to use. We can't just link to the system supplied LLVM, because no system supplies it: It's optional on OS X, third-party on other platforms.

Technically, the only advantage LLVM has over libjit is PPC support, and likely better ARM support. Those are not insignificant.

P
6084

P
6067

P
6066

P
6065

I would also like to install and test haiku... especially since they support the latest mesa drivers which include the experimental gallium OS agnostic HW 3d drivers. one HW driver for any OS which uses mesa for any of the gfx cards which have a gallium driver... this could be our solution for a stand alone OS running REBOL with HW accelerated 2D&3D out of the box.

I also want to start looking at LLVM as soon as possible, especially with rebolek's rebC dialect, which I would use as a basis for run-time compilable REBOL command! & extensions.

LLVM might not be the smallest jit engine out there, but it has a lot of traction, and its use with REBOL is a definite marketing advantage. LLVM is getting A LOT of industry attention. Ruby already uses it on Mac.

one possible use for LLVM would be for the compilation of 3D shaders and geometry deformers, using rebol code as source code.

we could also use it for the creation of fast 2D filters and pixel effects.

P
6064

Other x86 or x64 platforms can be added virtually as needed.

It might be a good idea to follow Fedora's lead and make the Win32 version of R3 optimized for Atom processors. People who want high-end performance on Win are going 64bit nowadays. On 32bit, most of the laptop and desktop processors are massive overkill. The only areas in Windows where 32bit performance is a real problem is in the netbook/top and microserver space, and most of those processors are Atom or much like them. Out-of-order execution can handle a lot of the subtle differences for a lot of code, but Atom is in-order.

More recent GCC versions (like the one Robert linked) have Atom optimizations. We should use them, at least for 32bit x86. For 64bit we can go all out though.

P
6063

P
6062

It's worth using the newe GCC version.

P
6049

P
6048

P
6047

P
6046

The build configuration system for R3 is done in REBOL itself, in order to make it manageable, because other methods are too complex.

In fact R3 uses different scripts from R2, because the R2 scripts evolved to became over time too complex to understand. So, a lot of simplifications were applied to R3 build methods.

The end result for Host code will be such a simple build... that it seems as simple as a beginner's C project, allowing almost anyone with C basics to be able to help support the host environment or port it to other OSes. We won't need gurus for that.

P
6045

For the graphics, I want to move the interface to use the command! datatype. So, the first step is to just get the core OS interfaces released, then go from there.

P
6042

have you tried MinGW on the POSIX platforms to get GCC and MinGW working there or where you already supporting it on POSIX?

P
6041

P
6040

P
6034

Some cleanup to do, but getting close now.

P
6033

Problems with MinGW solved by not using Posix OS model, but using the Win32 model.

Now, on to debugging (it crashes.)

P
6029

P
6028

However, the host dev-stdio.c errors out: SIGHUP not defined. So, it looks like we cannot use a pure Posix mode for Win32 core builds.

P
6027

P
6026

P
5999

P
5987

P
5927

that should be a good initial test from source host to running compiled version using an extension.

P
5913

P
5912

P
5911

P
5901

P
5055

P
5041

P
5040

P
5038

P
5036

It should also be noted that COMMAND func can be defined within your plugin as well. It is a plugin local function that stores the index counter for you.

P
5034

P
5032

P
4340

P
4336

P
4334

P
4212

The advantage to the plugin model is that you can implement the other models as plugins. We can implement a plugin that is included by default that provides a dialect to describe and dynamically link to DLLs with C APIs. This will give us the capabilities that R2 has, and more if we do a better dialect, as Ladislav suggests.

We can also have the community make other plugins to wrap other APIs that are not C compatible. Even a generic wrapper dialect for these other API models would be easier to use, since almost all of them provide reflection facilities.

I do think that we need a simple wrapper dialect for C APIs, and that this can be provided for all platforms by default. However, I think that we need to do it with a (internal) plugin, if only to make sure that thee plugin API will be powerful enough to implement such wrapper dialects.

P
4211

P
4210

P
4209

Then there are things like COM, which is mostly just an "agreement" how to layout a specific form of a DLL and how to use it. Mostly the language independent vtable structure is in there. Overall, COM is quite a smart approach.

And finally you have the runtime structure of languages like C++ and how objects are layouted. This varies a lot from language to language.

Overall, as long as a language can support the standard DLL ABI for a platform it will be useable with the DLL interface.

P
4208

P
4207

P
4206

I'm glad to see there is some interest in this area, because I was thinking no one was needing it.

Also, just a note, it's not necessary to rebuild the host for doing codecs like TIFF. A lot of such addons can be done with plugins. I'll be saying more about those very soon.

P
4205

Option 1 is limited to wrapping C-compatible libraries, and that is way too limited for us. We need to be more interoperable than that.

P
4204

P
4203

P
4202

P
4199

P
4194

P
2585

Also, I've got to get the GUI code up first, since that was promised even earlier.

P
2576

P
2571

P
2570

P
2569

P
1875

P
1874

For those new here, an "intrinsic" is a built-in function written in REBOL that is called by a built-in native code function. DO is one of those native functions in R3 that calls an intrinsic, system/intrinsic/do, which I wrote. All of the script handling code of DO is in the intrinsic (string! url! file!) and all of the rest of the code is in the native portion.

This means that not only can PeterWood's feature be implemented easily, but that is can be implemented by *anyone* at runtime if they want to. It doesn't have to bu built into REBOL, and there is good reason not to - it might break files on non-Unix platforms. It *absolutely should not* be built in *only* on Unix platforms, since that will make REBOL scripts for Unix incompatible with other operating systems, which is against REBOL principles (in R3 at least).

This all means that either this feature can be loaded in a module, or it will be built in for *everyone*, not just Unix users. No compromise.

Of course you can build in whatever you like into hosts you build yourself :)

Apparently it needs repeating: Incompatibility with R2 does not make something a bug in R3 - it needs to be a bug on its own merits. We are making deliberate changes in R3 relative to R2, and some functions in R3 are completely new, including functions named the same as functions in R2 that acted differently. Incompatibility between different platforms of R3 *is* a bug.

P
1872

Just be patient in the meantime, and submit (or do) those feature requests :)

P
1871

P
1870

P
1869

P
1868

P
1865

P
1863

P
1861

P
1860

P
1859

Plug-ins like for iPhoto, Editors, Filemanagers etc.

Being not able to use Rebol to write plug-ins is really something I'm missing.

P
1716

P
1696

P
1695

IMO it's not a waste of time because we need to analyze the old approach anway.

P
1683

But, I am also considering changing the primary external interface method to use "direct-rebin" because if I don't then there's no decoupling between internal and external, and we'll create all kinds of version incompatibilities for ourselves.

P
1671

P
1670

P
1667

P
1666

IMO getting DLL interface into R3 could really boost the testing. I would immediatly begin to port my app to R3.

P
926

Goal is to get the most complete overview about current R2 situation.

P
802

So, we have to make two sides simple and clean: 1. The Rebol side 2. The C level side

For 2nd we need a way that existing DLLs don't need to be changed etc. I just want to use them as one can find precompiled libs etc.

Take the DLL, read a bit of the API, plug the DLL into the host-lib environment, create a Rebol side DLL interface, use it.

P
761

Is the DLL interface meant to cover the system port functionality as well, or will we still have a system port?

Are there five or six prime libraries we can identify, that we can use as guinea pigs, to see how they are best integrated into REBOL (e.g. PCRE, ImageMagick)? I've done middleman DLLs as well, which is fine for more advanced users, but not for your average REBOLer. Today, you don't need to know C, have a compiler, etc. and you can use DLLs, even if you have to suffer a bit.

P
752

P
751

Option 2 is bringing your DLL into host-lib context (statically linked or dynamically loaded), maybe you add some helper functions to the DLL to call Rebol code etc.

Option 3: Is a mix of both. You control the DLL integration from Rebol (for people like Petr) and include the DLL into the host-lib context. Option 2 is a C-level only approach, Rebol gets what's defined in the host-lib. Option 3 makes it possible to tweak this a bit from Rebol side.

P
744

P
735

P
734

P
733

P
726

P
724

P
723

P
706

P
705

Of course, that method will not work for system DLLs unless they are wrapped with a plugin dll that provides the standard entry points.

P
704

Devices use a standard method API entry and event-driven results.

We will need to determine if this same mechanism will be what we need for DLLs.

P
702

P
701

Option 1 shouldn't be that hard to do nor have a influence on the Rebol internal timing. Option 2 needs to be strongly decoupled from the rest of R3 and just do its job.

P
700

P
697

For example, in R3, there is a constant stream of data moving through the event system and the WAIT function. This includes high precision delta timers for keeping the timing accurate. If a DLL causes problems in that flow, then that will buble up into REBOL code. We must avoid it.

Anyway, we can get this outlined on the wiki page.

P
696

P
690

P
689

2. A DLL loader that brings the specified functions to the Rebol world. I expect that we are more flexible to deal with DLLs on the C level.

3. A generic way how to do callbacks (or do you have a better idea)? The only one I have is to make a callback-catcher that translates this to something like a localhost interface. But maybe the new architecture of R3 allows to call Rebol code from C level.

P
687

1. I think the wiki page is a good idea for keeping track of it. 2. The DLL interface in R2 was pretty complicated. Not really that much code, but not easy code. Jeff Kreis wrote and debugged it. (See, I did not write everything!) 3. And here is really the most basic question: do we want to do it like R2 as a dialect, or is it better/easier/faster to do it in the host part of R3.

The reason I mention this is because the host is the wrapper around REBOL, so in theory, anything is possible. REBOL itself just runs like a DLL within it.

So, that's really the starting point on all this. I think perhaps it gives major advantages. For example, for video in a window, all of the window handles, funcs, and interfaces are accessible at the host level.

P
680

P
679

There is many guys interested in Core functionality more than in GUI. What about setting up wiki page, like we did with Parse? That way our ideas get recorded, and someone (e.g. Robert :-) would moderate it.

Carl would have to say, what is his plan here, as IIRC he said that DLL interface might be best added via plugins. But plugins are not ready yet.

P
678

P
677

P
676