REBOL Parser in 10 Lines (plus Pretty Printer)

Author: Carl Sassenrath
Return to REBOL Cookbook

Here is a 100% correct method of parsing REBOL text. This code is very handy if you need to create special utility scripts that parse the REBOL syntax. (Note that trying to parse REBOL any other way can be very complicated!)

Beginners Take Note

To convert REBOL code from text to REBOL block format, all you need is the LOAD function. For example:

    code: load text
    foreach item code [print code]

will load and print each value and block found in REBOL.

The examples shown below are only needed in rare cases for scripts that must parse each "token" of a REBOL text file separately and act on it.

Here is the REBOL parser:

    parse text blk-rule: [
        some [
            newline |
            #";" [thru newline | to end] new: (probe copy/part str new) |
            [#"[" | #"("] blk-rule |
            [#"]" | #")"] break |
            skip (set [value new] load/next str  probe :value) :new

The input string is TEXT and the output is printed to the console window (using the PROBE function). You can change that to create useful utilities, such as a pretty printer (converting scripts to standard REBOL indentation format) or datatype colorizer. See below.

This code takes advantage of the PARSE and LOAD/NEXT functions. PARSE uses a dialect of REBOL to specify how to process any sequence of symbols. LOAD/NEXT loads the next lexical symbol (value) of the REBOL language and returns it as well as the position in the string of the next symbol.

Here's an example that pretty prints REBOL code:

    REBOL [Title: "REBOL Pretty Printer"]

    out: none   ; output text
    spaced: off ; add extra bracket spacing
    indent: ""  ; holds indentation tabs

    emit-line: func [] [append out newline]

    emit-space: func [pos] [
        append out either newline = last out [indent] [
            pick [#" " ""] found? any [
                not any [find "[(" last out find ")]" first pos]

    emit: func [from to] [emit-space from append out copy/part from to]

    clean-script: func [
        "Returns new script text with standard spacing."
        script "Original Script text"
        /spacey "Optional spaces near brackets and parens"
        /local str new
    ] [
        spaced: found? spacey
        clear indent
        out: append clear copy script newline
        parse script blk-rule: [
            some [
                newline (emit-line) |
                #";" [thru newline | to end] new: (emit str new) |
                [#"[" | #"("] (emit str 1 append indent tab) blk-rule |
                [#"]" | #")"] (remove indent emit str 1) break |
                skip (set [value new] load/next str emit str new) :new
        remove out ; remove first char

    print clean-script read %clean-script.r

These examples and a colorizer can be found in the REBOL library at www.rebol.org.

2006 REBOL Technologies REBOL.com REBOL.net