This document covers the main concepts of REBOL's new asynchronous
network ports.
1.1 Newer REBOL Version Required
In order to run the examples shown below or your own code, you will need
to download one of the newer "async"
alpha releases of REBOL/Core.
Versions 2.5.53 and up should work, but the newest version is always the
best for testing purposes.
1.2 What is an Async Port?
An async port is a REBOL port that can perform an operation such as
sending and receiving data without forcing REBOL to wait for the result
each time. An async network port lets you connect and communicate over a
network while the rest of your program continues to run normally. This
is an important feature when writing interactive clients and high speed
servers.
Async operation is done as a mode of the port, and you indicate that you
want the port to run async when you open it the first time. See the
details below.
1.3 About the Built-in Protocols
The built-in protocols (HTTP, FTP, SMTP, etc.) of REBOL still operate
(as of this release) in synchronous mode only. If you need to use one of
these in async mode, you will need to create your own version of the
protocol or find one that is published by the REBOL community.
1.4 A Note for Beginners
If you are new to networking, it is important to know that Internet
style networking (TCP/IP) uses a streaming transfer method. Think of it
as an unending stream of bytes to and from another computer. It is the
job of higher level TCP protocols such as HTTP or FTP to determine the
sequence of actions between clients and servers.
A common question about protocols is how do you know when you have
received the full response (all the data) from the other side of the
connection (whether it be a client or a server)? This can be tricky,
because you must use either a special character sequence or provide a
transfer length indicator to know when a complete request or response
has been received. The job of detecting this is the responsibility of
the higher level protocol. If you decide to use REBOL ports to write
your own protocol, you will need to address this issue.
2. Opening an Async Port
Async port operations can now be specified as part of the REBOL open
function with the addition of the new /async refinement. This
refinement must be used in conjunction with /direct. Failing to do
so will generate an error message during the open operation.
The form of using open with the /async refinement is:
port: open/direct/binary/async spec handler
The spec is a URL or block as is standard for the open function. The
handler argument is a previously defined function that serves as the
handler for asynchronous port events. (See example below.)
3. Async Callback Handler
The handler that is passed as an argument to the /async refinement is
a callback function. It requires three arguments: the port, the action,
and an optional argument.
Here is an example of how to create a handler and pass it to the open
function:
The port argument of the function provides a way to access the port on
which the handler is acting, the action is the "event" that just
occurred, and the argument provides additional data for some actions.
A more useful port handler would look like this:
handler: func [port action arg] [
switch action [
read [append data copy/part port arg]
write-done [close port]
open [print "opened port"]
address [print ["DNS returned:" arg]]
close [close port]
]
]
port: open/direct/binary/async tcp://:9000 :handler
Here the SWITCH function dispatches the appropriate operation for the
port action that just occurred. It is suggested that you order these
actions by frequency, with the higher frequency actions at the top.
4. List of Async Actions (Events)
Here is a list of the action words that can be passed to your async
handler in the action argument. You can use these to dispatch the
appropriate processing within your function.
Action
Description
accept
Indicates that a new socket connection has been opened, and that the
port object has been cloned and is ready for I/O. Only used in listen
mode.
address
When opening a new socket connection that specifies a host name (rather
than IP address), this flags that address lookup (DNS) has been
completed. The IP address is provided as the argument.
close
Indicates that the socket connection has been closed from the other
side. Your port's side may still be open and you should call the
close function to close it. It is not a good practice to leave the
port open on your side (although the TCP stack will eventually purge
it).
error
Signals that a networking error has occurred. The argument contains the
armed error object. If you want to handle it, be sure to disarm it
first.
init
Signals that the port object has been fully initialized and is ready to
begin operation. This action is useful if, in the open, you used a
URL or a port spec block to specify your port. Once you get this event,
you can access the fields of your port.
listen
Indicates that a new socket connection is being requested. Only used in
listen mode. To accept the connection, use pick or first on the
listen port (as you would normally).
open
Indicates that the port has been opened successfully, and that data can
now be exchanged. Note: this is called "open" rather than "connected"
because async ports may eventually include files or other types of
devices.
read
Indicates that new data has been received in the port's input buffer.
Each new incoming packet will cause this event. The argument has the
number of bytes received. You can use copy/part to transfer the
bytes to your own buffer. (Or just use copy -- but note that more
bytes may have been received in the meantime, so you may get more than
what was indicated in the handler argument.) Note that it is not
critical to process the incoming data at this time. For instance, you
can ignore the read action until the port buffer has more than a given
number of bytes, then transfer all those bytes at once. However, be
sure not to let the port buffer overflow.
write
Indicates that a write operation has been started. The argument shows
the number of bytes remaining to be transmitted. This is useful for
updating progress bars or to maximize the buffering efficiency of your
program (by keeping the output buffer full while streaming). A write
operation is normally begun when an INSERT performed on the port.
write-done
Flags that a write operation has been completed and that there are no
more bytes in the buffer to be transferred. Note that this does not
mean that the data has been transferred to the receiver. The data may
still be "in-route".
The order and types of actions you receive depend on the type of the
port and what operations you are performing.
4.1 Typical Client Action Sequence
As an example, a typical client port that sends a request to a server
and receives a response back may see actions in this order:
init
port initialized
address
DNS lookup done
open
socket connected
write
send command to server
write-done
command sent (see note below)
read
reading response from server
read
reading more of response until done
close
server closed connection
Note that the read action repeats until all the data has been
transferred. In this example, the server closes the connection, causing
the close action to occur on the client.
4.2 Typical Server Action Sequence
A typical server that accepts new connections (listens), sends a welcome
message, receives a request, then sends a response is shown below:
init
server port initialized
listen
connection request from a client
accept
client connection opened
write
server sends "welcome" to client
write-done
server finished sending "welcome"
read
server gets request from client (multiple times)
write
server sends response to client
write-done
server finished sending response
close
client closed connection
The same notes apply (see Client Sequence above).
5. Optimizing Handler Actions
As an optimization you can tell REBOL not to call your handler function
on actions that you do not need to process. For example, if the only
actions you care about are READ, CLOSE, and ERROR, you can specify
that. REBOL will not call your handler for the other actions.
To specify the actions you want to receive, you can modify the
state/async-actions block in your port object. By default, all actions
will be enabled.
You can either remove the actions you don't want (the block is
automatically created on port initialization), as this does:
remove find port/state/async-actions 'write
or set it with a block that contains the list of actions you do want:
port/state/async-actions: [read close error]
6. Satisfying the Wait Function
After you have initialized one or more asynchronous ports, your code
will normally call the wait function to wait for an event to occur
(and to allow all port events to be processed). Because async ports
handle their own events the wait will not return for async port
events.
In order for a port to return from a wait you must do two things:
The port must be provided in the block that you pasted to the
wait function. See example below.
The port's wakeup? flag must be set TRUE. This can be done within
your port handler and it tells REBOL that your port handler wants
to keep the wakeup state active, forcing it to return from wait
if the above condition is set.
handler: func [port action arg] [
switch action [
...
close [
close port
port/wakeup?: true
]
]
]
port: open/direct/binary/async tcp://:9000 :handler
...other stuff...
who: wait [port]
or
who: wait/all [port port2 port3]
7. Setting Port Timeouts
The set-modes function can be used to set a timer for a specific
port. For example, in the code below:
port: open/direct/binary/async tcp://www.rebol.net:80 :handler
set-modes port [timeout: 30]
the set-modes function sets the port's timeout to 30 seconds. If
nothing happens in 30 seconds, the port will throw an error.
Note that you can also set the timeout within the various callback
phases of your async handler. For example, you might set a long timeout
for the initial TCP open, then shorten the timeout for the read and
write operations.
Of course, port timeouts accept the normal range of time values that you
would use with the wait function:
