Comments on: Load/save and decode/encode (Codecs in A42)

Here's a quick summary of changes in A42.

This is a first draft and adheres to the keep-it-simple principle. (I realize it will grow as we move forward, but let's be sure not to forget to keep it productive to all users.)

LOAD and SAVE changes

The load and save functions have been updated to call the appropriate codec (see section below) depending on the suffix. This keeps things simple.

Here is an example that converts a JPEG to a BMP:

save %photo.bmp load %photo.jpg

Here is an example to display an image:

load-gui ; (required for now, but not later)
gob: make gob! []
gob/image: load %photo.jpg
view gob

(BTW, we should extend view to accept an image! datatype directly. Easy to support. Little code.)

DECODE and ENCODE

These two mezzanine functions have been added. These functions convert between binary and REBOL datatypes.

>> ? decode
USAGE:
       DECODE type data

DESCRIPTION:
       Decodes a series of bytes into its specific media type.
       DECODE is a function value.

ARGUMENTS:
       type -- Media type (jpeg, png, etc.) (word!)
       data -- The data to decode (binary!)

and:

>> ? encode
USAGE:
       ENCODE type data /options

DESCRIPTION:
       Encodes media data type into a series of bytes.
       ENCODE is a function value.

ARGUMENTS:
       type -- Media type (jpeg, png, etc.) (word!)
       data -- The data to encode (image! binary! string!)

REFINEMENTS:
       /options -- Special encoding options

These are called by load and save, but you can also call them directly.

Examples:

image: decode 'jpeg data
data: encode 'bmp image

The special encoding options will be for variables, such as JPEG quality (once that's supported).

Codecs added to system object:

The codecs (encoders/decoders) used above are actually accessed via a general mechanism the system object.

>> ? system/catalog/codecs
SYSTEM/CATALOG/CODECS is an object of value:
   jpeg            object!   [entry name info suffixes]
   bmp             object!   [entry name info suffixes]

As you can see, two internal codecs are in this release. More will be added, and we will allow external codecs too. Also, currently, only the image! datatype is supported, but other types will be allowed.

The codec field names are not final. Also, a type field must be added.

The suffixes field is used to provide the basic mapping from a file suffix to one of these codecs. The suffixes are used to build system/catalog/file-types.

How to make comments and suggestions

I realize that this could become a hot topic with many comments. You can post them on DevBase (R3 Chat #3190) or here on this blog.

Please try to keep things concise and to the point. Provide examples where possible. We must take it a step at a time.

14 Comments

Comments:

Icarii 1-Apr-2009 2:43:37	Very nice! - simple and elegant. Well done :)
-pekr- 1-Apr-2009 3:29:52	I agree with Icarii, it is a good start, hopefully well abstracted to cover many cases. I like the aproach, because the same goes for modules. We can't probably think about any possible usage case right from the start. I am glad, that there is encode/decode functions, because I was afraid earlier we want to overload read/write functions instead. What should we keep in mind: 1) The design should allow to "chain" en(de)codings, as e.g. you might first decode text from binary format, then decode the unicode format the text is written in. 2) What happens, if there is more than one codec registered e.g. for jpg? We might have commercial codec-packs. What do we do with overlaps? Does it gets overriden? Or will there be a preference field? 3) More complicated stuff - streamed data. We want to have e.g. streamed decoding - e.g. mp3 radio. Now which part is responsible, if we want it being async? I can imagine - networking, decoder, parse all being onvolved. Or do we use usual trick to start a separate task for it? 4) Even with R2, we saw some attempts to decode some binary formats. We imo should not forget about proposed parse enhancements, as well as some set of natives for binary conversions - R2 is really bad here .... Now - take some good notes and ideas from above (if there are any :-), forget the rest ....
Goldevil 1-Apr-2009 5:38:32	It seems very elegant. Questions from a newbie : Is it possible to write my own codec in pure rebol code? I'm thinking about PDF or HTML generation from a string. How to manage data loaded by blocks to limit RAM usage ? I'm thinking of streaming of audio and video. Related to previous question : It can be handy to use codecs to extract only metadatas (Exif header, MP3 tags, Zip file catalog). In this case it can be cool to not load the complete file and just send to the codec the begining of the file. There are some "container" file formats that can contain different data encoding (AVI files). In this case the extension does not define the codec to use. As 'load cannot been used, the 'decode function imply to know the right codec. I suppose that we can define special codecs that only detects the embedded format from the header and use another codec to do the job.
Oldes 1-Apr-2009 7:45:47	Pekr, the streaming - it's very fast and easy to parse MP3 or AVI files/streams into chunks (in REBOL only), but we are missing the next part - how to decode the video chunk into a single image and or how to decode the MP3 chunk into pure audio data and a port which is able to play the audio. The next part must be in C and or Rebcode.
Oldes 1-Apr-2009 7:54:14	Goldevil... we don't need to use codecs for just extracting metadata... there is many scripts in R2 which can do that very well. Codecs should do the hard work of encoding/decoding data chunks which you provide.
Carl Sassenrath 1-Apr-2009 12:48:09	In reply to Pekr's questions: 1) Chaining: yes, that works. You can call one codec get results, call the next etc. In fact, you could create a new codec in REBOL code combines the two (or more) actions. 2) Multi-codecs for same type: currently, if you add a second codec for the same media type, it replaces the other one. I did this so, for example, you can replace the internal JPEG codec with one that does both encode and decode. Then it's a simple change, and you get the new features. But, it is possible to have alternative codecs, as long as you use different names. For example, MPEG-2 and MPEG-3 might be example names. 3) Streaming codecs: Normally streaming codecs deal with data in chunks, as Oldes points out (above). So, I think it can be done, but there is an additional requirement: state variables. Complex streams are often not stateless, so the state must be held between calls to the codec. This needs to be wrapped properly to allow the codec to be reentrant for multiple streams, etc. But, I think we can manage that by allowing the codec result to be a block, like we do for load/next. BTW, this method is needed for more than just streaming encodings, it's also useful for codecs like GIF that can contain multiple images. Also, there are times when you want to begin decoding without the full data... progressive modes, etc. 4) PARSE and BINARY! improvements: I agree. BrianH has made a good list and posted it, so let's be sure he knows of any other things we need on it.
Carl Sassenrath 1-Apr-2009 13:01:20	In reply to Goldevil's questions: 1) Can a codec be written in REBOL? A good question. Yes, that's why I made the top level a mezzanine that accesses the system object. It allows codecs to be written in REBOL code. 2) Managing memory: I've been thinking about the memory issue for a while. It's kind of tricky because REBOL uses it's own memory manager for performance reasons. Right now, there is extra overhead in the codecs to manage this issue. However, I think we can provide a method of sharing memory to reduce the overhead. This would require an additional entry point into the codec (for freeing resources), but it's worth it. 3) Metadata specified codecs: Yes, it is true that some suffixes can map to different codecs. There are various methods, both using REBOL code and native code to handle it. That's also a good example why DECODE is a mezzanine function - it gives us control over those situations.
Carl Sassenrath 1-Apr-2009 13:05	Oldes, yes, I agree with your comments, and I look forward to seeing some good things happen with codecs. (In fact, I think developers will be using them for more than just encode and decode. ;)
Brian Hawley 1-Apr-2009 14:57:57	I would also like to see codec port schemes, which might be able to handle the streaming and filter codecs. These could be port schemes that wrap around streaming codecs, not codecs implemented as port schemes. These are all higher-level code...
Brian Hawley 1-Apr-2009 15:16:45	As for the parse proposals, I was avoiding putting anything other than new operations on it since that was the original mandate. I have some ideas for binary and port parsing - should I add them to the proposals page?
popper 1-Apr-2009 16:40:47	regarding the so called "codec hell" thats become apparent in the windows domain, or as benwaggoner silverlight lead,said "I haven't talked with the Windows team about this issue in particular, but I know that they're tracking a lot of problems users have by the "codec arms race" where codecs rate themselves much higher than they technically merit, and playback experience gets fragile or outright dysfunctional. The people here know how to debug those issues, but I'm sure you've all had to do a fair amount of codec pack fixing as well...." "Quote: Originally Posted by clsid The real solution would be to include an application in which the user can select his preferred source filters and decoders. Similar to the "Default Programs" functionality of Windows, where users can configure their preferred software and file associations. Such app could default to Microsoft's own filters and label them as recommended or something. They could even add some filter management functionality for third party stuff. Or warn a user about "bad" installed filters. The possibilities are endless. " "You are just describing GraphEdit with profiles. How is that not what I said? " ----------- the consenseus seems to be some form of "GraphEdit with profiles", it seems to me a fair expandable way to do this with the old amigaOS twists regarding multiple *filters* and decoders at its core OC. incase your not aware "GraphEdit" basicly allows you to link inputs and outputs for a given content video,container etc and override the defaults as you see fit, we could make it far better OC, amiga device style....
popper 1-Apr-2009 16:42:27	sorry, i forgot the URL and codec thread these quotes came from, read it its interesting and related to this R3 subject OC http://forum.doom9.org/showthread.php?t=145906&page=2
Oldes 1-Apr-2009 17:00:22	As for binary parsing, I'm not using parse itself but rather a set of functions closed in a context like in this MP3 stream parser.
Chris 2-Feb-2013 14:55	What is the best way to proceed in implementing codecs? As an exercise (useful at that), I'd like to convert my Json Parser to R3 as a codec allowing the loading/saving to this common format.

Post a Comment:

You can post a comment here. Keep it on-topic.