Disclaimer
----------
This is not production code, so it's not clean, it's not something you can use
in medical, aeronautical, nuclear, or anything-important environment.
Use this code, advices and whatever at your own risk, and, no I don't work for
Sun so I cannot explain everything or fix any problem. ;)
Anyway, if you want to contact me, well feel free to send an email to:
Author's name: Paolo Di Francesco
Author's email: paipai<AT>tin.it (<AT> means @)
Introduction
------------
What is a codec? Well if you don't know what it is, maybe this code is not for
you! If you know what a codec is, and what you want to code, then well, write
it in JMF! It's fun and if you know where to put the hands, you can put together
something working in a very short time. If you don't follow the advices/skeleton
in this tutorial, then it will be hard to write something nice and working ;)
Thanks to...
------------
First of all thanks to Sun who released JMF. JMF is far from being a production
API (current date October 2003), but it can help you to solve some research
problems. At the current date, also consider that Sun released the source code,
so PLEASE help to fix the bugs and do some improvements.
Second, thanks to Amith!!! Without your precious advices I could be still trying
to understand why the bits of the code are not yet coming together ;)
Third, thanks to www.jmfapi.org. We need
more code, forum and what else to
improve JMF!
Downloading source code
------------------------------------
All the source code is contained in a single zip file here.
Codec=Encoder+Decoder
---------------------
Ok, so if you are reading here, it means you are crazy enough to go to the
battle... Good... ;)
Now you know that you need an Encoder to compress your data, and a Decoder to
uncompress them. This applies to audio/video Codecs. Anyway I am interested only
in Video codecs (I like video stuff!) so all the following things apply only to
video.
From JMF point of view an encoder/decoder is just a Codec or PlugIn interface.
So, start taking this into account and write the skeleton as shown in the
example code. Read also the JMF java docs and do not consider most of the things
you could read on the Sun pdf doc as the final statement: they are mostly
outdated, or they could be. This code should be enough commented to understand
how the skeleton should be made.
Now, I wrote a simple, very simple example to show how to realize a coder.
So follow me in my crazy battle plan and be patient!
Consider also that most people is interested in transmitting data via a network
connection. This example is really simple, so this is NOT possible for 2
reasons:
- the coder does not include the mechanism/classes to pack data and transmit
them over ip
- the coder would be much more complex. In fact consider that you do not loose
data when you fetch them from the Hard Disk! So, if some bits are missing how
should the codec re-construct the data? In a network environment there are many
reasons why data can be got lost ;)
Encoding/Decoding process
-------------------------
Ok, now I will explain in few words what I wanted to realize. The "how" I
realized that, well read the code, I hope it's well commented and
self-explaining. Unfortunately I don't have enough time to explain all the trick
I learned during it's development. :/
The idea is simple. Video is just a stream of bytes. The same is valid for
audio. Everything is "bytes", we live in a digital era! ;)
But what does it means from the JMF point of view? Well, let's consider the RGB
coding. Each frame of a movie is just an image. So if we code it into RGB it
means it's easier.
Also consider that JMF tries to construct a sort of graph to go from
here-to-there, so even if the original coding was mpeg, and you want to save it
with 'my-codec' and you support from RGB to 'my-codec', well JMF will try to
find a path (i.e. a codec combination) to "go" from mpeg to 'my-codec'. In most
cases it should be like this: mpeg->RGB->'my codec'. Ok, not so efficient, but
do you prefer to write your codec starting with simple RGB coding or from mpeg?
Now, let's go back to the codec. As I was saying we have the RGB coding. Right?
Ok. When I say RGB, I mean RGB24, i.e. we have 24 bits for the single pixel.
I hope it's clear to everybody what a pixel is and how an image is composed by
pixels, otherwise google will help you.. :)
So, we have 24 bits organized, and packed as bytes, and the rgb image is just a
huge amount of bytes. Now think to the pixel: we said 24 bits, so simply 3
bytes (1 for the red/green/blue component, that's why they call it RGB!) for
each pixel. It does not matter how the image is really organized (e.g. what is
the first pixel, top-left?), and the reason is simple: we will not take
advantage of this info! The codec is really stupid, so we will take into account
for our compression the simple fact that the image is represented as an array
of bytes (the input array). So, it's like going from 2D to 1D, but it's just a
matter of efficiency and also consider that in most cases, unless you do not
compress the stream, you cannot have a one-to-one correspondence with the single
pixel, so using 2D arrays would be useless. Now that we have the RGB image under
the form of an (HUGE!) array we have to compress it.
How? The algorithm I uses it simple: if you have a repetition of 3 or more
bytes,
represent the values as N times the value.
Example. A sequence of bytes like this:
10-10-10-10
can be easily represented as 4 times 10. Now the problem: how do I know when
I am representing a "good value" or when I am saying "N times this value"?
Hum.. ok let's put out of the set of values a "particular" value that need to be
represented in a special way. Let's say that if I read 0 from the stream then I
am saying "hey next one is a compressed sequence!". So, considering I need one
special byte for the compression, one for the N value (just representing no more
than 255 values) and another one for the real value, I need 3 bytes. So
compression would be useless for sequence less than 3 consecutive bytes, that's
why the name RLE3.
And for the "black" values? I mean when I have a component equal to zero? Well
that's simple and that's why the 0 has been chosen ;)
The second byte always tells how many consecutive values are represented, right?
It's meaningless to say you have N=0, and for sure N>3 because we said we do not
apply the algorithm when N<2 (unefficient!)
So, here we are! We can use two 0 to represent a single (or less than 3) 0
component. Another side effect is that usually single 0 ("black" or pure
components) are not so usually in images. (hum.. well it depends on the
images..)
So we expect that the bad cases will be very low.
To summaries, this is a loose-less example of compression. After the
encoding-decoding phase you will have always (if everything is working!) the
same amount of information, nothing has been lost. For better compression,
better strategies can be applied, but this is not the right document to read
how to compress video or images. (Just a simple idea, take the RGB under the
form or a byte array and zip it. Java has a zip-like class, isn't it?)
Now just few example to describe the compression strategy.
Example 1 (encoding process)
---------
input stream is: 12-33-128-76-9-9-9-9-10
output stream is: 12-33-128-76-0-4-9-10
Example 2 (encoding some 0s)
---------
input stream is: 22-233-77-0-0-0-0-0-0-39
output stream is: 22-233-77-0-6-0-39
Example 3 (encoding one single 0)
---------
input stream is: 22-233-77-0-39
output stream is: 22-233-77-0-0-39
Realization notes
-----------------
Ok, I realized my new code, everything compile, how do I test it? Well, install
it! How? I think somewhere on the Sun site it's explained how to install a new
plug in, or you can read something on the IBM mpeg4 site. Anyway consider a good
idea, to uninstall it and reinstall when you change something in the code.
(Just to be sure...)
Second tip: JMF is not so smart regarding the errors coming from codecs. As you
probably can see, not much is printed when you install a new codec. My
suggestion is to do this: put some System.out here and there, and call the
methods by hand using some foo-parameters.
In few words: try to emulate the JMF API if you have some dubts! So, you know
there is a specific sequence when the codec is called (see the JMF java docs!).
So try to do call them by hand and remember that it's not exaclty the same
because some parameters could be null when you artificially call them by hand,
but not when JMF does and vice versa!
Last thing to note: after you install your codec in the JMF api you can use it
in any JMF application! So you can use it directly from JMF or anything else.
There is a nice command line utility (Transcode) that you can download from the
Sun site to do your first experiment. It's much more useful than JMStudio
because you must open the shell (or the dos-like window under Windows) and you
can see the printed output there or modify as you like the original one.
HAVE FUN!!!!
About the Author (October 2003)
-------------------------------
I mostly spend my time doing research in IP networks (routing, QoS, etc.),
writing code (kernel, Java apps, etc), and trying to put in practice new ideas
about data transmission and new services on Internet.
Unfortunately, as most of the people on this planet, I am not rich and I have to
work to survive. So, I wrote this code and the document in my spare time. If you
have comments, I will love to hear them, but as most of "spare-time" projects do
not consider the code as a distilled version of black magic. It can be improved
and enhanced only with your help!
You can contact me (my personal email address) at:
Author's name: Paolo Di Francesco
Author's email: paipai<AT>tin.it (<AT> means @)
Copyright
---------
You can redistribute, print and do-almost-anything-you-want with this document
and code. BUT you HAVE not to cut&paste parts of it, and you must keep all the
document "as is" without modifying it. The use of the code is granted for non
commercial purposes. You can use the code (at your own risk) in your
experiments, but you cannot use it in commercial applications without the
explicit author's permission.
If you have any comments that you would like to add please email us.