This commit was generated by cvs2svn to compensate for changes in r26952,
which included commits to RCS files with non-trunk default branches. git-svn-id: https://svn.wxwidgets.org/svn/wx/wxWidgets/trunk@26953 c3d73ce0-8a6f-49c7-b76d-6d57e0e08775
This commit is contained in:
@@ -59,10 +59,10 @@ but saves time since there are both fewer insertions and fewer searches.
|
||||
|
||||
2.1 Introduction
|
||||
|
||||
The real question is, given a Huffman tree, how to decode fast. The most
|
||||
important realization is that shorter codes are much more common than
|
||||
longer codes, so pay attention to decoding the short codes fast, and let
|
||||
the long codes take longer to decode.
|
||||
The key question is how to represent a Huffman code (or any prefix code) so
|
||||
that you can decode fast. The most important characteristic is that shorter
|
||||
codes are much more common than longer codes, so pay attention to decoding the
|
||||
short codes fast, and let the long codes take longer to decode.
|
||||
|
||||
inflate() sets up a first level table that covers some number of bits of
|
||||
input less than the length of longest code. It gets that many bits from the
|
||||
@@ -77,58 +77,54 @@ table took no time (and if you had infinite memory), then there would only
|
||||
be a first level table to cover all the way to the longest code. However,
|
||||
building the table ends up taking a lot longer for more bits since short
|
||||
codes are replicated many times in such a table. What inflate() does is
|
||||
simply to make the number of bits in the first table a variable, and set it
|
||||
for the maximum speed.
|
||||
simply to make the number of bits in the first table a variable, and then
|
||||
to set that variable for the maximum speed.
|
||||
|
||||
inflate() sends new trees relatively often, so it is possibly set for a
|
||||
smaller first level table than an application that has only one tree for
|
||||
all the data. For inflate, which has 286 possible codes for the
|
||||
literal/length tree, the size of the first table is nine bits. Also the
|
||||
distance trees have 30 possible values, and the size of the first table is
|
||||
six bits. Note that for each of those cases, the table ended up one bit
|
||||
longer than the ``average'' code length, i.e. the code length of an
|
||||
approximately flat code which would be a little more than eight bits for
|
||||
286 symbols and a little less than five bits for 30 symbols. It would be
|
||||
interesting to see if optimizing the first level table for other
|
||||
applications gave values within a bit or two of the flat code size.
|
||||
For inflate, which has 286 possible codes for the literal/length tree, the size
|
||||
of the first table is nine bits. Also the distance trees have 30 possible
|
||||
values, and the size of the first table is six bits. Note that for each of
|
||||
those cases, the table ended up one bit longer than the ``average'' code
|
||||
length, i.e. the code length of an approximately flat code which would be a
|
||||
little more than eight bits for 286 symbols and a little less than five bits
|
||||
for 30 symbols.
|
||||
|
||||
|
||||
2.2 More details on the inflate table lookup
|
||||
|
||||
Ok, you want to know what this cleverly obfuscated inflate tree actually
|
||||
looks like. You are correct that it's not a Huffman tree. It is simply a
|
||||
lookup table for the first, let's say, nine bits of a Huffman symbol. The
|
||||
symbol could be as short as one bit or as long as 15 bits. If a particular
|
||||
Ok, you want to know what this cleverly obfuscated inflate tree actually
|
||||
looks like. You are correct that it's not a Huffman tree. It is simply a
|
||||
lookup table for the first, let's say, nine bits of a Huffman symbol. The
|
||||
symbol could be as short as one bit or as long as 15 bits. If a particular
|
||||
symbol is shorter than nine bits, then that symbol's translation is duplicated
|
||||
in all those entries that start with that symbol's bits. For example, if the
|
||||
symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a
|
||||
in all those entries that start with that symbol's bits. For example, if the
|
||||
symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a
|
||||
symbol is nine bits long, it appears in the table once.
|
||||
|
||||
If the symbol is longer than nine bits, then that entry in the table points
|
||||
to another similar table for the remaining bits. Again, there are duplicated
|
||||
If the symbol is longer than nine bits, then that entry in the table points
|
||||
to another similar table for the remaining bits. Again, there are duplicated
|
||||
entries as needed. The idea is that most of the time the symbol will be short
|
||||
and there will only be one table look up. (That's whole idea behind data
|
||||
compression in the first place.) For the less frequent long symbols, there
|
||||
will be two lookups. If you had a compression method with really long
|
||||
symbols, you could have as many levels of lookups as is efficient. For
|
||||
and there will only be one table look up. (That's whole idea behind data
|
||||
compression in the first place.) For the less frequent long symbols, there
|
||||
will be two lookups. If you had a compression method with really long
|
||||
symbols, you could have as many levels of lookups as is efficient. For
|
||||
inflate, two is enough.
|
||||
|
||||
So a table entry either points to another table (in which case nine bits in
|
||||
the above example are gobbled), or it contains the translation for the symbol
|
||||
and the number of bits to gobble. Then you start again with the next
|
||||
So a table entry either points to another table (in which case nine bits in
|
||||
the above example are gobbled), or it contains the translation for the symbol
|
||||
and the number of bits to gobble. Then you start again with the next
|
||||
ungobbled bit.
|
||||
|
||||
You may wonder: why not just have one lookup table for how ever many bits the
|
||||
longest symbol is? The reason is that if you do that, you end up spending
|
||||
more time filling in duplicate symbol entries than you do actually decoding.
|
||||
At least for deflate's output that generates new trees every several 10's of
|
||||
kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code
|
||||
would take too long if you're only decoding several thousand symbols. At the
|
||||
You may wonder: why not just have one lookup table for how ever many bits the
|
||||
longest symbol is? The reason is that if you do that, you end up spending
|
||||
more time filling in duplicate symbol entries than you do actually decoding.
|
||||
At least for deflate's output that generates new trees every several 10's of
|
||||
kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code
|
||||
would take too long if you're only decoding several thousand symbols. At the
|
||||
other extreme, you could make a new table for every bit in the code. In fact,
|
||||
that's essentially a Huffman tree. But then you spend two much time
|
||||
that's essentially a Huffman tree. But then you spend two much time
|
||||
traversing the tree while decoding, even for short symbols.
|
||||
|
||||
So the number of bits for the first lookup table is a trade of the time to
|
||||
So the number of bits for the first lookup table is a trade of the time to
|
||||
fill out the table vs. the time spent looking at the second level and above of
|
||||
the table.
|
||||
|
||||
@@ -158,7 +154,7 @@ Let's make the first table three bits long (eight entries):
|
||||
110: -> table X (gobble 3 bits)
|
||||
111: -> table Y (gobble 3 bits)
|
||||
|
||||
Each entry is what the bits decode to and how many bits that is, i.e. how
|
||||
Each entry is what the bits decode as and how many bits that is, i.e. how
|
||||
many bits to gobble. Or the entry points to another table, with the number of
|
||||
bits to gobble implicit in the size of the table.
|
||||
|
||||
@@ -170,7 +166,7 @@ long:
|
||||
10: D,2
|
||||
11: E,2
|
||||
|
||||
Table Y is three bits long since the longest code starting with 111 is six
|
||||
Table Y is three bits long since the longest code starting with 111 is six
|
||||
bits long:
|
||||
|
||||
000: F,2
|
||||
@@ -182,20 +178,20 @@ bits long:
|
||||
110: I,3
|
||||
111: J,3
|
||||
|
||||
So what we have here are three tables with a total of 20 entries that had to
|
||||
be constructed. That's compared to 64 entries for a single table. Or
|
||||
compared to 16 entries for a Huffman tree (six two entry tables and one four
|
||||
entry table). Assuming that the code ideally represents the probability of
|
||||
So what we have here are three tables with a total of 20 entries that had to
|
||||
be constructed. That's compared to 64 entries for a single table. Or
|
||||
compared to 16 entries for a Huffman tree (six two entry tables and one four
|
||||
entry table). Assuming that the code ideally represents the probability of
|
||||
the symbols, it takes on the average 1.25 lookups per symbol. That's compared
|
||||
to one lookup for the single table, or 1.66 lookups per symbol for the
|
||||
to one lookup for the single table, or 1.66 lookups per symbol for the
|
||||
Huffman tree.
|
||||
|
||||
There, I think that gives you a picture of what's going on. For inflate, the
|
||||
meaning of a particular symbol is often more than just a letter. It can be a
|
||||
byte (a "literal"), or it can be either a length or a distance which
|
||||
indicates a base value and a number of bits to fetch after the code that is
|
||||
added to the base value. Or it might be the special end-of-block code. The
|
||||
data structures created in inftrees.c try to encode all that information
|
||||
There, I think that gives you a picture of what's going on. For inflate, the
|
||||
meaning of a particular symbol is often more than just a letter. It can be a
|
||||
byte (a "literal"), or it can be either a length or a distance which
|
||||
indicates a base value and a number of bits to fetch after the code that is
|
||||
added to the base value. Or it might be the special end-of-block code. The
|
||||
data structures created in inftrees.c try to encode all that information
|
||||
compactly in the tables.
|
||||
|
||||
|
||||
@@ -210,4 +206,4 @@ Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3,
|
||||
pp. 337-343.
|
||||
|
||||
``DEFLATE Compressed Data Format Specification'' available in
|
||||
ftp://ds.internic.net/rfc/rfc1951.txt
|
||||
http://www.ietf.org/rfc/rfc1951.txt
|
||||
|
Reference in New Issue
Block a user