2. The method as defined in claim 1 wherein the at least one starting leaf node is a root node of the tree structure and the set of colors in the root node includes all M distinct colors.
3. The method as defined in claim 1 wherein the at least one starting leaf node comprises a plurality of starting leaf nodes containing a plurality of disjoint subsets of the M distinct colors, wherein each disjoint subset of colors is included in one and only one starting leaf node in the plurality of starting nodes.
4. The method as defined in claim 1 wherein step c) further comprises determining at least one of A) an associated increased entropy rate resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes by allocating each color in n to one of the two new leaf nodes and, B) a new total number of leaf nodes resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes; if the associated increased entropy rate is determined, then repeating step c) if the associated increased entropy rate is less than a target entropy rate; and if the new total number of leaf nodes is determined, then repeating step c) if the new total number of leaf nodes is less than a target number of leaf nodes.
5. The method as defined in claim 1 wherein the step of determining a representative color for a leaf node comprises for each color in the leaf node, determining a number of pixels in the associated subset of pixels having that color in the digitized color image; and, determining a centroid of all of the colors in the leaf node based on all of the colors in the leaf node and the number of pixels in the subset of pixels associated with each of the colors in the leaf node.
6. The method as defined in claim 1 wherein step (c)(ii) comprises allocating each color in n to one of the two new leaf nodes by determining, for each of the two new leaf nodes, an associated biased distortion measure involving both rate and distortion for allocating the color to that new leaf node, and allocating the color in n to the new leaf node having a lower associated biased distortion measure.
7. The method as defined in claim 6 wherein, for each of the two new leaf nodes, the associated biased distortion measure is determined by determining an associated change in distortion resulting from allocating the color to the new leaf node; determining an associated increase in entropy rate resulting from allocating the color to the new leaf node; and, calculating the associated biased distortion measure based on the associated change in distortion and the associated increase in entropy rate.
8. The method as defined in claim 1 further comprising initializing a lower resolution image based on the at least one representative color for each starting node; where step (c)(iv) further comprises determining information bits for a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes according to allocation of colors in n to one of the two new leaf nodes in step (c)(ii); and encoding the information bits based on the lower resolution image by using arithmetic coding.
9. The method as defined in claim 1 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost for turning the leaf node into a non-leaf node linked to two new leaf nodes; and, selecting the leaf node based on comparing the associated achievable cost for that leaf node with the associated achievable cost for at least one other leaf node.
10. The method as defined in claim 9 wherein step (c)(i) comprises, for each leaf node, determining if the associated achievable cost for the selected leaf node is lower than the associated achievable cost for each of the other leaf nodes
11. The method as defined in claim 10 wherein step (c)(i) comprises, for each leaf node, determining the asssociated achievable cost based on adding the associated change in distortion to a product obtained by multiplying the associated increase in the entropy rate by a multiplier.
12. The method as defined in claim 11 where the multiplier is a fixed Lagrangian multiplier.
13. The method as defined in claim 10 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost by determining an associated achievable tradeoff between the associated increase in entropy rate and the associated reduction in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, determining the associated achievable cost based on the associated achievable tradeoff, the associated increase in entropy rate and the associated reduction in distortion.
14. The method as defined in claim 7 wherein, for each of the two new leaf nodes, the step of determining an associated increase in entropy rate resulting from allocating that color to the new leaf node comprises, determining, for each pixel in the subset of pixels for that color, a set of neighboring pixels, wherein each pixel in the set of neighboring pixels borders the pixel; determining an entropy rate increase for the pixel based on the set of neighboring pixels; and determining an associated increase in entropy rate resulting from allocating that color to the new leaf node based on entropy rate increases of all pixels in the subset of pixels for that color.
15. A computer program product for use on a computer to progressively encode a digitized color image with M distinct colors allocated to a set of pixels, the digitized color image being provided by assigning each of the M distinct colors to an associated subset of pixels in the set of pixels, the computer program product comprising: a recording medium; and, means recorded on the recording medium for instructing the computer system to perform the steps of: (a) initializing a tree structure by providing at least one starting leaf node comprising a set of colors from the M distinct colors; (b) determining at least one representative color for each starting leaf node; and (c) growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is determined by determining an associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; determining an associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, determining the associated achievable cost based on the associated change in distortion and the associated increase in entropy rate; (ii) creating the two new leaf nodes by allocating each color in n to one of the two new leaf nodes; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.
16. The computer program product as defined in claim 15 wherein the at least one starting leaf node is a root node of the tree structure and the set of colors in the root node includes all M distinct colors.
17. The computer program product as defined in claim 15 wherein the at least one starting leaf node comprises a plurality of starting leaf nodes containing a plurality of disjoint subsets of the M distinct colors, wherein each disjoint subset of colors is included in one and only one starting leaf node in the plurality of starting nodes.
18. The computer program product as defined in claim 15 wherein step c) further comprises determining at least one of A) an associated increased entropy rate resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes by allocating each color in n to one of the two new leaf nodes and, B) a new total number of leaf nodes resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes; if the associated increased entropy rate is determined, then repeating step c) if the associated increased entropy rate is less than a target entropy rate; and if the new total number of leaf nodes is determined, then repeating step c) if the new total number of leaf nodes is less than a target number of leaf nodes.
19. The computer program product as defined in claim 15 wherein the step of determining a representative color for a leaf node comprises for each color in the leaf node, determining a number of pixels in the associated subset of pixels having that color in the digitized color image; and, determining a centroid of all of the colors in the leaf node based on all of the colors in the leaf node and the number of pixels in the subset of pixels associated with each of the colors in the leaf node.
20. The computer program product as defined in claim 15 wherein step (c)(ii) comprises allocating each color in n to one of the two new leaf nodes by determining, for each of the two new leaf nodes, an associated biased distortion measure involving both rate and distortion for allocating the color to that new leaf node, and allocating the color in n to the new leaf node having a lower associated biased distortion measure.
21. The computer program product as defined in claim 20 wherein, for each of the two new leaf nodes, the associated biased distortion measure is determined by determining an associated change in distortion resulting from allocating the color to the new leaf node; determining an associated increase in entropy rate resulting from allocating the color to the new leaf node; and, calculating the associated biased distortion measure based on the associated change in distortion and the associated increase in entropy rate.
22. The computer program product as defined in claim 15 wherein the instructions recorded on the recording medium are further operable to instruct the computer system to initialize a lower resolution image based on the at least one representative color for each starting node; and step (c)(iv) further comprises determining information bits for a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes according to allocation of colors in n to one of the two new leaf nodes in step (c)(ii); and encoding the information bits based on the lower resolution image by using arithmetic coding.
23. The computer program product as defined in claim 15 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost for turning the leaf node into a non-leaf node linked to two new leaf nodes; and, selecting the leaf node based on comparing the associated achievable cost for that leaf node with the associated achievable cost for at least one other leaf node.
24. The computer program product as defined in claim 23 wherein step (c)(i) comprises, for each leaf node, determining if the associated achievable cost for the selected leaf node is lower than the associated achievable cost for each of the other leaf nodes
25. The computer program product as defined in claim 24 wherein step (c)(i) comprises, for each leaf node, determining the asssociated achievable cost based on adding the associated change in distortion to a product obtained by multiplying the associated increase in the entropy rate by a multiplier.
26. The computer program product as defined in claim 25 where the multiplier is a fixed Lagrangian multiplier.
27. The computer program product as defined in claim 24 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost by determining an associated achievable tradeoff between the associated increase in entropy rate and the associated reduction in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, determining the associated achievable cost based on the associated achievable tradeoff, the associated increase in entropy rate and the associated reduction in distortion.
28. The computer program product as defined in claim 21 wherein, for each of the two new leaf nodes, the step of determining an associated increase in entropy rate resulting from allocating that color to the new leaf node comprises, determining, for each pixel in the subset of pixels for that color, a set of neighboring pixels, wherein each pixel in the set of neighboring pixels borders the pixel; determining an entropy rate increase for the pixel based on the set of neighboring pixels; and determining an associated increase in entropy rate resulting from allocating that color to the new leaf node based on entropy rate increases of all pixels in the subset of pixels for that color.
29. A system for progressively encoding a digitized color image with M distinct colors allocated to a set of pixels, the digitized color image being provided by assigning each of the M distinct colors to an associated subset of pixels in the set of pixels, the system comprising: a recording medium; and, means including a processor for performing the steps of: (a) initializing a tree structure by providing at least one starting leaf node comprising a set of colors from the M distinct colors; (b) determining at least one representative color for each starting leaf node; and (c) growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is determined by determining an associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; determining an associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, determining the associated achievable cost based on the associated change in distortion and the associated increase in entropy rate; (ii) creating the two new leaf nodes by allocating each color in n to one of the two new leaf nodes; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.
30. The system as defined in claim 29 wherein the at least one starting leaf node is a root node of the tree structure and the set of colors in the root node includes all M distinct colors.
31. The system as defined in claim 29 wherein the at least one starting leaf node comprises a plurality of starting leaf nodes containing a plurality of disjoint subsets of the M distinct colors, wherein each disjoint subset of colors is included in one and only one starting leaf node in the plurality of starting nodes.
32. The system as defined in claim 29 wherein step c) further comprises determining at least one of A) an associated increased entropy rate resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes by allocating each color in n to one of the two new leaf nodes and, B) a new total number of leaf nodes resulting from turning the leaf node n into the non-leaf node n linked to the two new leaf nodes; if the associated increased entropy rate is determined, then repeating step c) if the associated increased entropy rate is less than a target entropy rate; and if the new total number of leaf nodes is determined, then repeating step c) if the new total number of leaf nodes is less than a target number of leaf nodes.
33. The system as defined in claim 29 wherein the step of determining a representative color for a leaf node comprises for each color in the leaf node, determining a number of pixels in the associated subset of pixels having that color in the digitized color image; and, determining a centroid of all of the colors in the leaf node based on all of the colors in the leaf node and the number of pixels in the subset of pixels associated with each of the colors in the leaf node.
34. The system as defined in claim 29 wherein step (c)(ii) comprises allocating each color in n to one of the two new leaf nodes by determining, for each of the two new leaf nodes, an associated biased distortion measure involving both rate and distortion for allocating the color to that new leaf node, and allocating the color in n to the new leaf node having a lower associated biased distortion measure.
35. The system as defined in claim 34 wherein, for each of the two new leaf nodes, the associated biased distortion measure is determined by determining an associated change in distortion resulting from allocating the color to the new leaf node; determining an associated increase in entropy rate resulting from allocating the color to the new leaf node; and, calculating the associated biased distortion measure based on the associated change in distortion and the associated increase in entropy rate.
36. The system as defined in claim 29 wherein the steps performed further comprise initializing a lower resolution image based on the at least one representative color for each starting node; and, step (c)(iv) further comprises determining information bits for a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes according to allocation of colors in n to one of the two new leaf nodes in step (c)(ii); and encoding the information bits based on the lower resolution image by using arithmetic coding.
37. The system as defined in claim 29 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost for turning the leaf node into a non-leaf node linked to two new leaf nodes; and, selecting the leaf node based on comparing the associated achievable cost for that leaf node with the associated achievable cost for at least one other leaf node.
38. The system as defined in claim 37 wherein step (c)(i) comprises, for each leaf node, determining if the associated achievable cost for the selected leaf node is lower than the associated achievable cost for each of the other leaf nodes
39. The system as defined in claim 38 wherein step (c)(i) comprises, for each leaf node, determining the asssociated achievable cost based on adding the associated change in distortion to a product obtained by multiplying the associated increase in the entropy rate by a multiplier.
40. The system as defined in claim 39 where the multiplier is a fixed Lagrangian multiplier.
41. The system as defined in claim 38 wherein step (c)(i) comprises, for each leaf node, determining the associated achievable cost by determining an associated achievable tradeoff between the associated increase in entropy rate and the associated reduction in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, determining the associated achievable cost based on the associated achievable tradeoff, the associated increase in entropy rate and the associated reduction in distortion.
42. The system as defined in claim 35 wherein, for each of the two new leaf nodes, the step of determining an associated increase in entropy rate resulting from allocating that color to the new leaf node comprises, determining, for each pixel in the subset of pixels for that color, a set of neighboring pixels, wherein each pixel in the set of neighboring pixels borders the pixel; determining an entropy rate increase for the pixel based on the set of neighboring pixels; and determining an associated increase in entropy rate resulting from allocating that color to the new leaf node based on entropy rate increases of all pixels in the subset of pixels for that color.
43. The system as defined in claim 28 wherein the system is a mobile wireless device.
44. The method as defined in claim 1 further comprising initializing an encoded image based on the at least one representative color for each starting node; wherein step (c) further comprises progressively updating the encoded image by appending the encoded bits obtained in step c)iv).
45. The computer program product as defined in claim 15 wherein the instructions recorded on the recording medium are further operable to instruct the computer system to initialize an encoded image based on the at least one representative color for each starting node; wherein step (c) further comprises progressively updating the encoded image by appending the encoded bits obtained in step c)iv).
46. The system as defined in claim 29 wherein the steps performed further comprise initializing an encoded image based on the at least one representative color for each starting node; wherein step (c) further comprises progressively updating the encoded image by appending the encoded bits obtained in step c)iv).
The present invention relates to progressive encoding of image data. The application claims the benefit of U.S. Provisional Application Ser. No. 60/719,585, filed Sep. 23, 2005, the entire contents of which is hereby incorporated by reference.
FIELD OF THE INVENTION
Background of the Invention
Pictures with thousands or millions colors can be represented as true color images, storing the exact color for each pixel in the formats such as BMP (bitmap) and JPEG. In contrast, pictures containing only a few (typically up to 256) colors can be stored more efficiently as colormapped images or palettized images like the well known GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format. A significant portion of web images are in the format of GIF or PNG.
When browsing palettized images over low-bandwidth communication links, such as wireless links or dial-up links, progressive transmission of these palette images can support fast browsing since it enables the users to view complete versions of these images (but at a low resolution) before all progressively coded image data has been received. However, even though both GIF and PNG support an interlaced format, which refines the resolution of an image progressively by encoding the image in multiple passes and skipping a number of pixels in each pass, interlaced GIF and PNG are not in a progressive format in the strict sense that the color information of each pixel is refined progressively up to lossless recovery.
Rauschenbach investigated the progressive refinement of color information for palette images in his so-called MCQ (Mobile visualization Color Quantized format) method (U. Rauschenbach, “Compression of palettized images with progressive coding of the color information,” in Proc. SPIE Visual
Communications and Image Processing (VCIP2000), Perth, Australia, June 2000, hereinafter referred to as [1]). Unlike the interlaced GIF and PNG formats, the refinement of color information in MCQ is done bit-plane by bit-plane, and allows the early recognition of shapes with fine details (e. g. text), normally after decoding a small portion of the compressed bit stream. Although MCQ supports progressive refinement of color information in contrast to resolution refinement, it only achieves a compression ratio comparable to GIF or PNG format. To improve the compression ratio while supporting the progressive transmission of palette images, Chen et al. (X. Chen, S. Kwong, and J. Feng, “A new compression scheme for color-quantized images,” IEEE Transactions on Circuits and System for Video Technology, Vol. 12, No. 10, pp. 904-908, October 2002, hereinafter referred to as [2]) proposed a new color refinement scheme based on MCQ and the distortion-based hierarchical splitting color quantization algorithm pioneered by Orchard and Bouman (M. Orchard and C. Bouman, “Color quantization of images,” IEEE Transactions on Signal Processing, Vol. 39, No. 12, pp. 2677˜2690, December 1991, hereinafter referred to as [3]). Unlike MCQ where the color information is refined bit-plane by bit-plane with the number of bit-planes equal to the color depth of the palette, Chen's algorithm refines the color information color by color. For lossless compression of palette images, the color splitting algorithm proposed in [2] achieves around 30˜40% size reduction compared to GIF or PNG format in addition to its decent feature of supporting the progressive transmission.
SUMMARY
In accordance with an aspect of an embodiment of the invention, there is provided a method for progressively encoding a digitized color image with M distinct colors allocated to a set of pixels using a data processing system, the digitized color image being provided by assigning each of the M distinct colors to an associated subset of pixels in the set of pixels. The method comprises: (a) initializing a tree structure by providing at least one starting leaf node comprising a set of colors from the M distinct colors; (b) determining at least one representative color for each starting leaf node; and (c) growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is determined by 1) determining an associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; 2) determining an associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, 3) determining the associated achievable cost based on the associated change in distortion and the associated increase in entropy rate; (ii) creating the two new leaf nodes by allocating each color in n to one of the two new leaf nodes; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.
In accordance with an embodiment of the invention, there is provided a computer program product for use on a computer to progressively encode a digitized color image with M distinct colors allocated to a set of pixels, the digitized color image being provided by assigning each of the M distinct colors to an associated subset of pixels in the set of pixels. The computer program product comprises a recording medium; and, means recorded on the recording medium for instructing the computer system to perform the steps of: (a) initializing a tree structure by providing at least one starting leaf node comprising a set of colors from the M distinct colors; (b) determining at least one representative color for each starting leaf node; and (c) growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is determined by 1) determining an associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; 2) determining an associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, 3) determining the associated achievable cost based on the associated change in distortion and the associated increase in entropy rate; (ii) creating the two new leaf nodes by allocating each color in n to one of the two new leaf nodes; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.
In accordance with a further embodiment of the invention, there is provided a system for progressively encoding a digitized color image with M distinct colors allocated to a set of pixels, the digitized color image being provided by assigning each of the M distinct colors to an associated subset of pixels in the set of pixels. The system comprises a recording medium; and, means including a processor for performing the steps of: (a) initializing a tree structure by providing at least one starting leaf node comprising a set of colors from the M distinct colors; (b) determining at least one representative color for each starting leaf node; and (c) growing the tree structure by (i) selecting a leaf node n to become a non-leaf node n linked to two new leaf nodes based on an associated achievable cost, wherein the associated achievable cost is determined by 1) determining an associated change in distortion resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; 2) determining an associated increase in entropy rate resulting from turning the leaf node into the non-leaf node linked to the two new leaf nodes; and, 3) determining the associated achievable cost based on the associated change in distortion and the associated increase in entropy rate; (ii) creating the two new leaf nodes by allocating each color in n to one of the two new leaf nodes; (iii) determining a representative color for each of the two new leaf nodes; and (iv) encoding the index information associated with the leaf node n, the representative colors of the two new leaf nodes, and information regarding a plurality of pixels of the digitized color image corresponding to the two representative colors of the two new leaf nodes.
BRIEF DESCRIPTION OF THE DRAWINGS
For a better understanding of the various embodiments described herein and to show more clearly how they may be carried into effect, reference will now be made, by way of example only, to the accompanying drawings which show at least one exemplary embodiment and in which:
FIG. 1 in a flow chart, illustrates an overall algorithm for entropy constrained color splitting for palette images with color wise splitting between leaf nodes in accordance with an aspect of an embodiment of the invention;
FIG. 2 in a flow chart, illustrates an overall algorithm for entropy constrained color splitting for palette images with pixel wise splitting between leaf nodes in accordance with a further aspect of a further embodiment of the invention;
FIG. 3 in a flow chart, illustrates an initialization sub-process of the methods of FIGS. 1 and 2;
FIG. 4 in a flow chart, illustrates an example of a sub-process of the method of FIG. 1 in which a leaf node is split;
FIG. 5, in a flow chart, illustrates an example of a sub-process of the method of FIG. 2 in which a leaf node is tentatively split;
FIG. 6, in a flow chart, illustrates a sub-process of the method of FIG. 2 in which a leaf node is permanently split;
FIG. 7, in a flow chart, illustrates a variant of a sub-process of the method of FIG. 1 for splitting a leaf node;
FIG. 8, in a flow chart, illustrates a further variant of splitting a leaf node in the method of FIG. 1;
FIG. 9, in a flow chart, illustrates a sub-process of the methods of FIGS. 1 and 2 in which new representative colors are progressively encoded;
FIG. 10, in a block diagram, illustrates a computer system in accordance with an embodiment of the invention;
FIG. 11, in a block diagram, illustrates a processor of the computer system of FIG. 10 in more detail;
FIGS. 12 a - d, in graphs, illustrates the progressive splitting of leaf nodes to provide progressive encoding of colored data in accordance with an aspect of an embodiment of the invention;
FIG. 13 is a block diagram of an exemplary embodiment of a mobile device;
FIG. 14 is a block diagram of an exemplary embodiment of a communication subsystem component of the mobile device of FIG. 13;
DETAILED DESCRIPTION OF THE INVENTION
Pictures with thousands or millions colors can be represented as true color images, storing the exact color for each pixel in the formats such as BMP (bitmap) and JPEG. In contrast, pictures containing only a few (typically up to 256) colors can be stored more efficiently as colormapped images or palettized images like the well known GIF (Graphics Interchange Format) or PNG (Portable Network Graphics) format. A significant portion of web images are in the format of GIF or PNG.
When browsing palettized images over low-bandwidth communication links, such as wireless links or dial-up links, progressive transmission of these palette images can support fast browsing since it enables the users to view complete versions of these images (but at a low resolution) before all progressively coded image data has been received. However, even though both GIF and PNG support an interlaced format, which refines the resolution of an image progressively by encoding the image in multiple passes and skipping a number of pixels in each pass, interlaced GIF and PNG are not in a progressive format in the strict sense that the color information of each pixel is refined progressively up to lossless recovery.
Rauschenbach investigated the progressive refinement of color information for palette images in his so-called MCQ (Mobile visualization Color Quantized format) method (U. Rauschenbach, “Compression of palettized images with progressive coding of the color information,” in Proc. SPIE Visual Communications and Image Processing ( VCIP 2000), Perth, Australia, June 2000, hereinafter referred to as [1]). Unlike the interlaced GIF and PNG formats, the refinement of color information in MCQ is done bit-plane by bit-plane, and allows the early recognition of shapes with fine details (e. g. text), normally after decoding a small portion of the compressed bit stream. Although MCQ supports progressive refinement of color information in contrast to resolution refinement, it only achieves a compression ratio comparable to GIF or PNG format. To improve the compression ratio while supporting the progressive transmission of palette images, Chen et al. (X. Chen, S. Kwong, and J. Feng, “A new compression scheme for color-quantized images,” IEEE Transactions on Circuits and System for Video Technology, Vol. 12, No. 10, pp. 904˜908, October 2002, hereinafter referred to as [2]) proposed a new color refinement scheme based on MCQ and the distortion-based hierarchical splitting color quantization algorithm pioneered by Orchard and Bouman (M. Orchard and C. Bouman, “Color quantization of images,” IEEE Transactions on Signal Processing, Vol. 39, No. 12, pp. 2677˜2690, December 1991, hereinafter referred to as [3]). Unlike MCQ where the color information is refined bit-plane by bit-plane with the number of bit-planes equal to the color depth of the palette, Chen's algorithm refines the color information color by color. For lossless compression of palette images, the color splitting algorithm proposed in [2] achieves around 30˜40% size reduction compared to GIF or PNG format in addition to its decent feature of supporting the progressive transmission.
The color splitting algorithm in [2], [3] splits a representative color based on a distortion criterion only and does not consider the rate constraint at all. On the other hand, each color in a palette image is a 3-D vector. Color quantization or color splitting is essentially a vector quantization (VQ) problem (Y, Linde, A, Buzo, and R. Gray, “An algorithm for vector quantizer design,” IEEE Trans. Communications, Vol. COM-28, No. 1, pp. 84-95, January 1980, hereinafter referred to as [4]). Entropy constrained vector quantization (ECVQ) or conditional entropy constrained vector quantization (CECVQ) proposed by Chou et al. (P. Chou, T. Lookabaugh, and R. Gray, “Entropy-constrained vector quantization,” IEEE Trans. Acous., Speech, and Signal Processing, Vol. 37, No. 1, pp. 31-42, January 1989 , hereinafter referred to as [5], and P. Chou and T. Lookabaugh, “Conditional entropy constrained vector quantization,” Proceeding ICASSP 1990, pp. 197-200, April 1990, hereinafter referred to as [6]) can trade off the distortion with the entropy rate. To reduce the complexity of a normal vector quantization algorithm and make it feasible in practical applications, tree-structured vector quantization (TSVQ) algorithms (A. Buzo, A. Gray, Jr., R. Gray, and J. Markel, “Speech coding based upon vector quantization,” IEEE Trans. Acous., Speech, and Signal Processing, Vol. ASSP-28, No. 5, pp. 562-574, October 1980, hereinafter referred to as [7], E. Riskin, and R. Gray, “A greedy tree growing algorithm for the design of variable rate vector quantizers,” IEEE Transactions on Signal Processing, Vol. 39, No. 11, pp. 2500-2507, November 1991, hereinafter referred to as [8], and M. Mahesh, W. Pearlman, and L. Lu, “Variable-rate tree-structured vector quantizers,” IEEE Trans. Information Theory, Vol. 41, No. 4, pp. 917-930, July 1995, hereinafter referred to as [9]) have been developed. In this approach, we apply the idea of ECVQ/CECVQ and TSVQ into a binary tree based color splitting scenario and develop four entropy constrained color splitting algorithms for palette images, which not only result in progressive transmission schemes but significantly outperform the scheme in [2] in R-D (rate-distortion) sense.
The rest of this document is organized as follows. The next section first describes the unconditional entropy constrained color splitting problem, and then provides two splitting algorithms using a fixed Lagrangian multiplier and a dynamically determined Lagrangian multiplier, respectively, in accordance with an embodiment. In Section III, in accordance with another embodiment, a pixel-wise conditional entropy constrained color splitting algorithm using a fixed Lagrangian multiplier is described. In Section IV, a color-wise conditional entropy constrained color splitting algorithm is presented according to another embodiment, which adds a restriction that all the pixels with the same color value in the original image can be updated by the same representative color values during the splitting process.
II. Unconditional Entropy Constrained Color Splitting
A binary tree structure, which can preserve more correlation between color indices than a linear structure, is used in [2] and [3] to represent the relationship of the colors of a palette image for the purpose of partitioning the color map, or equivalently, splitting a representative color progressively during the encoding process. Assume that the color map of a palette image is C={c 0 ,c 1 , . . . c M−1 }, where M is the number of distinct colors in the original palette image. Each color vector (or called color entry) c i corresponds to a color value (r i ,g i ,b i ). Let f(c i ) be defined as the occurrence number of c i in the given image. Our goal is to design a binary tree-based color splitting algorithm that minimizes the distortion subject to an unconditional entropy constraint. Such a color splitting algorithm partitions the color map C or splits the representative color of C progressively until the original palette image is losslessly reconstructed or a user-specified target number of colors is reached. The partitioning process yields a dynamic binary tree with each leaf node n corresponding to a subset S n of C such that C=∪ n S n and S m ∩S n =Ø where m≠n, 0≦m,n<N and N is the total number of leaf nodes of the dynamic binary tree. Denote q n as the representative color of S n that is defined as the centroid of S n . Then the dynamic binary tree, together with q n , 0≦n<N, gives rise to a tree-structured vector quantizer such that as the dynamic binary tree grows, the current vector quantizer is fully embedded into a subsequent high rate vector quantizer. A key question is how to grow the dynamic binary tree.
Starting from the root node, which represents the original color map C, we will grow the dynamic binary tree by splitting a desired leaf node each time. The process of selecting and splitting such a desired leaf node is conducted so as to minimize a Lagrangian cost
J=D+λ·R (1)
where D and R are, respectively, the distortion and entropy of the reconstructed palette image corresponding to the newly created dynamic binary tree after one leaf node is split into two new leaf nodes. The Lagrangian multiplier λ, which represents the tradeoff of rate for distortion, may be determined dynamically for each leaf node being split or fixed during the entire splitting process. We consider both cases in this section.
Let T N be a binary tree having N leaf nodes and T N+1 a binary tree generated from T N by splitting a leaf node n of T N into two child nodes n′ and n″. Let d(c i ,q i ) be the color quantization error when a color entry c i =(r i ,g i ,b i )∈C is replaced by a representative color q n =({circumflex over (r)} n ,ĝ n ,{circumflex over (b)} n ). Specifically,
d ( c i ,q n )=∥ c i −q n ∥ 2 =( r i −{circumflex over (r)} n ) 2 +( g i −ĝ n ) 2 +( b i −{circumflex over (b)} n ) 2 (2)
Then the distortion reduction resulting from splitting the leaf node n into n′ and n″ is
where D N and D N+1 are, respectively, the distortion corresponding to the trees T N and T N+1 ; q n′ and q n′ are, respectively, the representative colors of S n′ and S n′ . Note that S n =S n′ ∪S n″ and S n′ ∩S n′ =Ø. Furthermore, let
and
be, respectively, the numbers of pixels whose color indices fall into S n′ and S n″ when S n is partitioned into S n′ and S n″ , and denote f(S n )=f(S n′ )+f(S n″ ). Then the increment of the unconditional entropy resulting from splitting the leaf node n into n′ and n″ can be expressed as
where R N and R N+1 are, respectively, the entropy corresponding to the trees T N and T N+1 . Note that when splitting the leaf node n, neither ΔD N nor ΔR N depends on other leaf nodes in this binary tree-based color splitting.
An iterative process can be applied to split the leaf node n such that a minimum cost is achieved. The unconditional entropy constrained color splitting algorithm with a fixed λ is summarized as follows.
Algorithm 1: Unconditional Entropy Constrained Color Splitting Algorithm With a Fixed λ.
Step 1—initialization.
Given a palette image / and a target number of colors N target or a target rate R target , obtain a binary tree T 1 which only has a root node. The representative color corresponding the root node is the centroid of the original color map, i.e.
where
Calculate the principal eigenvalue of the cluster covariance matrix associated to S 0 as discussed in [2] and [3]. Set N=1; R 1 =0; and D 1 =σ 2 , where σ 2 is the variance of C. Select a convergence criterion ε and label the root node as a leaf node.
Step 2—find the minimum achievable cost for splitting each leaf node.
For each leaf node n of T N , use the following steps to find the minimum achievable cost from tentatively splitting n.
Step 2.1
Initialize J old n as D N +λ·R N .
Step 2.2
For each color c i ∈S n , assign c i to leaf node n′ or n″ based on the biased distortion measure. That is, assign c i to n′ if
Otherwise, assign c i to n″. Note that q n′ , q n′ , f(S n′ ), and f(S n″ ) are unknown for the first iteration. Instead, the first iteration splits node n using distortion-based criterion as described in [2].
Step 2.3
Calculate the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ as the centriods of the new partitions, S n′ and S n″ obtained in Step 2.2, and update f(S n′ ) and f(S n″ ). Calculate ΔD N and ΔR N resulting from the current splitting using (3) and (4), which results in a new cost J new n =(D N −ΔD N )+λ·(R N +ΔR N ). If |J old n −J new n |/J old n ≦ε or only one color entry is included in S n′ or S n″ , the minimum achievable cost from splitting n is taken as J new n . Otherwise, replace J old n with J new n and go to step 2.2 for further iteration.
Step 3—grow T N to T N+1 .
Find leaf node n which achieves the minimum cost among all the leaf nodes,
Split n to obtain a new binary tree T N+1 via the following iterative procedure.
Step 3.1
Initialize J old as D N +λ·R N .
Step 3.2
For each color entry c i ∈S n , assign c i to leaf node n′ or n″ using the biased distortion measure (6). Note that q n′ , q n″ , f(S n′ ), and f(S n″ ) in (6) are unknown for the first iteration. Instead, the first iteration splits node n using distortion-based criterion as described in [2].
Step 3.3
Calculate the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ as the centriods of the new partition S n′ and S n″ obtained in Step 3.2, and update f(S n′ ) and f(S n″ ). Calculate ΔD N and ΔR N resulting from the current splitting using (3) and (4), which results in a new cost J new =(D N −ΔD N )+λ·(R N +ΔR N ). If |J old −J new |/J old ≦ε or only one color entry is included in S n′ or S n″ , stop the iteration of splitting the leaf node n, extend the node n into two new leaf nodes labeled as n′ and n″ respectively, and go to step 4. Otherwise, replace J old with J new and go to step 3.2 for further iterations.
Step 4—progressively transmitting (or merely encoding and then, say, storing) the color updating bits.
Send out the node index being split (i.e., n), the new representative colors (i.e., q n′ and q n″ ) as well as the color updating bits for pixels whose color fall into S n . Update the total bit rate as R N+1 =R N +ΔR N . If N+1<N target or R N+1 <R target , then update the new distortion as D N+1 =D N −ΔD N ; increase N by 1; and then go to step 2 for further splitting. Otherwise, stop the splitting and transmitting process.
During the above splitting process, Algorithm 1 employs a fixed λ, which has the physical meaning of the slope of the distortion-rate function. Another approach is to use a variable λ and perform the splitting so as to find a good trade-off between the overall distortion decrease and rate increase. In other words, when we grow T N into T N+1 , we aim at maximizing λ, where
over all possible iterations at each leaf node of T N and across all leaf nodes of T N . At each leaf node of T N , we find its maximum achievable λ through an iterative process, which is similar to the one to be used when the node is to be split. Specifically, the splitting begins with the assumption that λ is zero. The two representative colors that minimize (1) for this value of λ are found. The decrease in distortion and increase in rate are calculated. The new value of λ is the ratio of the decrease in distortion to the increase in rate. The splitting is now redone with this new value of λ. This iterative process continues until λ does not increase. The converged value is then regarded as the maximum achievable λ at this leaf node. The actual leaf node to be split is the one giving rise to the maximum converged value across all leaf nodes of T N . Algorithm 2 below gives the detailed description.
Algorithm 2: Unconditional Entropy Constrained Color Splitting Algorithm With a Dynamically Determined λ.
Step 1—initialization.
Given a palette image / and a target number of colors N target or a target rate R target , obtain a binary tree T 1 which only has the root node. The representative color corresponding to the root node is the centroid of the color map that can be calculated using (5). Set N=1; R 1 =0; and D 1 =σ 2 . Select a convergence criterion ε and label the root node as a leaf node.
Step 2—find the maximum achievable λ for each new leaf node.
For each leaf node n of T N , use the following steps to find the maximum attainable λ n,max from splitting n as if the leaf node n is to be split.
Step 2.1
Set λ n t =0 where t in λ n t is an iterative index.
Step 2.2
For each color c i ∈S n , assign c i to an intended new leaf node n′ or n″, based on the biased distortion measure. That is, assign c i to n′ if
Otherwise, assign c i to node n″. Note that q n′ and q n″ are unknown for the first iteration. Instead, the first iteration splits the leaf node n using the distortion-based spitting algorithm discussed in [2]. Alternatively, we may perturb q n into q n′ , q n″ for the first iteration. Even though f(S n′ ) and f(S n″ ) are also unknown for the first iteration, the algorithm works fine since λ n t is assumed to be zero for the first iteration.
Step 2.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ to be the centriods of the new partition S n′ and S n″ obtained in Step 2.2, respectively, and update f(S n′ ) and f(S n″ ) into the numbers of pixels whose colors fall into S n′ and S n″ obtained in Step 2.2, respectively. Calculate ΔD N and ΔR N resulting from the current splitting using (3) and (4). If
and then go to step 2.2 for further iterations. Otherwise, take the current λ n t as the maximum achievable λ n,max at the leaf node n.
Step 3—grow T N to T N+1 .
Find the leaf node n that has the maximum value λ n,max among all the leaf nodes of T N , i.e.,
Split the leaf node n to obtain a new binary tree T N+1 via the following iteration process.
Step 3.1
Initialize J old as D N +λ n,max ·R N .
Step 3.2
For each color entry c i ∈S n , assign c i to an intended new leaf node n′ or n″ based on the resulting Lagrangian cost. That is, assign c i to node n′ if
Otherwise, assign c i to node n″. Note that q n′ , q n″ , f(S n′ ), and f(S n″ ) are unknown for the first iteration. Instead, the first iteration splits node n using the distortion-based criterion as described in [2].
Step 3.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ to be the centriods of the new partition S n′ and S n′ obtained in Step 3.2, respectively, and update f(S n′ ) and f(S n″ ) into the numbers of pixels whose colors fall into S n′ and S n″ obtained in Step 3.2, respectively. Calculate ΔD N and ΔR N resulting from current splitting using (3) and (4), which results in a new cost J new =(D N ΔD N )+λ n,max ·(R N +ΔR N ). If |J old −J new |/J old ≦ε, or only one color entry is included in S n′ or S n″ , stop the iteration of splitting the leaf node n, extend the node n into two new leaf nodes labeled as n′ and n″ respectively, and go to step 4. Otherwise, replace J old with J new and go to step 3.2 for further iterations.
Step 4—progressively transmitting (or merely encoding and then, say, storing) the color updating bits.
Send out the node index being split (i.e., n), the new representative colors (i.e., q n′ and q n″ ) as well as the update bits for pixels whose colors fall into S n . Update the total bit rate as R N+1 =R N +ΔR N . If N+1<N target or R N+1 <R target , then update the new distortion as D N+1 =D N −ΔD N ; increase N by 1; and then go to step 2. Otherwise, stop the splitting and transmitting process.
III. Conditional Entropy Constrained Color Splitting: Pixel-wise Splitting
In practical applications, context-based arithmetic coding like the one used in [2] is usually employed to encode the update bits in the entropy coding stage in Algorithms 1 and 2 so as to obtain a high compression gain. In Algorithms 1 and 2, however, the rate constraint used is an unconditional entropy. If we could use a conditional entropy as the constraint in a color splitting algorithm, the entropy used in the splitting process would match the entropy of the encoding stage more accurately. Consequently, better results in the rate-distortion sense can be obtained. In this section, we will extend the unconditional entropy constrained color splitting algorithm with a fixed λ to the conditional entropy constrained scenario.
For conditional entropy constrained color splitting, we will consider two separate cases. We call them pixel-wise splitting and color-wise splitting, respectively, in what follows. Pixel-wise splitting means pixels with the same color in the original image may be mapped to different leaf nodes during the splitting process. In other words, pixels with the same color in the original image may be updated by different representative colors during the splitting process based on different context. For a palette image with M distinct colors, in general, the original image may not be perfectly reconstructed after M−1 steps of node splitting even though the original palette image can be losslessly reconstructed eventually. On the other hand, color-wise splitting means all the pixels with the same color in the original image must be mapped to the same node during the splitting process. In other words, all the pixels with the same color in the original image will be updated by the same representative colors during the splitting process. For a palette image with M distinct colors, the original image is guaranteed to be perfectly reconstructed after M−1 steps of node splitting. We will develop the pixel-wise conditional entropy constrained color splitting algorithm in this section and consider the color-wise conditional entropy constrained color splitting algorithm in Section IV. We only consider using a fixed λ in both cases.
The main difference between the unconditional entropy constrained color splitting algorithm and conditional entropy constrained color splitting algorithm is the calculation of the increase of entropy rate ΔR N shown in (4) and the biased distortion measure as shown in (6). To calculate the conditional entropy rate during the splitting process, we may use an arbitrary number of neighboring pixels for the context modeling purpose. In what follows, we calculate the conditional entropy rate conditioned on the left and upper pixels only. The extension to more neighboring pixels is straightforward.
As indicated by the name, a pixel-wise conditional entropy constrained color splitting algorithm updates the color information in a pixel-wise manner. Let c p be the original color of pixel p in C and denote Ψ n as the pixel set corresponding to the leaf node n during the splitting process. For a pixel p∈Ψ n , whose left and upper neighboring pixels' representative color indices are u and v in the current reconstructed image, mapping p to the leaf node n′ results in an increase of entropy under context u and v as
where f u,v (Ψ n′ ) and f u,v (Ψ n″ ) are respectively the number of pixels mapped to the leaf node n′ or n′ under context u and v from the previous iteration. Similarly, mapping p to the leaf node n″ results in an increase of entropy under context u and v as
Each pixel p∈Ψ n is mapped to the leaf node n′ or n″ separately based on the resulting incremental Lagrangian cost. Pixel-wise conditional entropy constrained color splitting algorithm is summarized as follows.
Algorithm 3: Pixel-wise Conditional Entropy Constrained Color Splitting Algorithm With Fixed λ.
Step 1—initialization.
Given a palette image / with color map C and a target number of colors N target or a target rate R target , obtain a binary tree T 1 which only has a root node. Initialize N as 1. Set R 1 =0 and D 1 =σ 2 . Calculate the representative color q 0 and the principal eigenvalue of the cluster covariance matrix associated to the root node. Select a convergence criterion ε and label the root node as a leaf node.
Step 2—find the minimum achievable cost for splitting each leaf node.
For each leaf n of T N , use the following steps to find the minimum achievable cost from tentatively splitting n.
Step 2.1
Initialize J old n as D N +λ·R N .
Step 2.2
For each pixel p∈Ψ n , whose left and upper neighboring pixels' representative colors are u and v in current reconstructed image, map p to leaf node n′ or n″ based on the biased distortion measure. That is, map p to node n′ if
d ( c p ,q n′ )+λ·H n′ u,v ≦d ( c p ,q n″ )+λ·H n″ u,v (12)
Otherwise, map p to node n″. Note that q n′ , q n″ , H n′ u,v , and H n″ u,v are unknown for the first iteration. Instead, the distortion-based criterion is used for the first iteration.
Step 2.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ and the statistics f u,v (Ψ n′ ) and f u,v (Ψ n″ ). Calculate H n′ u,v and H n″ u,v using (10) and (11). Calculate ΔD N and ΔR N where
which results in a new cost J new n =(D N −ΔD N )+λ·(R N +ΔR N ). If |J old n −J new n |/J old n ≦ε or only one pixel is included in Ψ n′ or Ψ n″ , the minimum achievable cost from splitting n is taken as J new n . Otherwise, replace J old n with J new n and go to step 2.2 for further iteration.
Step 3—grow T N to T N+1 .
Find leaf node n which achieves the minimum cost among all the leaf nodes, i.e.,
Split n to obtain a new binary tree T N+1 via the following iterative procedure.
Step 3.1
Initialize J old as D N +λ·R N .
Step 3.2
For each pixel p∈Ψ n , whose left and upper neighboring pixels' representative colors are u and v in current reconstructed image, map p to leaf node n′ or n″ using the biased distortion measure (12). Note that q n′ , q n″ , H n′ u,v , and H n″u,v in (12) are unknown for the first iteration. Instead, the distortion-based criterion is used for the first iteration.
Step 3.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″ and the statistics f u,v (Ψ n′ ) and f u,v (Ψ n″ ). Calculate H n′ u,v , and H n″u,v using (10) and (11). Calculate ΔD N and ΔR N resulting from the current splitting using (13) and (14), which results in a new cost J new =(D N −ΔD N )+λ·(R N +ΔR N ). If |J old −J new |/J old ≦ε, or only one pixel is included in Ψ n′ or Ψ n″ , stop the iteration of splitting node n, label nodes n′ and n″ as new leaf node, and go to step 4. Otherwise, replace J old with J new and go to step 3.2 for further iterations.
Step 4—progressively transmitting (or merely encoding and then, say, storing) the color updating bits.
Send out the node index being split (i.e., n), the new representative colors (i.e., q n′ and q n″ ) as well as the color updating bits for all pixels p∈Ψ n . Update the total bit rate as R N+1 =R N +ΔR N . If N+1<N target or R N+1 <R target , then update the new distortion as D N+1 =D N −ΔD N ; increase N to by 1; and then go to step 2 for further splitting. Otherwise, stop the splitting and transmitting process.
IV. Conditional Entropy Constrained Color Splitting: Color-wise Splitting
Pixel-wise splitting means the pixels with the same color value in the original image may be updated to their final color through different intermediate representative colors. In other words, the pixels with the same color in the original image may be mapped to different nodes during the splitting process. In order to ensure the same color in the original image will be updated by the same representative colors during the splitting process, we may add a restriction that all the pixels with the same color in the original image will be mapped to the same node during the splitting process. This leads to the color-wise conditional entropy constrained splitting algorithm that will be described in detail in this section.
During splitting a leaf node n into n′ and n″, the pixels corresponding to a color entry c i ∈S n may have different contexts u and v in the current reconstructed image, where each pixel p (c p =c i ) corresponds to an entropy H n′ u,v or H n″ u,v as defined in (10) and (11). Assign c i to n′ results in an increase of entropy per pixel as
Similarly, assign c i to n″ results in an increase of entropy per pixel as
Each color entry c i ∈S n is still mapped to leaf node n′ or n″ to minimize the resulting Lagrangian cost through an iterative process. The color-wise conditional entropy constrained color splitting algorithm is summarized as follows.
Algorithm 4: Color-wise Conditional Entropy Constrained Color Splitting Algorithm With a Fixed λ.
Step 1—initialization.
Given a palette image / with color map C and a target number of colors N target or a target rate R target , obtain a binary tree T 1 which only has a root node. Set S 0 =C and N=1. Set R 1 =0 and D 1 =σ 2 . Calculate the representative color of S 0 using Equation (5) and the principal eigenvalue of the cluster covariance matrix associated to S 0 . Select a convergence criterion ε and label the root node as a leaf node.
Step 2—find the minimum achievable cost for splitting each leaf node.
For each leaf node n of T N , use the following steps to find the minimum achievable cost from tentatively splitting n.
Step 2.1
Initialize J old as D N +λ·R N .
Step 2.2
For each color c i ∈S n , assign c i to leaf node n′ or n″ based on the biased distortion measure. That is, assign c i to n′ if
d ( c i ,q n′ )+λ· H n′ c
Otherwise, assign c i to n″. Note that q n′ , q n″ , H n′ c
Step 2.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″, and the statistics f u,v (Ψ n′ ) and f u,v (Ψ n″ ). Calculate H n′ c
Step 3—grow T N to T N+1 .
Find leaf node n which achieves the minimum cost among all the leaf nodes, i.e.,
Split n to obtain a new binary tree T N+1 via the following iterative procedure.
Step 3.1
Initialize J old as D N +λ·R N .
Step 3.2
For each color c i ∈S n′ , assign c i to leaf node n′ or n″ using the biased distortion measure (17). Note that q n′ , q n″ , H n′ c
Step 3.3
Update the new representative colors, q n′ and q n″ , of the leaf nodes n′ and n″, and the statistics f u,v (Ψ n′ ) and f u,v (Ψ n″ ). Calculate H n′ c
Step 4—progressively transmitting (or merely encoding and then, say, storing) the color updating bits.
Send out the node index being split (i.e., n), the new representative colors (i.e., q n′ and q n″ ) as well as the color updating bits for pixels whose color fall into S n . Update the total bit rate as R N+1 =R N +ΔR N . If N+1<N target or R N+1 <R target , then update the new distortion as D N+1 =D N −ΔD N ; increase N to by 1; and then go to step 2 for further splitting. Otherwise, stop the splitting and transmitting process.
Referring now to FIG. 1, the basic operation of algorithms 1, 3, and 4 will be described. Algorithms 1, 3, and 4 all employ a fixed Lagrangian multiplier λ representing a trade-off between distortion and entropy rate. Process 110 shows the steps common to these three algorithms.
Process 110 begins at step 111 and immediately progresses to step 112 where the splitting process is initialized. Step 112 will be described in greater detail below with reference to FIG. 3. Once the splitting process is initialized, the Lagrangian multiplier λ is selected at step 113 . The first iteration of sub-process 127 then begins.
Each iteration of sub-process 127 splits a leaf node n by creating two new leaf nodes n′ and n″ and progressively encodes the new representative colors for the new leaf nodes as well as information regarding which pixels in the image should be updated using these colors. Once n is split, it becomes a non-leaf node.
The first objective of sub-process 127 is to determine which leaf node must be split. This is done in steps 114 - 118 . At step 114 , a leaf node index n is initialized to 0. Leaf node n (the leaf node with index n) is then split according to the splitting process in algorithm 1, 3, or 4 at step 115 . Each of these splitting processes are described in greater detail below with reference to FIGS. 4, 7 and 8 respectively. Each of these processes includes the determination of a minimum cost for splitting node n into two new leaf nodes based on the associated change in distortion and the associated increase in the entropy rate. Once the splitting of leaf node n is complete, sub-process 127 determines whether or not all leaf nodes in the tree structure have been split at step 117 using the variable N, representing the number of leaf nodes in the tree structure, which is initialized at step 314 of process 310 described below with reference to FIG. 3. If all N leaf nodes have not been split, leaf node index n is incremented by one at step 116 and process 110 returns to step 115 to continue the splitting process. Once all leaf nodes in the tree structure have been split, process 110 continues to step 118 . At step 118 , the leaf node n which achieves the lowest minimum cost among all the leaf nodes is selected.
Once the leaf node n with the lowest minimum cost has been determined, n is split according to the splitting process in algorithm 1, 3, or 4 at step 119 . Each of these splitting algorithms are described in greater detail below with reference to FIGS. 4, 7 and 8 respectively. During this process, leaf node n becomes a non-leaf node and two new leaf nodes, n′ and n″, are created. At step 120 , leaf nodes n′ and n″ are labeled as new leaf nodes.
At step 121 , the new representative colors are progressively encoded along with the color updating bits. Step 121 will be described in greater detail below with reference to FIG. 9.
At step 122 , the entropy rate for the new tree structure which includes n′ and n″ is calculated by adding the entropy rate for the old tree structure before n was split and the increase in entropy caused by the splitting of n into n′ and n″.
At step 123 , the entropy rate for the new tree structure, calculated in step 122 , may be compared to a target entropy rate set during the initialization process described below with reference to FIG. 3. Alternatively, the number of representative colors, which corresponds to the number of leaf nodes in the new tree structure, is compared to a target number of colors set during the initialization process described below with reference to FIG. 3. If the new entropy rate is greater than or equal to the target entropy rate or if the number of colors in the new tree structure is greater than or equal to the target number of colors then process 110 proceeds to step 126 and terminates. Otherwise, process 110 continues to step 124 .
At step 124 , the distortion for the new tree structure is calculated as the sum of the distortion of the tree structure before node n was split and the decrease is distortion resulting from splitting node n into leaf nodes n′ and n″. The number of leaf nodes in the tree structure N is then incremented by one at step 125 , the process returns to step 114 and the splitting process continues.
Referring now to FIG. 2, the basic operation of algorithm 2 will be described. Process 210 functions similarly to process 110 except that process 210 employs a dynamically determined Lagrangian multiplier rather than a fixed Lagrangian multiplier as in process 110 .
Process 210 begins at step 211 and immediately progresses to step 212 where the splitting process is initialized. Step 212 will be described in greater detail with reference to FIG. 3. The first iteration of sub-process 227 then begins.
Each iteration of sub-process 227 splits a leaf node n by creating two new leaf nodes n′ and n″ and progressively encodes the new representative colors for the new leaf nodes along with information regarding which pixels in the image should be updated using these colors. Once n is split, it becomes a non-leaf node.
The first objective of sub-process 227 is to determine which leaf node must be split. This is done in steps 214 - 218 . At step 214 , a leaf node index n is initialized to 0. Leaf node n (the leaf node with index n) is then tentatively split according to the tentative splitting process for algorithm 2 at step 215 . This tentative splitting process is described in greater detail below with reference to FIG. 5. The tentative splitting processes includes the determination of a maximum achievable Lagrangian multiplier λ n,max for each leaf node n. Once the tentative splitting of leaf node n is complete, the process determines whether or not all leaf nodes in the tree structure have been tentatively split at step 217 using the variable N, representing the number of leaf nodes in the tree structure, which is initialized at step 314 of process 310 described below with reference to FIG. 3. If all N leaf nodes have not been tentatively split, leaf node index n is incremented at step 216 , process 210 returns to step 215 and the tentative splitting process continues. Once all N leaf nodes in the tree structure have been tentatively split, process 210 proceeds to step 218 . At step 218 , the leaf node n with the highest maximum achievable Lagrangian multiplier λ n,max , calculated in step 215 , is selected.
Once the leaf node n with the highest maximum achievable Lagrangian multiplier λ n,max has been determined, leaf node n is permanently split according to the permanent splitting process for algorithm 2 at step 219 . The permanent splitting process is described in greater detail below with reference to FIG. 6. During the permanent splitting process, leaf node n becomes a non-leaf node and two new leaf nodes, n′ and n″, are created. At step 220 , leaf nodes n′ and n″ are labeled as new leaf nodes.
At step 221 , the new representative colors are progressively encoded along with the color updating bits. Step 221 will be described in greater detail below with reference to FIG. 9.
At step 222 , the entropy rate for the new tree structure which includes n′ and n″ is calculated by adding the entropy rate for the old tree structure before n was split and the increase in entropy caused by the splitting of n into n′ and n″. The increase in entropy is calculated using equation (4).
At step 223 , the entropy rate for the new tree structure, calculated in step 222 , may be compared to a target entropy rate set during the initialization process described below with reference to FIG. 3. Alternatively, the number of representative colors, which corresponds to the number of leaf nodes in the new tree structure, is compared to a target number of colors set during the initialization process described below with reference to FIG. 3. If the new entropy rate is greater than or equal to the target entropy rate or if the number of colors in the new tree structure is greater than or equal to the target number of colors then process 210 proceeds to step 226 and terminates. Otherwise, the distortion for the new tree structure is calculated at step 224 as the sum of the distortion of the tree structure before node n was split and the decrease is distortion resulting from splitting node n into leaf nodes n′ and n″. The number of leaf nodes in the tree structure N is then incremented at step 225 , process 210 returns to step 214 and the splitting process continues.
Referring now to FIG. 3, the initialization of the splitting process will be described. Process 310 begins at step 311 and immediately progresses to step 312 .
At step 312 , a target number of colors N target or a target entropy rate R target are specified. These values will be used in steps 123 and 223 of processes 110 and 210 respectively to determine when the splitting process should be terminated.
A number of variables are then initialized. At step 313 , the initial representative color for the starting leaf node is calculated, using equation 5, as the centroid of the original color map for the image. The number of leaf nodes in the tree structure, N, is initialized to 1 at step 314 and the entropy rate for the initial tree structure is set to 0 in step 315 . The distortion associated with the initial tree structure is set to be the variance of the original color map for the image at step 316 . At step 317 a convergence criterion ε is selected. The convergence criterion is used in steps 422 , 622 , 723 and 824 of processes 410 , 610 , 710 and 810 respectively to determine when the splitting process for a node should terminate. These processes will be described in detail below. Once these variables have been initialized, process 310 continues to step 318 and terminates.
Referring now to FIG. 4, the process of splitting a leaf node n according to algorithm 1 will be described. Specifically, process 410 transforms n into a non-leaf node, creates two new leaf nodes n′ and n″ and assigns each color in n to either n′ or n″. The set of colors assigned to a node n is denoted as S n .
Process 410 begins at step 411 and immediately progresses to step 412 . At step 412 , a cost measurement J n old is initialized to the value attained by adding the distortion of the current tree structure to a product obtained by multiplying the entropy rate of the current tree structure by the Lagrangian multiplier λ selected in step 113 of process 110 .
At step 413 , an iterative index t, used to keep track of the number of iterations of sub-process 427 , is initialized to 0. The iterative index t is used at step 414 to determine whether this is the first iteration of sub-process 427 (ire. t=0). If this is the first iteration of sub-process 427 , at step 415 , each color in n is assigned to n′ or to n″ based on a distortion based criterion as described in [2]. In subsequent iterations of sub-process 427 , where t≠0, each color in n is assigned to n′ or n″ based on the biased distortion measure of equation (6) at step 416 .
Next, a number of values are calculated based on n′ and n″. First the new representative colors for n′ and n″, q n′ and q n″ , are calculated at step 417 . At step 418 , the number of occurrences in the original image of each color in n′ and n″, f(S