(Featured image taken from here.)
Data compression is very important part of our digital world where we have tons of files with huge size. Now we have better and bigger quality data, specially, image. Most of smartphones have better quality camera and the picture taken from those takes more storage too. With more complex pixel combinations, more storage is taken. There are different compression algorithms like JPEG and PNG but my task here is to explain little bit about Lossless Compression using Run Length Encoding. The term lossless means there should not be any loss of data.
Image
What is image? Image is a combination of pixels in the digital world. Just like 2d plane, Image also have plane and it only have positive coordinates. I will be using Python to do image operations because it is easy to code on.
First I will try to compress a dummy image then will go into the real world image and see what can I achieve.
It shows that both of these files have same size. But if we see the file created on our folder, then the random image is taking 10.2KB while zero image is taking 214B. Why is that happening? OpenCV is using some compression algorithm to compress as much as possible and it becomes easier to compress an image which have least details or less complex, less varied pixel combination.
Lets find our the size of file from code.
def get_size(filename="dd.png"): stat = os.stat(filename) size=stat.st_size return sizeprint(get_size())print(get_size("d.png"))
21410180
There is huge difference of the size taken by two same sized image. Lets get into the RLE now.
Run Length Encoding
This is a lossless compression algorithm and it is very simple to practice on. The concept of RLE compression is that check for the consecutive runs of the current pixel value. This algorithm works best on Binary image.
Lets take an example, image [1 0 1 1 1 0 0 0 1 1 0 1 1 0 0 0] have 16 pixels and it is binary. Now using RLE, we do something like below.
At the begining, current pointer is first value i.e. 1. Now compressed image [11]. Which means that 1 has repeated 1 times consecutively.
Next, value is 0. Again it is repeated one times, hence compressed image is [11 01]. Which means, 1 is repeated one times then 0 is repeated one times.
Next, value is 1. Now 1 is repeated three times. Hence compressed image is [11 01 13].
Similarly, next compressed image is, [11 01 13 03].
I think above code is self explained. But I would describe it anyway.
Either convert image into Binary or use it as it is.
Then convert image into flat, i.e 1d vector.
Now scan from left to right.
If previous value is same as current then count the run else append (value, run) on encoded. And also check the run length, i.e. if it becomes 2**bits - 1 then append it. If it exceeds that value, then our values will be rounded off to 8 bit range later.
The decode function is simpler than encoding. Just perform repeat for values by run numbers and reshape it back.
Now save it as npz, npz.npy, tif and png format then find out which extension will compress most.
# save the encoded list into npz array fileearr=np.array(encoded)# earr=earr.astype(np.uint8)np.savez("np1.npz", earr)np.save("np2.npz", earr)
# store that array as image# the earr has shape of (x, 2) which can work fine as an image.cv2.imwrite("encoded.tif", earr)cv2.imwrite("encoded.png", earr)
# now check size of each filesfiles = ["encoded.png", "encoded.tif", "np1.npz", "np2.npz.npy"]for f in files: print(f"File: {f} => Size: {get_size(f)} Bytes")
From above example, it shows that when saving RLE as NumPy compressed, it will take 7696 and 7560 but when saving the encoded array as png image, it takes 2088 Bytes. But using TIFF format, it takes 1532. And also the decoding can be done easily.
Now lets use non binary format of an image.
# save the encoded list into npz array fileearr=np.array(encoded)# earr=earr.astype(np.uint8)np.savez("np11.npz", earr)np.save("np22.npz", earr)# store that array as image# the earr has shape of (x, 2) which can work fine as an image.cv2.imwrite("encoded1.tif", earr)cv2.imwrite("encoded1.png", earr)
True
# now check size of each filesfiles = ["encoded1.png", "encoded1.tif", "np11.npz", "np22.npz.npy"]for f in files: print(f"File: {f} => Size: {get_size(f)} Bytes")
be = RLE_encoding(b, binary=False)ge = RLE_encoding(g, binary=False)re = RLE_encoding(r, binary=False)
I have sent each channel but there doesn't seem huge difference on these channels. Lets combine these channels. But I doubt that we can combine these encoded arrays because each channel will have different runs so lets first see each channel's shape.
be.shape, ge.shape, re.shape
((2138548, 2), (2131812, 2), (2137244, 2))
There is no way we could treat these arrays as a channel of an image. What we could do is save each channel on drive.
files = "bgr"snp = 0stif = 0for f in files: ft=f+"e"+".npz" snp+=get_size(ft) ft=f+"e"+".tif" stif+=get_size(ft)print(f"Original size: {get_size('scene.jpg')/1024}, TIFF: {stif/1024}, NPZ: {snp/1024}")
Original size: 640.6875, TIFF: 7423.484375, NPZ: 50060.1796875
Original size is 640KB, TIFF size is 7.4MB and NumPy size is 50MB. It is not a good result. So how did it happen? There a re tons of reasons, first of them is that the image is complex and its pixel change is too frequent and there are not much runs. If this is the reason then I will try with the first image.
This was all an experiment and a little bit of time killing task without any surprising result. But the conclusion from this experiment is that, RLE works great when:
Image is Binary.
Image's pixel frequency is not huge.
Saving on TIFF format.
If you found this blog helpful then I would like to recommend you to support our work via subscribing our YouTube Channel.
All the codes are available on GitHub Repository.
Hello surfer, thank you for being here. We are as excited as you are to share what we know about data. Please subscribe to our newsletter for weekly data blogs and many more. If you’ve already done it, please close this popup.