SNA Power Computer Vision

Histogram is a graphical representation that is used to represent the frequency of variables in the data. In this tutorial, we will discuss about how to generate a 3D histogram in OpenCV in Python and then we will use this histogram to find the color with the most number of pixels.

So, what we need to get a histogram in OpenCV?

In order to calculate the 3D histogram,

We need to specify the number of bins in each axis. The number of bins in each axis can be different, but in the tutorial we will set all of them equal to 16.

The next step is to specify the ranges of these bins. Again, we can select different ranges, but we will set the range from 0 to 256 for each axis. The value 256 is exclusive.

Next, we specify the number of channels whose histogram we want to calculate. In our case, its all three channels i.e [0, 1, 2]. If, it was a 2D array we might have had selected [0, 1] or [1, 2] or [0, 2] as per our choice.

Finally, is the set of images whose histogram we want to calculate and we, will set the mask to None. 

Once, the histogram is calculated using calcHist(), we sort the indexes using np.argsort() in ascending order. Now, the last value is the max color index. In the end, we multiply the index values by relevant constants to get the color in range [0,255]. Let's look at the code now - 

OUTPUT

Input Image

OUTPUT

Recently, I needed to compare a number of colors to get the 2 most identical colors. This problem at the facade looks like a cake walk, even for beginners, but is a bit involved. 

A naive implementation is to calculate to Euclidean distance (as shown below) between the RGB values of the 2 colors.

The 2 colors that have the lowest Euclidean Distance are then selected. But, there is a serous flaw in this assumption. Although RGB values are a convenient way to represent colors in computers, we humans perceive colors in a different way from how colors are represented in the RGB color space. What looks like identical colors to us, might give us a Euclidean distance that is greater than the Euclidean distance of colors that look different to us when comparing in RGB color space.

So, what to do now?

Delta-E distance metric comes to our rescue. It uses the CIE Lab color space which approximates how we perceive colors, although its also not fully accurate. Once we represent the color in the CIE color space, we can calculate the Delta-E distance metric using Euclidean Distance. Ever since its release in 1976, it has been modified 2 times to cope with shortcomings of the previous versions. The diagram below shows that by shifting along the x and y axis we get different shades of the same color.

  CIE76 -> CIE94 -> CIE2000

For, this tutorial we will use CIE2000. If you are interested in the actual mathematics, refer to the wikipedia page. It's is pretty easy to write your own code to compare the 2 colors, but we'll not try reinvent the wheel. Python provides a high utility package colormath to convert between different colorspaces, delta E comparison, density to spectral operation etc. 

Using colormath, we just don't need to put in much effort to find the Delta-E metric. Check the code below.

I hope you liked the post, THANK YOU.

ADDITIONAL NOTES - 

If you want to test various Delta-E converters, check here.

Download package colormath from here.

Gist link of script.

In this tutorial, we will write 2 python scripts - 

To load a image, extract the keypoints and save them in a file.

To load the file saved in the previous script and draw the contours on the same image used in step 1.

For loading and saving data, we will use cPickle package. To store data in a file we use dump and to load data we use loads. The problem we have is that we cannot directly dump keypoints objects, if we try to do so, we'll get an error as shown below - 

pickle.PicklingError: Can't pickle <type 'cv2.KeyPoint'>: it's not the same object as cv2.KeyPoint

So, we need to serialize and deserialize the keypoints data.

So, let's write code, for the first script.

SAVING THE KEYPOINTS

import cv2 import cPickle im=cv2.imread("/home/bikz05/Desktop/dataset/checkered-3.jpg") gr=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY) d=cv2.FeatureDetector_create("SIFT") kp=d.detect(gr) index = [] for point in kp: temp = (point.pt, point.size, point.angle, point.response, point.octave, point.class_id) index.append(temp) # Dump the keypoints f = open("/home/bikz05/Desktop/dataset/keypoints.txt", "w") f.write(cPickle.dumps(index)) f.close()

We have saved our keypoints in keypoints.txt . Now, let's load those keypoints in a separate script and draw them on the same image.

LOADING THE KEYPOINTS

import cv2 import cPickle im=cv2.imread("/home/bikz05/Desktop/dataset/checkered-3.jpg") index = cPickle.loads(open("/home/bikz05/Desktop/dataset/keypoints.txt").read()) kp = [] for point in index: temp = cv2.KeyPoint(x=point[0][0],y=point[0][1],_size=point[1], _angle=point[2], _response=point[3], _octave=point[4], _class_id=point[5]) kp.append(temp) # Draw the keypoints imm=cv2.drawKeypoints(im, kp); cv2.imshow("Image", imm); cv2.waitKey(0)

That's it, the above script when run will display the image with all the keypoints marked on it.

RESULTS

INPUT IMAGE 

OUTPUT IMAGE

In this tutorial, I will enumerate the steps needed to perform OCR using Google's Open Source OCR engine Tesseract. It was developed initially at HP Labs. 

INSTALLATION

Install the dependencies. 

sudo apt-get install libpng-dev libjpeg-dev libtiff-dev zlib1g-dev sudo apt-get install gcc g++ sudo apt-get install autoconf automake libtool checkinstall

We need image processing toolkit Leptonica to compile Tesseract, otherwise unlike older versions it will not compile.

wget http://www.leptonica.org/source/leptonica-1.69.tar.gz tar -zxvf leptonica-1.69.tar.gz cd leptonica-1.69 ./configure make sudo checkinstall sudo ldconfig

Compile and install Tesseract. 

wget https://tesseract-ocr.googlecode.com/files/tesseract-ocr-3.02.02.tar.gz tar -zxvf tesseract-ocr-3.02.02.tar.gz cd tesseract-ocr ./autogen.sh ./configure make (this may take a while) sudo make install sudo ldconfig

Get and install the English language data.

wget https://tesseract-ocr.googlecode.com/files/tesseract-ocr-3.02.eng.tar.gz tar -zxvf tesseract-ocr-3.02.eng.tar.gz sudo mv ./tesseract-ocr/tessdata/tessdata/* /usr/local/share/tessdata/

This is it!! We are done with installing Tesseract on Ubuntu. On let's test it on a image. 

WRITING A "HELLO WORLD"

Now I have got this pretty old scanned page of a poem eulogizing Sherlock.

We will run Tesseract from command line as shown below.

tesseract image.png output

Here - 

tesseract - is the command.

image.png - is the path to the image on which we are running OCR. I am assuming that image.png is in pwd.

output - The output will be stored in an image text file named output.txt. By default it will be stored in the current directory. 

Now let's check the output in output.txt

2213 (rout wan w. suns) HERE dwell rogether still two men of note Who never lived and so can never die: How very near they seem, ye: how remote Tm age berm me world went all awry. But still the game’: afoot for rhose with ears Avtuned to catch the distant View-halloo: England is England yer, for all our fenrs— Only those lhlngs the heart ézlin/ex are true. A yellow fog swirls pm the window-pane A: night descends upon lhls fabled street: A lonely hansom splashes through the rain, The ghostly gas lamps ran at (Wenly feet. Here, though the world explode, these two survive, And it is always eighteen ninety-five. MAW. H‘ .9“ Vmczwr Snuuus-rr

There are a lot of misspelled words in the output file like rogether instead of together. These words can be corrected by simply using a spell checker like Hunspell . 

MORE INFORMATION

Unofficial Tesseract Documentation - Click here

More Language packs - Click here

REFERENCES

opencv-android-cpp-test - Android OpenCV Native + Java Feature Detection using FAST

SCREENSHOT OF APP

(If the image does not load, click on the box below)

More Information

This tutorial is inspired by this Official OpenCV tutorial . You can find it the Samples folder of OpenCV4Android.

Android NDK setup to run C++ code.

Introduction

<To Be Written>

CODE

#include <iostream> #include <opencv2/opencv.hpp> using namespace std; using namespace cv; int main() { // First declaring three image containers - // image contains input image, // gray_image contains the grayscale converted input image // and keypoint_image contains final image with keypoints drawn on it. Mat image, gray_image, keypoint_image; // This is vector will contain the keypoint values vector<KeyPoint> keypoints; // Read the image in color format image = imread("manu.png",CV_LOAD_IMAGE_COLOR); // Declare Window to display the image namedWindow("Image", CV_WINDOW_AUTOSIZE); // Transform image from BGR format ro grayscale and cvtColor(image, gray_image, CV_BGR2GRAY); // Create a feature detector using FAST algorithm // You can also use - // "STAR" – StarFeatureDetector // "SIFT" – SIFT (nonfree module) // "SURF" – SURF (nonfree module) // "ORB" – ORB // "BRISK" – BRISK // "MSER" – MSER // "GFTT" – GoodFeaturesToTrackDetector // "HARRIS" – GoodFeaturesToTrackDetector with Harris detector enabled // "Dense" – DenseFeatureDetector // "SimpleBlob" – SimpleBlobDetector // Just replace the "FAST" string in the input of create() function by // the desired algorithms string defined above. Ptr<FeatureDetector> feature_detector = FeatureDetector::create("STAR"); // Detect keypoints in the gray_image and store them in keypoints. feature_detector->detect(gray_image, keypoints); // Display the number of keypoints detected cout << "Number of keypoints detected = " << keypoints.size() << endl; // Draw keypoints on the keypoint_image drawKeypoints(image, keypoints, keypoint_image, Scalar::all(-1), DrawMatchesFlags::DEFAULT); // Display the image imshow("Image",keypoint_image); // Wait until, user generates a keypoint event. waitKey(0); // return return 0; }

RESULTS

Feature Detection using FAST algorithm. Circles represent key points.

Also, when we use STAR algorithm, we get this result -

OpenCV 

Official OpenCV tutorials (C++)

Official OpenCV tutorials (Python)

Useful Books

Forums

Android 

Official Android Training Tutorials (Free) [RECOMMENDED]

Udacity Android Course (Free)

Official Java Tutorials (Free)

Eclipse

Cheatsheet

To develop Android apps using OpenCV Library, we need the following tools (all of these are either Open Source or free software, or both) - 

Eclipse with ADT Bundle (You can choose any other IDE like Android Studio or even the puissant command line!!, but ADT bundle is a must.)

OpenCV4Android (although, you can compile from source code, it is recommended for novice users to download the binaries.)  Caveat - Download the file named OpenCV-2.4.9-android-sdk.zip and not  opencv-2.4.9.zip. (Version might differ)

CygWin (Only for Windows, not needed on Linux or Mac)

Java JDK (JRE would not be sufficient) 

Android Native Development Toolkit (NDK)

You will need to set environment variables on your OS for the tools to correctly configure. For the sake of brevity, I would not discuss each step in detail but you can follow the link[1] given at the bottom of this post for detailed instructions.

Alternatively, nVidia also provides a suite of developing tools -- TADP. The advantage of TADP over the piece-by-piece method is that you don't need to follow the often perplexing task of setting up the development environment. The default installation will download a lot of superfluous packages that are not needed (The download size can be greater than 2GB and on slow internet connections, it can turn into a prolonged activity.), rather select the packages manually at the time of download (A dialog box will appear, asking you to select either Complete, Custom or Manually.)