#ml kit

MLKit Text Recognition Find out how you can use the latest ML Kit library from Google to utilize computer vision and machine learning to recognize text within an image.

at Google I/O 2018 Google has announced an updated software library called the ML Kit that allows developers to make use of machine learning on mobile devices. It allows you to add some standard machine learning features to your app without being an expert in machine learning!

What is an ML Kit? ML Kit is a mobile SDK that brings Google’s machine learning to Android apps in easy to use format. Google announced the ML Kit in Google I/O 2018, it also became one of the interesting announcements for the mobile app development company. Various ML technologies, provided are Google Cloud Vision API, TensorFlow Lite, and the Android Neural Networks API together in a single SDK.

Feature Highlights ML Kit provides some awesome features that are available on device and cloud. Let’s take a look at these features. It takes some ML uses cases and decorates them in a beautiful API.

On device features – Text recognition – Face detection – Barcode scanning – Image labeling – Custom model inference

Cloud features – Text recognition – Image labeling – Landmark recognition

Text Recognition The issue of text detection in natural scene pictures is challenging because of the abandoned size, colour, background and alignment of the character. Text Recognition provides detection and recognition of textual information in videos, documents, images, and other sources.

ML Kit’s text recognition APIs allows you to recognize text in any Latin-based language. Text recognition can be an easy data entry for receipts, business cards. With the cloud-based Text recognition API, you can extract the text from any type of document and also keep track of the real-life objects.

For example, you can use text recognition to obtain the number of a credit card, license card, name of the credit card or license holder, and the date of expiry from the front of a card.

Face Detection Face detection is a process of detecting the face from the images. You can use this to identify key facial features and get the contours of faces. ML Kit’s Face Detection API has been designed to work on the device itself which makes it fast, accurate and proficient of identifying faces in real time.

Below are the features provided by face detection in ML Kit ~ Recognize and locate facial features Provide the coordinates of the ears, eyes,  nose, cheeks, and mouth of every face that was detected.

~ Get the outlines of facial features Easily extract the contours of identified faces and their eyes, eyebrows, lips, and nose.

~ Recognize facial expressions It determines a person is smiling and his eyes are open or close.

~ Track faces across video frames Get a detector for each individual person’s face that is detected. for example, do image manipulation on a particular person in a video stream.

~ Process video frames in real time Face detection is performed on the device and used in real-time applications, such as video manipulation.

Barcode scanning Barcode Scanning is done to interpret the data encoded in the barcodes. ML Kit barcode scanning is working on the device so it is easy to implement and make it work fast and accurate. Using the barcode scanning we will be able to encode the 2D barcode and also encode the data like wifi network credential.

Below are the features provided by face detection in ML kit ~ Reads most standard formats – Linear formats: Codabar, Code 39, Code 93, Code 128, EAN-8, EAN-13, ITF, UPC-A, UPC-E – 2D formats: Aztec, Data Matrix, PDF417, QR Code

~ Automatic format detection It supports scanning for all supported barcode formats immediately, without having to mention the format you are looking for. Or, enhances the scanning speed by limiting the detector to only the formats you’re interested in.

~ Extracts structured data Structured data that is stored by using one of the supported 2D formats are automatically interpreted. The supported information types comprise of the URLs, contact details, calendar events, email addresses, phone numbers, SMS message prompts, ISBNs, WiFi connection information, geographic location, and AAMVA-standard driver information.

~ Works with any orientation Barcodes are identified and scanned regardless of their orientation: right-side-up, upside-down, or sideways.

Image Labeling ML Kit’s Image Labeling is an API and model that can identify items in an image, and provide information about those items in the form of labels. On-device APIs can process the data quickly without an internet connection. This is helpful if you don’t want to utilize the mobile data of your users and you need fast processing.

The on-cloud APIs provide much more compelling processing capabilities. Thanks to Google Cloud Platform’s ML technology, however, these APIs need the Internet connection to continue to work. In case you are using the Google Cloud Platform, after 1000 requests you will have to make payment.

Example, if you have a Facebook or Instagram account, you might be familiar with face recognition. As soon as a user uploads a photo, Facebook will instantly suggest tags of your friends. Besides the tagging feature, image recognition also does the content translation for visually impaired people with screen readers. It also helps to identify irrelevant or objectionable images.

Landmark Recognition ML Kit’s landmark recognition API can provide well-known landmark labels directly from image pixels. It helps people better understand and maintain their photo collections. This API recognizes the number of famous landmarks over the world including bridges, building, place of worship etc. Landmark recognition provides the information about the images such as place name, a region of the image, coordinate of landmark etc. Landmark recognition API is fast and accurate. This API is free for 1000 uses of this API per month.

Custom Model Interface The custom model interface of ML kit provides you features. You can easily host your TensorFlow Lite models with the help of Firebase or bundle them with your app. Using the custom model for hosting provides your user updates with the latest version. It reduces your app’s binary size and provides automatically downloading of new versions of your model. It can automatically manage multiple model resources.

Conclusion ML Kit is one of the most interesting innovations of Google. The announcement of it had added a lot of excitement among the Android & iOS developers and is cool and fun to use too. It is highly recommended to start using ML Kit & play with its features to find your way out and check how it will be beneficial to you. Click Here if you want to get started right away.

9series is known for providing custom mobile app development services India. If you are looking for any such services then you have come to the right place. We have experts who will be able to provide you with the best solutions for your business using the ML Kit.  In one of our post, we also spoke about What is Core ML and What are its advantages & disadvantages.

In today’s fast-moving information technologies in the world, it is becoming more to build an application in intelligent way process to tackle the user based information. Artificial Intelligence is quickly becoming an essential tool in software development. Machine language is an easier way to identify user data process.

In this Post, we will look at the ML kit Mobile SDK that brings all Google’s expertise on machine learning techniques to mobile developers in an easy-to-use package. We will look at the various APIs offered by the SDK, and then we will take one of the APIs on a test drive by creating an Android application that makes use of it.

Google I/O 2018 developer conference introduced the ML Kit (Machine Learning Kit). It is a new software development SDK the for the developer of Android and iOS. Let’s learn about what the new API and what they are providing to recognize the various objects like barcode scanning, text recognition, image labeling, face detection, and landmark recognition.

Let’s Nurture - A Leading Mobile app development company is giving an overview of each API of ML Kit SDK offers after that we can look how can we use one of the API in Android Application Development.

1) Recognize Text:

Using ML Kit’s text recognition APIs, your application can recognize text in any Latin-based language and more using the cloud-based API). It will detect text from credit cards, receipts, and business cards. Using cloud-based API, it will also extract text from the document. Also, the application can track the real-world objects like reading the numbers on trains.

2) Detect Faces:

ML Kit’s face detection API, it can detect faces in an image and it will get the contours of detected faces. It will return the position, size, and orientation for any detected faces from visual media like digital image and video. You can also get the information about landmark and classification. Landmarks are points of interest within face such as left eye, right eye, note base, bottom mouth, etc. Classification is determined certain facial characteristics.

3) Scan Barcodes:

ML Kit’s barcode scanning API, it will read data encoded using most standard barcode formats. It is available on-device and supports the following barcode formats:

1D barcodes: EAN-13, EAN-8, UPC-A, UPC-E, Code-39, Code-93, Code-128, ITF, Codabar

2D barcodes: QR Code, Data Matrix, PDF-417, AZTEC

ML Kit can automatically recognize and parse this data, your app can respond intelligently when a user scans a barcode.

4) Label Images:

Image labeling APIs can recognize entities in an image. The API returns a list of recognized entities that were recognized people, things places, activities, and so on. Each label found comes with a score that indicates the confidence the ML model has in its relevance. Image label API available in an on-device API or a cloud-based API.

5) Recognize Landmarks:

The landmark recognition API detect well-known landmarks in an image. From the image, it will detect landmark’s geographic coordinates and the region of the landmark was found along with reading the metadata information. Landmark recognize which is available on cloud-based API.

Now, let’s move to the coding part by making example of Face Detection API of ML Kit.

First, you need to create and add your project in Firebase Console. Then put your google-services.json file in your project’s app module.

Now, add below code in your build.gradle (Module:app):

implementation ‘com.google.firebase:firebase-ml-vision:18.0.1’

Add followig code to initialise FirebaseVisionFaceDetectorOptions,

FirebaseVisionFaceDetectorOptions options =

new FirebaseVisionFaceDetectorOptions.Builder() .setClassificationMode(FirebaseVisionFaceDetectorOptions.ALL_CLASSIFICATIONS)

.build();

FirebaseVisionFaceDetector detector = FirebaseVision.getInstance().getVisionFaceDetector(options);

Add following code to initialise CameraSource and to start image recognisation,

private CameraSource cameraSource = cameraSource = new CameraSource(this, graphicOverlay);

cameraSource.setMachineLearningFrameProcessor(new FaceDetectionProcessor());

By running the application you will get the following response:

Wooahhaa, your app is a detects your inner beauty happiness. :)

In Google I/O‘18 we saw the introduction of Firebase MLKit, a piece of the Firebase suite that tends to give our apps the power to support intelligent features without breaking a sweat. The SDK presently comes with a group of predefined capabilities that are commonly required in applications you will be able to implement these in your application regardless of whether you know machine learning or not.

Image Recognition: In The Context Of ML

The ability of software to distinguish objects, places, people, writing and actions in pictures. PCs can utilize machine vision advancements, together with a camera and artificial intelligence software, to achieve image recognition. It’s accustomed to performing a large number of machine-based visual tasks, like labeling the content of images with meta-tags.

Now, what Firebase ML Kit offers to us is already possible to implement yourself using various machine-learning technologies.

The thing with Firebase ML is that and also offering these abilities underneath a type of wrapper, it like these innovations and offers their abilities within a solitary SDK.

In light of these, it tends to be hard to know where to begin. This is one of the principal objectives of Firebase ML Kit to make Machine Learning to our Android and iOS applications more accessible to developers and available in more applications. Right now ML Kit offers the capacity to:

Image Labeling to classify common elements in pictures.

Text Recognition to process and recognize text from pictures.

Face Detection to help you know if a face is smiling, tilted, or frowning in pictures.

Barcode Scanning to read data encoded in standard barcode formats like QR Codes.

Landmark Recognition to identify popular places in images.

Present and Future Uses of Image Recognition

Organization of Pictures

A popular use of image recognition is the automated organization of photograph collections. Have you ever traveled to another nation and wound up with many pictures stored on your phone? Google Photos is an extraordinary case of such an application to store pictures. It helps you organize your photos in albums by identifying common places, objects, friends or even pets.

Image Labeling on Social Networks

You might be familiar with face recognition in Facebook and Instagram. Whenever a user uploads a photograph, Facebook promptly proposes tagging some of your friends. Other than the tagging feature, image recognition translates content for visually impaired people with screen readers. It additionally perceives improper or hostile pictures.

Since you have some background on the possible use cases for image recognition, it’s time to learn about On-Device and On-Cloud APIs in Firebase.

Which is better? On-Device or On-Cloud

On-Device APIs can process information rapidly without the requirement for an Internet connection. This is useful if you don’t want to consume the mobile data of your users and you require quick processing.

The principal disadvantage is the certainty of results given by ML. The confidence is a value showing how happy the ML algorithm is with the answer is provided.

On-Cloud APIs offer significantly more powerful processing capabilities because of Google Cloud Platform’s ML innovation, yet these APIs require an Internet connection with work. On account of utilizing the Google Cloud Platform, this requires a payment after the initial 1,000 requests.

Now let’s talk about how to implement image recognition in your current application.

Setting Up Firebase and ML Kit using Kotlin

ML Kit is a subset of Firebase and it is very to configure, you just need to add your app to Firebase service.

Now create new android project and give name it like ML Kit Demo and add below dependencies to your app level build.gradle file in the dependencies block:

implementation ‘com.google.firebase:firebase-core:16.0.1’ implementation‘com.google.firebase:firebase-ml-vision:16.0.0’ implementation‘com.google.firebase:firebase-ml-vision-image-label-model:15.0.0’

Sync your gradle file to ensure everything is working.

Now we are following one of the popular food models of ML Kit so we will create an app in which we capture a photo and it will tell us whether it is food or not.

First, you need to set up the camera and for that, you need to handle runtime permissions, storage location, file formats and much more. Fortunately, there are many Android libraries that can help you in handling all those complexities via an easier API.

We use one sdk named ParaCamera to configure runtime permissions. To use it, open your app level build.gradle and add following dependency in dependencies section:

implementation ‘com.mindorks:paracamera:0.2.2’

Sync your files and open the MainActivity.kt file and add the following just above onCreate() method:

private lateinit var camera: Camera private val PERMISSION_REQUEST_CODE = 1

Add the following code in the onCreate():

camera = Camera.Builder() .resetToCorrectOrientation(true)//1 .setTakePhotoRequestCode(Camera.REQUEST_TAKE_PHOTO)//2 .setDirectory(“pics”)//3 .setName(“delicious_${System.currentTimeMillis()}”)//4 .setImageFormat(Camera.IMAGE_JPEG)//5 .setCompression(75)//6 .build(this)

Taking each commented section in turn:

Rotates the camera bitmap to the correct orientation from meta data.

Sets the request code for your onActivityResult() method.

Sets the directory in which your pictures will be saved.

Sets the name of each picture taken according to the system time.

Sets the image format to JPEG.

Sets a compression rate of 75% to use less system resources.

Now, add the takePicture() method as below:

fun takePicture(view: View) { if (!hasPermission(android.Manifest.permission.WRITE_EXTERNAL_STORAGE) || !hasPermission(android.Manifest.permission.CAMERA)) { // If do not have permissions then request it requestPermissions() } else { // else all permissions granted, go ahead and take a picture using camera try { camera.takePicture() } catch (e: Exception) { // Show a toast for exception Toast.makeText(this.applicationContext, getString(R.string.error_taking_picture), Toast.LENGTH_SHORT).show() } } }

Next you need to take permission so add requestPermissions() method:

private fun requestPermissions(){ if (ActivityCompat.shouldShowRequestPermissionRationale(this, android.Manifest.permission.WRITE_EXTERNAL_STORAGE)) { mainLayout.snack(getString(R.string.permission_message), Snackbar.LENGTH_INDEFINITE) { action(getString(R.string.OK)) { ActivityCompat.requestPermissions(this, arrayOf(android.Manifest.permission.WRITE_EXTERNAL_STORAGE, android.Manifest.permission.CAMERA), PERMISSION_REQUEST_CODE) } } } else { ActivityCompat.requestPermissions(this, arrayOf(android.Manifest.permission.WRITE_EXTERNAL_STORAGE, android.Manifest.permission.CAMERA), PERMISSION_REQUEST_CODE) return } }

requestPermissions() asks for the CAMERA and WRITE_EXTERNAL_STORAGE permissions. If the user rejected those permissions previously, it will display a nice snack bar with a message.

Now, implement onRequestPermissionsResult() method,

override fun onRequestPermissionsResult(requestCode: Int, permissions: Array<String>, grantResults: IntArray) { when (requestCode) { PERMISSION_REQUEST_CODE -> { // If request is cancelled, the result arrays are empty. if (grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED && grantResults[1] == PackageManager.PERMISSION_GRANTED) { try { camera.takePicture() } catch (e: Exception) { Toast.makeText(this.applicationContext, getString(R.string.error_taking_picture), Toast.LENGTH_SHORT).show() } } return } } }

Above code calls the method takePicture() when user granted all the permission to use camera. Now you need to handle activity result in onActivityResult() and set image in your image view,

override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) { super.onActivityResult(requestCode, resultCode, data) if (resultCode == Activity.RESULT_OK) { if (requestCode == Camera.REQUEST_TAKE_PHOTO) { val bitmap = camera.cameraBitmap if (bitmap != null) { imageView.setImageBitmap(bitmap) detectDeliciousFoodOnDevice(bitmap) } else { Toast.makeText(this.applicationContext, getString(R.string.picture_not_taken), Toast.LENGTH_SHORT).show() } } } }

That’s all, now build and run the app and you will see:

Hurray, your app is a toast detecting marvel!

How Let’s Nurture helps for building Apps using ML Kit for Image Recognition?

Let’s Nurture, a top mobile app development company in India has got the expertise in providing many solutions based applications with E-Commerce and Social Networking.