Create TensorFlow Image Detection In Angular 9

In this tutorial, we will see how to Create TensorFlow Image Detection In Angular 9. Creating a small functionality like an AI – Image Detection becomes so easy by using the TensorFlow modules.

TensorFlow can be used in the web application by using the JS library of the TensorFlow. For this Demonstration, we will be using the Image Classification module.

TensorFlow Image Detection With Image Classification Module

Using Image Classification Module we can detect any image containing people, activities, animals, plants, and places. We will use this module in the Angular 9 application.

let me show you now can we implement that in angular 9 application.

Install TensorFlow Packages

First, all we need to do is install all the required packages using the npm tool.
The following are the list of packages:

• @tensorflow/tfjs: TensorFlow JS
• @tensorflow-models/mobilenet: TensorFlow Mobilenet Module (Image Classification)
• @tensorflow/tfjs-node: TensorFlow backend Via NodeJS

Install Via NPM

# Main Dependencies
npm i @tensorflow/tfjs @tensorflow-models/mobilenet @tensorflow/tfjs-node --save
# Peer Dependencies
npm i events node-fetch string_decoder --save

...
"dependencies" : {
  ...
  "@tensorflow-models/mobilenet": "2.0.4",
  "@tensorflow/tfjs-converter": "1.2.11",
  "@tensorflow/tfjs": "2.0.0",
  "events": "^3.1.0",
  "node-fetch": "^2.6.0",
  "string_decoder": "^1.3.0",
  ...
}

TensorFlow Image Detection With Image

Detecting objects that can be done in two different ways i.e. via image or via webcam. Lets first cover by uploading image.

import { Component, VERSION, ViewChild, ElementRef } from '@angular/core';
import * as mobilenet from '@tensorflow-models/mobilenet';

export interface Prediction {
   className: string;
   probability: number;
}

...
export class AppComponent  {
 public model: any;
 public loading = true;
 public imageSrc: string;
 public predictions: Prediction[];
 @ViewChild('img') public imageEl: ElementRef;

 public async ngOnInit(): Promise<void> {
   this.model = await mobilenet.load();
   this.loading = false;
 }

 public async fileChangeEvent(event): Promise<void> {
   if (event.target.files && event.target.files[0]) {
     const reader = new FileReader();
     reader.readAsDataURL(event.target.files[0]);
     reader.onload = (res: any) => {
       this.imageSrc = res.target.result;
       setTimeout(async () => {
         const imgEl = this.imageEl.nativeElement;
         this.predictions = await this.model.classify(imgEl);
       }, 0);
     };
   }
 }
}

basically we have loaded a module mobilenet.load() on a component init and on image selection, we passed the image to classify(imgEl) method which gives us the predictions.
and from html we have to just accept file and display data, for example

<input type="file" (change)="fileChangeEvent($event)">
...
<tr  *ngFor="let prediction of predictions">
  <td>{{prediction.className}}</td>
  <td>{{prediction.probability}}</td>
</tr>

TensorFlow Image Detection With Webcam

For object detection via video/webcam the same approach can be considered instead of input field we will display a video.

import { Component, OnInit, AfterViewInit, ElementRef, ViewChild } from '@angular/core';
import * as mobilenet from '@tensorflow-models/mobilenet';

export class VideoDetectionComponent implements OnInit, AfterViewInit {
  public model: any;
  public loading = true;
  public predictions: Prediction[];
  @ViewChild('video') public video: ElementRef;
  constructor() { }

  public async ngOnInit(): Promise<void> {
    this.model = await mobilenet.load();
    this.loading = false;
    setInterval(async () => {
      this.predictions = await this.model.classify(this.video.nativeElement);
    }, 3000);
  }

  public async ngAfterViewInit(): Promise<void> {
    const vid = this.video.nativeElement;
    if (navigator.mediaDevices.getUserMedia) {
      navigator.mediaDevices.getUserMedia({video: true}).then((stream) => {
        vid.srcObject = stream;
      })
      .catch((error) => {console.log(error);});
    }
  }
}

As we load and assigned elements for image the same approach will be applicable for the video elements.
and from html to show video and display data, for example:

<video autoplay playsinline muted id="webcam" width="300" height="300" #video></video>
...
<tr  *ngFor="let prediction of predictions">
  <td>{{prediction.className}}</td>
  <td>{{prediction.probability}}</td>
</tr>

Demo