The Windows Club

Search, Edit and Recreate Videos with Video Breakdown from Microsoft Garage

Video Breakdown from Microsoft Garage is a video service that recreates search, editing and sharing moments. It activates a series of services & APIs to extract cognitive insights from the video content. Let us take a look at it.

Video streaming capabilities are generally designed around three concepts

  1. Quality
  2. Security
  3. Flexibility

The streaming capabilities haven’t changed much in recent years but cognitive abilities have undergone significant changes and evolved with time, dramatically. It was for this reason, Microsoft Garage came up with the idea of Video Breakdown.

Video Breakdown from Microsoft Garage

Video Breakdown is a cognitive video indexing platform that creates an index of the content within the videos users upload using Microsoft Cognitive Services and Azure Media Analytics.

The working mechanism of the web app is completely different from how we perform search action currently, where video content is sorted by channels or metadata – and very subjective manual tagging. Video indexing intends to change all this in one go. It looks to develop more innovative ways via which videos are searched, edited, played and shared.

The tool extracts a linguistic transcript and OCR. It then detects and identifies faces and extracts keywords and topics, analyzes sentiment, classifies voice activity and do more.

After that, based on this index, Video Breakdown users can perform any functions such as search, explore, understand, editing videos and playlists in a powerful, simple, intelligent manner.

How to search a video using Video Indexing Web App

When you first go to the main page, sign-in using your Outlook account. Alternatively, you can login using your Facebook or Google account also.

After you upload a video file to the platform, its contents undergo a thorough analysis through a series of Microsoft Cognitive Services and Azure Media Analytics.

The analysis of contents leads to the production of the following,

  1. An audio transcript
  2. Face tracking
  3. Grouping and identification
  4. Differentiation of speakers
  5. Optical character recognition
  6. Extracts sentiments and topics

All of these pieces in combination generates an index using automated transcript or OCR that enables searching of a keynote for the precise moment where a keyword was mentioned in the speech.

In addition to the above, the platform employs recognition on several fronts which include computer vision, language understanding, speech to text, face tracking, linguistic analysis, text analytics, and identification and image search.

Apart from the above i.e., performing search operations, Video Indexing platform also would also help you create videos by stitching together video segments or creating your own compilation of any video from the gallery, adding dynamic video segments to it and republishing them as new videos or share them on social networks.

Read more about it at Microsoft and try out Video Breakdown here.