To take on Google Search, Microsoft joins IIT Kharagpur to create ‘deeper’ search engine

In a bid to take on Google Search, Microsoft said on Monday it is working with professors from the Indian Institute of Technology (IIT)-Kharagpur towards developing a system that can form the basis for a deeper, more meaningful search engine.

Bing is a web search engine owned and operated by Microsoft.

While current search engine algorithms are great at working with fact-based queries and providing structured answers, they are surprisingly ineffective at answering subjective and personal questions.

Queries based on human experiences and personal opinions are difficult for a standard search engine to comprehend.

Therefore, they fail to answer questions such as "How to make small talk with new friends," "People's favourite memories from school," "How does it feel to immigrate to a new country?" or "The songs that defined the 80s."

Microsoft's Senior Applied Researcher Manish Gupta partnered with professors from IIT Kharagpur to conduct a study on extracting meaningful information from social conversations to help search engines answer social list queries better by deploying artificial intelligence and machine learning.

The team used multi-word hashtags, also called "idioms" from Twitter, to conduct an in-depth study to make search results much more personal and human.

"While traditional search engines may struggle with such deeply human queries there are online platforms specifically tailored for personal opinions and conversations -- social media. Twitter, specifically, has become a forum for people to create sustained online conversations held together by a common hashtag," Gupta said.

The researchers collected around four million hashtags that were trending between January 2015 and June 2015, and used a SVM (Support Vector Machine) classifier to conduct this research.

The classifier worked on deeply personal and human hashtags such as #foreveralone, #awkwardcompanynames, #childhoodfeels, and #africanproblems using factors such as duration of hashtag popularity, related hashtags and URLs to detect context and classify the social lists accurately.

The algorithm used to conduct this study forms the basis for a better search engine for social platforms which can assist users looking for subjective information and trusted opinions, Microsoft said.

The study eventually proved that, by analysing multi-word hashtags search engines can now scour social networks and detect valuable insights from public conversations, using a highly effective and precise algorithm to provide a deeper, more meaningful search experience.