In conversation with Debdoot Mukherjee, the AI man at ShareChat

Localisation has been a mainstream subject for internet companies like Google, Facebook for over a decade, with most tech giants striving to perfect that elusive formula while importing a product built essentially for one geography and being sold in geographies all over the world. Most multinationals have experimented with multiple models of localisation but without much cogent success. 

ShareChat, a home-grown multi-language social media platform, has gained huge traction with its model of selling a ‘local’ product, something that has come to everyone’s attention just recently. 

Just look around, Chingari, Trell, Moj and many more have become centre of attraction. While some of the Indian applications have capitalised on the vacuum created by TikTok’s departure in India, ShareChat has had the edge with a longer stay in the game. 

A hyper-local model saw an immediate measure of tangible success, with users and downloads swelling by the month. So, what was it that ShareChat did right? 

ShareChat ignored English and built a completely non-English social media platform targeting people who were more comfortable in their native languages, most of them being first-time internet users. 

ALSO READ: ShareChat acquires video production company HPF Films

“ShareChat was the first social media platform to understand the needs of regional India. The platform was built with a vision to make the internet experience comfortable for the people who prefer conversations in their own mother tongues,” said Debdoot Mukherjee, Vice President - AI, ShareChat

However, building an internet platform from ground zero and scaling it up to a platform of choice for millions of language-first users was not an easy task, says Mukherjee. 

While English-only platforms have traditionally addressed one language, ShareChat built a platform targeting various Indian languages. “This was the real challenge given that there were references available for Indian languages”, Mukherjee added.

That created another problem. How to deliver content to the users in their preferred language? ShareChat started building an AI (artificial intelligence) framework to address the problem.

“We believe that for India to become truly digital, we need to make technology accessible and productive for all.  As we aim to serve the first time internet users, building a robust AI framework for Indian languages was the first foundation towards the growth,” said Mukherjee.

Content genres on ShareChat mostly revolve around images and short videos. The requirement, therefore, was to design AI capabilities that only did not just understand the text but also visual and audio data--essentially this means that AI models need to interpret content in the same way as humans do.

For instance, “Purani Yaadein” is a popular tag on ShareChat. ShareChat’s AI model succeeded in discerning whether a video, text or audio with this tag has a reference to some sense of nostalgia in order to figure out whether it would be relevant for this tag.

Mukherjee’s team started building an NLP (Natural Language Processing) for Indian languages. Though the process was similar to interpreting English, the team brought in few important differentiation to counter the challenges of making it work in a low resource set up, since most Indian languages have a small fraction of data in digitized formats as compared to English, Mandarin and other European languages. 

“A key ingredient in these models lies in the concept of Transfer Learning – the ability of models trained to interpret semantics on a language where data is abundant (e.g., Hindi) to recognize semantics of text in other languages with little data (e.g., Oriya, Assamese).” he elaborated further while explaining the differentiations.

Even though he has been driving ShareChat’s AI and data science vision, he also feels the entire Indian startup ecosystem is getting creative with their AI framework - be it social media, e-commerce, edu-tech etc.

According to Mukherjee, “Every startup needs to find what the virtuous cycle of AI looks like for them – one where the core user experience is improved by AI, which leads to more adoption of the product and that in turn leads to continuous improvement in the quality of the models with more data being collected.” 

While the Indian startup ecosystem is maturing in the AI landscape, the world is catching up with the GPT3 model. GPT3 is the third generation language prediction model, created by OpenAI with a capability of 175 billion machine learning parameters.

Although optimistic, Mukherjee remains equally cautious. He said, “The exact capabilities of the model and the possibilities that lay ahead will only become clear when OpenAI publishes details on the dataset that was used to train the model. Until that happens, we can’t be sure what patterns have been memorized by the model and how effectively it can generalize to perform on new tasks.”

At ShareChat, he has done some GPT experimentation too. 

The experimentation started with an internal team session on how can AI be taught to be creative? Developing AI for artistic tasks has seen a lot of momentum in recent times but Debdoot’s team could not find examples of doing this for Indian languages.  

His team decided Shayari to be the subject of this exploration. The team investigated Hindi Shayari using GPT2 model and used Reinforcement Learning to increase fluency, coherence, and meaningfulness of the generated poems by designing rewards focused to enhance each of these dimensions.  

Debdoot said, “It was a toy project to explore the possibilities for AI-driven creative content generation. It allowed us to deepen our understanding of how AI models can be tuned to infuse greater fluency and coherence for generating poetry in Indian languages.” 

ShareChat, the platform is speeding ahead with an equally ruthless momentum. The platform today clocks over 160 monthly active users. Moj, the newly launched short video format by ShareChat, has over 80 million active users.