From AI to computer vision: How cult.fit stepped up to deliver holistic solutions post Covid-19

Covid-19 locked us in our homes and shut gyms down, amongst other things. But here's a company that stepped up its game to change things for the better, virtually and otherwise.

Ankit Gupta, Product and Engineering lead at cult.fit (cult.fit)

If you are somebody who cares about fitness at all or ended up paying for a year's membership at your local gym, you would remember the sense of utter frustration when Covid-19 shut it all down. Not only could we not go workout, the parks, and other recreation spots were also shut. Almost overnight, if we wanted, or needed, to work out, our homes became the only safe place and smartphones/tabs our only companions. 

For a company like cult.fit, this was a chance to enhance their offerings so as they could virtually cater to their existing customer base. But what also happened in all this was that cult.fit managed to unlock a massive market that was otherwise beyond the reach of their centers. And driving all of this forward was technology like computer vision and artificial intelligence (AI). We spoke to Ankit Gupta, Product and Engineering lead at cult.fit, to understand how things happened. And here are some excerpts from our chat:

Going back by a year what are the initial things that you almost immediately took on for cult.fit once the lockdown happened?

Ankit Gupta: Rewinding back to March of last year (2020), even though we are a holistic health care company - one of our biggest businesses was our own cult.fit centers where people used to come and work out in groups led by a trainer. Overnight we had to shut down these centers because of how the pandemic spread. So our initial thought was that we have many lakhs of customers who are coming to our gyms, how will we keep them active during this period because fitness is something that if you lose out on, then it's very difficult to come back and pick it up again. Hence, we decided to fulfill people's fitness needs completely digitally.

One key effort in this was scaling our live fitness classes. This is something that we had been working on for almost half a year or so already and that helped us out. Initially, that idea was just to keep our user base active and make sure that their fitness needs are met but quickly we figured out that we had unlocked a huge market that we were not addressing earlier because of our centers.

These were people for whom our centers were not accessible before. Perhaps the timings did not match their convenience, a lot of people are also shy when it comes to gyms, and there is also the pricing issue. But overnight we were able to see that we have unlocked almost 5X the user base that we were not addressing before and all of them had fitness needs. But, now that parks were shut and people were not able to go for jogs or play sports, we saw a huge amount of traffic on our app because our brand has always been well-known when it comes to fitness, plus we had also initially made the classes free. We also introduced a lot of formats that earlier weren't even happening in our centers. This became like a product that was just waiting to be launched.

On the tech side, we were definitely not initially ready for the traffic that hit us. So within five or six days, our systems had to be scaled up to manage 50 times the traffic than what we were used to because initially, these classes were completely free. Later on, we came up with a paid offering on this front, but essentially we created a holistic digital health offering that had live classes, digital meditation, recipes people could follow and cook at their home, online doctor consultations because all the hospitals, clinics, etc, a lot of them shut down and people had a real need of talking to doctors, and so on. And this is where our tech DNA helped us out and we came up with all of these digital health offerings within a month or so.

Did you see a lot more people signing up because now that your classes were online and much cheaper?

Yes, definitely. Like with any other product pricing is a big factor in determining how many people are able to consume it. And even though our cult centers are mass products, it comes with real costs of rent and salaries of employees, and there are costs that go into running a business that is high scale. I think because the digital offering was much more economical when it comes to pricing we saw a lot of people who were not able to consume our fitness offerings before from Tier 2 cities, and even from Tier 1 cities, join us on our app. We unlocked a huge market. The digital offering, just by the way of being digital, is much more scalable and economical.

Even accessibility is huge here. If we tell anyone to work out in a center that is 5 kilometers from their home, it may not happen because it's a daily thing that you have to do. It's not like a restaurant you can go to once in a while, spend one hour and then not think about it. Fitness is a daily habit that one has to inculcate, which means convenience plays a very big factor. Hence, distance, timing, pricing – these are all part of convenience. Because our live classes are completely digital, you don't have to worry about when or where to work out and also about what you should wear while doing it. It's at your convenience that you can work out – which is, as I said, the biggest factor for most.

You mentioned that you guys brought in some new offerings once you went online, so could you tell me a bit more about those? And did you have to adopt any specific kind of technology to facilitate that?

So, I'll talk about a couple of offerings that we started. The first one is live classes which are right now our biggest digital fitness offering. Typically, if you look at any kind of digital fitness on YouTube or Instagram, there is zero aspect of interactivity or feedback from the product. We wanted to make these digital classes truly live and interactive. 

So we adopted technologies and algorithms from the field of computer vision where basically a user who has fitness needs can work out in front of the phone or laptop and our technology looks at them and also gives real-time feedback on how much energy they are spending, how their form is, how they compare against a lot of our other users, how they compare against themselves from one week back, etc. This becomes a much more enjoyable experience rather than just following a video. 

The second was machine learning. These algorithms are very computationally expensive and take a lot of computational power. Because we were coming up with a mass product, we had to make sure that all of the tech can run on low-scale phones or low-end devices too. So there's a huge amount of innovation that went into making sure that we are not consuming a lot of battery and all of our algorithms work even on low-end devices. This is something we had to do which made it truly distinguishable as compared to our competitors or traditional digital fitness mediums.

We also launched a product called Online Personal Training. A lot of people actually have personal training needs. Till last year the only way to do it was to hire a personal trainer at a gym or one who comes to our homes. With Covid-19 this was no longer possible because human interaction became very limited. So we launched a digital or online personal training product where based on your needs you are connected to an actual human being on the other side who engages with you for one hour through a video medium and in real-time the trainer is able to look at you, correct your postures, answer your questions, motivate you, ask you to cool down, etc. It's truly a personal and a human product. So again here also we had to figure out how to do this at scale because on one hand we had all of our trainers from our centers who could pitch in and help us out on this product and at the same time there are a large number of people with personal training needs. By using video conferencing at scale, we had to make this product happen. And even today this is running and solving for people with personal fitness needs.

On the doctors' side as well, our clinics had to shut down because of the pandemic. Thus, we made our doctor teleconsultation product completely pan-India. This also happened in just one week and anyone in India with a need to talk to a doctor for any reason, whether preventive or curative or diagnostic, could get on a call – again a video call - with any of our doctors from any specialty and we could do a 15- or 30-minute consultation to address those medical needs, prescribe medicine, and even prescribe tests which are available on the app at any time.

Since you use computer vision to follow a person's workout and give real-time feedback, use personal training, and consultations, how do you keep things in check from a privacy perspective? Was it a problem convincing people to keep their cameras on when they are working out? 

In the case of live classes, there are two points. First of all, it's an opt-in feature. You can do a live class even with your camera off. Obviously, the interactivity element will get limited if your camera is off, but it remains an opt-in feature. So if you are concerned about it you can completely choose to keep your camera off. 

Second, none of the video feed actually comes to our servers. All of these computer vision algorithms are run on the user's phone. They don't come to our servers - only the calculations that have happened do. So let's say our app on your phone thinks this person has spent X energy or has burned so many calories – only this information comes to us rather than the actual video feed. We have always communicated this and this has been true from day one even before Covid-19 happened. When we were working on this feature or on this product we always knew that privacy would be a concern. As a health company, we take privacy very seriously and the video feed is something that is truly private. Hence we have done everything from our end to ensure that none of this video actually comes to us.

On the personal training front as well we have always used third-party solutions like Zoom or Twilio where the feed actually does not come to us. We have always made sure that is always the case across such products. Whenever our app is actually looking at the user only the aggregate information or the result of some algorithm comes to our servers which enables us to give reports to people.

So to sum up, we do two things. First of all, we make sure nothing comes to us, and second, all of these things are still opt-in so if you are talking to a doctor, you can choose to keep your camera off. In the case of personal training, obviously the utility of the product is better when the trainer can look at you in real-time, but even in that case, the live feed does not come to us.

What about the other offerings you mentioned, like consulting a doctor? How do you ensure the privacy of it all from a user endpoint?

First of all, we adhere to the standards. We are also undergoing a more rigorous certification so that we can also put those standards against our products. Also, from our end whatever information the doctor enters - such as the diagnosis, the prescription, or anything else that they have discusses with the customer - all of this is encrypted. Internally we make sure that only the doctor, nutritionist, or the therapist has the information - no other doctor and no other internal employee has access to that information. But even then we keep this encrypted. So only the user and the expert who is your personal doctor or personal nutritionist can actually decrypt that information. We have always implemented this both online and offline. Even in our clinics when doctors would sit in front of you and enter all of this information into their system – it would always be encrypted and hence remain safe.

Speaking about these doctors or these medical offerings that you have started offering over the last year, did you see a certain spike in certain kinds of consultations?

Yes, definitely. Overall we saw an increase in all of our specialties. Because a lot of clinics and a lot of small hospitals had to shut down we saw an increase in all the specialties – ortho, cardiac, etc. But we definitely saw a more-than-usual spike in two of our offerings. One was mental therapy and the second was Covid-related consultations or respiratory-related consultations. 

With both of these, it is clear why there had been a spike. With home quarantine and lockdowns, everyone has different mental wellness needs, which for a lot of people actually got compounded. The stress increased and social isolation increased. Under normal circumstances before Covid-19, we could talk to friends, family, colleagues, etc. That was therapeutic. But all of that decreased so the human element switched off almost overnight in everyone's lives and these mental health needs became amplified.

Even though we always had a mental therapy offering, over the last 2-3 years this saw a huge spike. To cater to this, from our supply point of view, we got more therapists on board to make sure that we are able to service all mental therapy needs.

Besides these what are the other medical offerings that you have on the app? What are the other things that you offer right now besides fitness and mental health?

We have a lot of Wellness offerings as well. First of all, we have nutrition. For anyone who has undergone weight loss or any kind of physical training or any issue - food plays the biggest part. We have nutrition consulting on the app where you can go talk to a nutritionist, again via a video medium, and get a customised meal plan to address your needs.

Apart from that we also have diagnostic offerings on the app where you can book an appointment with us for any kind of a health checkup and someone comes to your doorstep, collects the samples and you can look at the report right there on the app.

We also have launched a new feature called an AI fitness planner where if you have a fitness need, you can talk to the app. The app asks you a lot of questions about what your fitness needs are or how advanced you are in your journey, what your weight is, what your BMI is, etc. Then it creates a customised fitness routine or a fitness plan for you, which you can either fulfill in your gym or in your home.

We have just launched a diabetes reversal product as well. In India diabetes is a huge problem. According to studies, one out of every seven Indians has either diabetes or prediabetes. We have just launched our product, sugar.fit, which looks at your lifestyle, metabolic profile and again ties you up with diabetologists and lifestyle coaches to understand and service your needs.

We have also launched one more product called Transform. Right now it is in public beta, so it's not open to all because it's very new. However, the idea here is that if you have weight loss needs or if you have a particular goal in mind that you want to work towards, you can buy this product and we tie you up with a human coach who understands your body, your mind, your motivation, and who works with you every day till you achieve that goal. So there are a lot of offerings on the Wellness side which are live on the app now, apart from just live classes and personal training.

You talked about an AI creating a training program for you. How customisable are your questions if somebody has a medical complication or who is recovering from Covid or who maybe has a back problem?

With any AI product you start small first. You take, let's say 10 questions, then you add more and more features. This is what we have done as well. Today the AI coach understands most of your macro attributes - what your weight is, what your height is, what is your goal? Are you looking for weight loss, building muscle, or improving endurance? It also asks you about your injuries and tries to filter out and adapt your workout plan according to those needs. But there are still some things that it does not understand today. But as with any AI it will become more and more intelligent.

Today it understands whether you are someone who can work out daily or who can work out only on weekdays or three times a week. It also understands what your past performance has been. Moreover, let's say it creates a plan for you and as a part of the plan it told you that you should first start with a 15 minute run on a treadmill. It will ask you at the end of the workout for feedback. If you tell the AI that it was too tough for you, it will, in the next workout, modify that aspect. Or if it told you to use 20 kg dumbbells and if that was too tough for you, the next time it will tell you to take 10 kg dumbbells and pull lesser reps. 

So while there are some things it understands, there is also room for it to grow more intelligent. This is our first version, or V1, that we just launched a month ago. It understands first of all, how far along the fitness journey you are in, what your motivation is, whether you like to work out in the morning or in the evening, whether you want to work out on the weekdays or on the weekend, etc. But there are some things it will not understand today but with more input and more work, we will be able to personalise it further.

Is your AI equipped to understand that there is a medical problem somewhere and therefore perhaps recommend consultation with a doctor?

Right now I think it's slightly limited. For instance, if you have a heart issue it will actually not create a plan for you because this is an extreme case. It's better to be safe. So it will at that point not create a plan for you and ask you to talk to a doctor or talk to a specialist first. But after a point, because we have a cardiologist on board and we have doctors on board, the AI will be able to refer you to a human on our platform itself. Right now we are still in the early stages so it won't do this yet but it is a work-in-progress.

So to sum it up, we definitely don't build these things in isolation. The true power of a holistic platform is when we understand your nutrition, your mental well-being needs, what your sleep patterns are and then all of these things should work together rather than independently.