On this episode, Ankur Mehrotra dives deep into Amazon Personalize, a fully managed machine learning service that goes beyond static, rule-based recommendation systems and trains, tunes, and deploys custom ML models. Ankur explains how it allows for delivery of highly customized recommendations to customers across industries, how Personalize can drive meaningful business results, and much more.
On this episode, Ankur Mehrotra dives deep into Amazon Personalize, a fully managed machine learning service that goes beyond static, rule-based recommendation systems and trains, tunes, and deploys custom ML models. Ankur explains how it allows for delivery of highly customized recommendations to customers across industries, how Personalize can drive meaningful business results, and much more.
--------
“Consumer habits are changing and the way they're engaging with different businesses are changing. We're seeing the need for personalization or the need for creating better user experiences in digital channels increase and we've seen that happen across different verticals." - Ankur Mehrotra
--------
Time Stamps
* (04:31) Metrics for customers on AWS and how Amazon Personalize can impact their business
* (12:46) How personalization can directly impact business output
* (13:42) Measuring relevance cycle time with Amazon Personalize
* (16:34) Three pieces of advice to customers as they're getting started with Amazon Personalize
* (18:31) How to think about quantum of data
* (22:11) Recommendations to avoid cost overruns while using Amazon Personalize
* (26:36) Success stories with Amazon Personalize
--------
Sponsor
This podcast is presented by CloudFix
CloudFix automates easy, no risk AWS cost savings. So your team can focus on changing the world.
--------
Links
Connect with Ankur Mehrotra on LinkedIn
Connect with Rahul Subramaniam on LinkedIn
Speaker:
Hello and welcome to AWS insiders. On this podcast, we will uncover how today's tech leaders can stay ahead of the constantly evolving pace of innovation at AWS. You'll hear the secrets and strategies of Amazon's top product managers on how to reduce costs, and improve performance.
Speaker:
Today's episode features an interview with Ankur Mehrotra, director and general manager for AWS AI. On this episode, Ankur dives deep into Amazon Personalize, a fully managed machine learning service that goes beyond static, rule-based recommendation systems, and trains, tunes, and deploys custom ML models. It allows for delivery of highly customized recommendations to customers across industries. But before we get into it, here's a brief word from our sponsor.
Speaker:
This podcast is brought to you by Cloud fix. Ready to save money on your AWS bill? Cloud Fix finds and implements 100% safe, AWS recommended account fixes that can save you 10 to 20% on your AWS bill. Visit cloudfix.com for a free savings assessment.
Speaker:
And now here's your host, AWS super fan and CTO of ESW capital, Rahul Subramaniam.
Rahul Subramaniam:
So Ankur, welcome to the show, AWS Insiders. Really excited to talk today about AWS personalized. We'd like to understand a little bit about what you guys are doing with your customers with AWS Personalized. What os the problem that you're really trying to solve with it?
Ankur Mehrotra:
Thanks, Rahul. Thanks for having me. Yeah. Over the last few years, you can see that businesses are engaging with the users more and more through digital channels and that has only accelerated during the pandemic.
Ankur Mehrotra:
So every digital touch point provides businesses an opportunity to create an experience for the users that's more tailored to their needs. And that's where Amazon Personalized comes in. It's an AWS service that helps businesses create more personalized experiences for the users by creating recommendations that are powered by machine learning. They'll be seeing a lot of customer interest in adoption, and a lot of AWS customers are able to build a high performing personalization system using Amazon Personalized and they're seeing great results.
Rahul Subramaniam:
Great. And I completely understand the need for providing this personalized content. These days, we get inundated by choices, especially around all the digital channels and getting the right kind of recommendations for what customers want to consume is incredibly valuable. So what are the changes since the beginning of the pandemic that you're seeing in the kinds of use cases that customers are applying AWS Personalized?
Ankur Mehrotra:
As consumer habits are changing and the way they're engaging with different businesses are changing. We are seeing the need for personalization or the need for creating better user experiences in digital channels increase, and we're seeing that happen across different [inaudible 00:03:20 ]. So obviously as retail has been a great example, and also where Amazon really pioneered personalization in the late nineties. But now we are seeing the need in all different kinds of workloads.
Ankur Mehrotra:
For instance, in banking we are seeing customers or businesses trying to create more personalized experiences for the users. And we see even in other cases, for example, a very unique case that I will point out to you is, there's an app called Calm. It's an app for meditation and mindfulness. So they use Amazon Personalized to create recommendations for the users for mindfulness exercises. And they've told us how much impact that's created during the pandemic. These kind of stories really make our efforts work.
Rahul Subramaniam:
That sounds pretty interesting. If somebody were to start with Personalize, I think the thing that most businesses want to know is, how is going to impact my business and how do I measure impact of starting to use a service like Personalized? So how do you see customers measuring those metrics? What kind of metrics are they really looking at to measure the success?
Ankur Mehrotra:
So I think this really depends on the kind of business. For instance, in an online retail scenario, businesses are looking to improve conversion rates, improve revenue for order. So this is where if you can add the right kind of personalization at the right customer touch points... For example, at the right point in the customer journey on the retail website, if you can add recommendations that drive more up sales, then you can change, you can increase your average revenue per order. It just changes from business to business.
Ankur Mehrotra:
For example, in the media and entertainment space, if you have a, let's say video or on-demand application, that's where you may want to optimize for the video engagement and video watch time. And the great thing is that Amazon Personalize gives you an ability to optimize, to specify a metric, that you want to optimize your recommendations from.
Rahul Subramaniam:
So, as I understand it, there are basically three constructs or fundamental constructs of how you end up using AWS Personalized. There's the concept of the user, there are the items list that you have, and then a whole lot of events that you feed it. The events basically are interactions or impressions that the user has with those particular items.
Ankur Mehrotra:
That's right. So the way you can get started with Amazon Personalized is you have your interaction events, which is interactions between your users and items and these items can be products in your, if you are an online retail business, these can be products in your catalog. Or if you are, let's say in media and entertainment, this can be videos or titles in a catalog. So in the interaction dataset, which sometimes also sends a spark of click stream dataset, that the most important dataset that you need to start using Amazon Personalize to build a recommendation system.
Ankur Mehrotra:
Then if you have other metadata about your items or users. So for instance, if you have metadata such as product category or brand about the items in your catalog, then that is optional for you to use, but that can really improve the quality of your recommendations because Amazon Personalize is able to automatically understand trends between products that have similar metadata, for instance.
Ankur Mehrotra:
Then if you have any information metadata about your users, such as their location, or their preferences, age group, et cetera, that also helps. So when you start using Amazon Personalize, you take these three types of datasets, the interactions one being most important, and you can add it to Amazon Personalize, either through a set of APIs, by just making API calls, or by moving this data through an Amazon S3 bucket. And then once you've done that, you can use a set of APIs to then select your use case or the type of recommendations that you're trying to generate. At Reinvent recently we launched more use case specific recommenders for retail on retail and media entertainment. So if your use case falls within those two buckets, then you can really pick the domain specific use case within the service through the API.
Ankur Mehrotra:
So in retail, for example, you have frequently bought together. So you can directly pick the recommenders that map to your use case. And if your use case does not fall in the retail or M&E bucket, then you can pick the recommendation types that most closely match a use case. So for in instance, we have recommendation types such as, user level recommendations. If you're trying to generate a set of item recommendations for a particular user or a personalized ranking, if you have a set of items in mind, but you want to rank them differently for each user, you can select that what we call as recipe. If you want to create similar items product widget on a product detail page, then that recipe can be really useful.
Ankur Mehrotra:
Then recently we also launched a new type of recommendations called intelligent user segmentation, where we flip the problem. We say, instead of generating recommendations for a given user you can specify an item or let's say an item category or a genre, and generate a segment of users who may be interested in that item or genre. It can be used with different marketing tools, for email campaigns, ads, and even for other market [inaudible 00:09:17] that are upper in funnel or in the consideration.
Ankur Mehrotra:
So these are the different kind of recommendations that you can choose within Amazon Personalized and it can be done through a simple API call. And once you do that, then in the background, Amazon Personalized uses machine learning, trains a bunch of different machine learning models to create recommendations that optimize for these use cases.
Ankur Mehrotra:
Then you can also specify... You have all these knobs and dials that you can use to optimize or augment these recommendations. We see that our customers don't want to directly display recommendations that are coming out of the machine learning model, they want to specify specific rules and constraints on their recommendations.
Ankur Mehrotra:
For example, you may want to filter recommendations for a given product category or metrics. So you can do that easily through a filtering and business rules capability. It's super easy to retrieve the recommendations through a low latency API and integrate that into your application or website or whatever experience you're trying to create for your users. So that's basically the end to end flow. You start with this data, you can add it to the service through APIs. Then you can pick the type of recommendations you want to create.
Rahul Subramaniam:
Most folks who haven't been in the AWS ecosystem feel like if they were to get onto using any of the AWS services, they literally have to create these plans to migrate to the cloud and have these enormously long exercises of moving to the cloud. All you really need is to take your data, move it into an S3 bucket, and then basically pass that on to the service that can create these models and then just access the outcomes of the recommendations directly via an API. And all of that comes prebuilt for them. They miss out on these amazing higher order services that you guys are creating. That can literally be leveraged by anybody without having to migrate their entire applications or data centers over to the cloud.
Ankur Mehrotra:
Yeah. That's absolutely correct, Rahul. I think if you were to implement such a recommendation system on frame, you would have to think about servers and instances and how you scale. When you're using Amazon Personalized you don't have to have any of your existing infrastructure data on AWS. This is a completely managed service. Which means all you have to do is haul a set of APIs and you've got highly relevant recommendations ready for you to integrate into your application.
Rahul Subramaniam:
Right. So I think given that almost every application has tons of data or items of some sort that they're presenting, whether it be documents, whether it be whatever comes up basically on your screen, can be treated pretty much like an item. And you have to optimize your UX to make sure that you're only recommending stuff that is most relevant to the customer. So this becomes even more relevant, and given that there's such low friction to get started with something like this, if your product doesn't today have a recommendation capability using AWS, Personalize seems like the easiest way to get started. Take your data, put it in S3 and start getting recommendations right out of the box.
Ankur Mehrotra:
Exactly. Given how personalization can directly impact the business output, all businesses should try and use the service to see how this drives overall improvement in the business metrics.
Rahul Subramaniam:
Are there any constraints or limits on the number of items? I think sometime ago we were running into some limits of how many documents or items you can actually have in the system. Are there limits or constraints around that, that have changed?
Ankur Mehrotra:
There is a limit where for one type of recommendation, which we also refer to as a recommended or a campaign, you can have to, to 750,000 items in the catalog. Now that usually is sufficient. You can split it into multiple recommendation solutions, but then if you need higher limits, then we are happy to work with you to look at your specific use case and see how we can help you.
Rahul Subramaniam:
So what does the cycle time really look like with Personalized? And how are folks really measuring relevance of their recommendations that they're getting?
Ankur Mehrotra:
That's a great question. We always tell our customers that when you are starting out with Amazon Personalize or building a personalization system, always think about how you're going to measure success. In success for you would mean, an uptake in [consumer 00:14:00] rates or an uptake in conversion rates. When you're using Amazon Personalize as a member of service, underneath is training machine learning models, it does give you offline metrics in terms of how well the system thinks that the machine learning model form. But those are offline metrics and understand the efficacy of those recommendations to a certain extent through those offline metrics.
Ankur Mehrotra:
We always recommend our customers do experiments with real customer traffic to measure the performance against the business metrics that they're trying to optimize. So we see the classic way of doing this is, run an AB test compared to where you have coming from Amazon Personalize, running in an experiment against an experience that has no recommendations, that's not personalized, or any baseline.
Rahul Subramaniam:
Great. Coming to the AB test set up. What are some easy recommendations for getting started with AB testing? Because it seems like a really hairy problem for someone just getting started with a simple AWS service like Personalize.
Ankur Mehrotra:
Yeah. You're right, Rahul. We're working on making this much more easy for Amazon Personalize customers. We do have some resources, including documentation blocks that walk our customers through some of the best practices they should use while setting up AB tests. But one recommendation I would make is, don't start from scratch. Don't start implementing your own AB testing framework just to do this. There are a lot of other... Lot of a AB testing frameworks out there that are widely used and leverage them. And some of them, we also do have reference architectures where they can seamlessly integrate with Amazon Personalize and the sample code out there that you can use. So that's that's one advice I would give. If you don't have an AB testing framework already in place, try and look around and see what exists out there and what can be used.
Rahul Subramaniam:
Great. I think that's great advice. I think folks who are usually coming in from the on-prem data center world are in the habit of trying to create everything from scratch, because they don't have that support system of all these other services and recipes that other people have created that can literally be deployed with a click, either by the marketplace or via other tools and services that are available online.
Rahul Subramaniam:
So let me use this to segue into your three pieces of advice that you would give customers as they're getting started.
Ankur Mehrotra:
Sure. When we see customers starting out with a personalization or a recommendation system project, I think it's starting out small by building out a proof of concept is the right thing to do. However, sometimes what we see is that when customers are trying to start small, they take a very small amount of data and try to build a recommendation system.
Ankur Mehrotra:
Now, the challenge is that if you don't have sufficient data that presents your overall data set well, then the quality of recommendations that you may get will not be as good as what you may get if you're using a larger set of your data with Amazon Personalize. So that is one thing to think about as you're doing a POC or you're trying to build a smaller version of a recommendation system before you scale.
Ankur Mehrotra:
So having sufficient data in the right, paying attention to the quality of data is really important as you start out. That's the first thing I would say.
Rahul Subramaniam:
So is there a thumb rule for the amount of data that you need to have? I assume most of it is specifically around the events' data. Is there a thumb rule that you'd recommend to folks about what they should think about in terms of quantum of data that they should at least have?
Ankur Mehrotra:
There is no hard and fast rule as such, but the high level guidance I would say is, make sure... So if let's say you are in an eCommerce business, make sure that the data that you're using is representative of your entire catalog, or at least all product categories, rather than just picking the data from just one or two categories and then later you may end up wondering why recommendations for other categories aren't good enough. And the same goes for users as well. If let's say you have users of, if you have different features on a website and different kinds of users, you want to make sure that's also representative of your larger user base. It differs from use case, to use case, but that's the high level of guidance I would provide.
Ankur Mehrotra:
And this does require some experimentation to get right. Yeah. So there are other things that I would say is the ability to provide click stream events in real time. Consumer habits can change quite rapidly. So when you're browsing something, you may be looking for something right now that you were not looking for, let's say three days ago.
Ankur Mehrotra:
So if your recommendation system is not aware of your recent browsing behavior, then sometimes the recommendations can end up being not relevant, aligned with what you're looking for right now. We always advise our customers to try and use the legal client events capability and stream those events to Amazon Personalize, so you can do it quite easily, either using an API that the service provides or using the [edifice amplify 00:19:39] as the creative part integrations with Amazon Personalize. That really improves the quality of recommendations you may receive in to the service. And it can also really make a big difference in the overall success.
Rahul Subramaniam:
So I have a very specific question around this specific scenario because we struggled with it for a while as we were using Personalize. So I thought this would be a good learning experience for everyone else as well. Especially around the holiday seasons, the recommendations might vary quite a bit from what you typically end up recommending, let's say in a usual, or through the rest of the year. So in such scenarios where there are these unique events that you can foresee when they're going to come up, but your recommendation style needs to change, how do as one end up using Personalize to drive the new mechanism of recommendations for such scenarios? What are the customers doing in those scenarios?
Ankur Mehrotra:
Good question. There are two ways to address this. The first one is, which I just mentioned that, if you have real time events flowing into the service then the ML models will latch onto the latest, the most recent browsing behavior of your users and change recommendations in real time. So to solve this so that as your browsing trends and shopping trends start changing due to seasonality, the ML models will quickly adapt to it, changing its recommendations.
Ankur Mehrotra:
The second way to do this, to feed into the service. And when I say data, the historical data. The historically interactions data that you use to give to the service, and the service uses that are train machinery models. So in that historical data, if you have sufficient coverage where you have data from previous, let's say holidays or seasons built in, and if you have metadata that indicates that yes, this day was a holiday. Let's say your trends on Black Friday or Cyber Monday may be different. So if you can annotate your data set by saying, and let the service know that while this day was actually a holiday, then that can be really used for the service to learn how consumer behavior and patterns and shopping patterns change during a particular [inaudible 00:22:06].
Rahul Subramaniam:
That's actually really useful. My last question is going to be around your recommendation towards avoiding any costs overruns, or losing control over costs. Now at the face of it, AWS Personalize seems like a ridiculously cheap service for the value that it provides. If I'm not mistaken, you guys price it in terms of tens of thousands of users and it's a fraction of a dollar for providing recommendations up to 4,000 recommendations, it's practically free.
Rahul Subramaniam:
So what are the mistakes that a customer could make, where they suddenly find that the way they're using the service, or leveraging the service, they end up losing control. And this is common with a lot of these services where you find that customers make one mistake. They realize only later because of the way consumption based billing works, you realize that only when the bill arrives that you've made a big mistake. So what would be the one or two recommendations that you would have for someone using Personalize to avoid crazy cost overruns?
Ankur Mehrotra:
Yeah. So first of all, you're right, we get this from so many of our customers. The amount of positive impact or the uptake in the business output that they saw using Amazon Personalize is far, far greater than what they're paying for the service. And we're happy to see that customers are deriving so much value from the service.
Ankur Mehrotra:
There aren't too many ways where you could end up cost over runs with Amazon Personalize, but there are a couple things I will say. So Amazon Personalize can scale automatically as your traffic increases. Let's say, if you have more traffic during the daytime and less traffic during the nighttime, then the service can scale up and down automatically to provision the right amount of resources to serve your recommendations.
Ankur Mehrotra:
However, we also do allow the ability for you to reserves a certain recommendations per second. And that can be really useful when you're expecting the traffic to suddenly spike up. So for those cases, the service does provide you a way to reserve higher, let's say TPS. So one advice I would give is, think about when you really need to reserve that provisioning because the increased reserved provisioning you're also paying more. So use that feature where you need it. And when it's business as usual, just rely on the auto scaling features.
Rahul Subramaniam:
Would you recommend that folks, especially in auto scaling, set up certain kinds of alarms and alerts so that when it exceeds beyond what you anticipate, in terms of traffic or flow of events and so on, you're at least aware of it in real time, rather than realizing way later that you had some kind of a service event within your application that just suddenly caused a gazillion API calls to be made to Personalize, and it just kept auto scaling up without you really expecting or anticipating that event?
Ankur Mehrotra:
Yeah, I think that's always a good idea to keep some alarms to monitor how the overall integration with Personalize and how the number of feedback calls being made are trending and alerted. If there's something that's out of the ordinary.
Rahul Subramaniam:
Great. I'd love for you to share a few success stories and non-retail would actually be really helpful, because I think the number of people who are looking for recommendations as a service that actually really works well for their non-retail scenario. So are there examples that you could share with us in those spaces or in those domains?
Ankur Mehrotra:
Sure. Yeah. Yeah. So especially during the pandemic, the consumption of media and entertainment services has certainly spiked. So we see personalization becoming more and more relevant. One of our AWS customers discovery a few months ago, they launched their first director consumer streaming service called Discovery Plus. They had a very tight timeline to build a service end, to end and launch it.
Ankur Mehrotra:
They chose Amazon Personalize to bar their recommendations and they were able to do that from absolutely nothing to this running and production at scale, ready for prime time in a matter of few months, just going from POC to just integrating it all the way into their system. So that's been a great story and love working with the team at Discovery. They've been great. They recommended videos and titles and similar videos and other recommendations that you see in the Discovery Plus app are programs on Personalize.
Ankur Mehrotra:
The other one that comes to my mind is Intuit is a data list customer. They've bought an app called Mint, which helps users track their budget and their spending, et cetera. Intuit used Amazon Personalize to create recommendations within the Mint app for financial offers and recommendations for financial products. It is a great use case where you're able to apply the service in all kinds of different use cases. And they're seeing great value from it. That's another one that I really, really like.
Ankur Mehrotra:
So we are seeing use cases across different industries and pleasant piece of price to see how well the models can adapt to different use cases. We're excited and we're working on many new features and launches, and it's going to be a great year for the service.
Rahul Subramaniam:
Thanks so much, Ankur for sharing all of this knowledge with us. As always, we remain huge fans of all of the higher order services at AWS, including Forecast Personalize and the likes. And I look forward to working with you and your team through the year with all the new stuff that you guys are coming up with.
Rahul Subramaniam:
For the audience I would really urge you to go try it Personalize. It's really simple to get started with. And for any application on prem or otherwise, personalization is now becoming pretty much a standard function that you have to offer. AWS Personalize is probably one of the easiest ways to get that into your app without doing a lot of heavy lifting. So go check it out and thank you once again for coming here and talking to us.
Ankur Mehrotra:
Thanks, Rahul. It's always great talking to you.
Speaker:
We hope you enjoyed this episode of AWS Insiders. If so, please take a moment to rate and review the show. For more information on how to implement 100% safe AWS-recommended account fixes that can save you an average of 25% on every AWS bill, visit cloudfix.com. Join us again next time for more secrets and strategies from Amazon’s top product managers. Thank you for listening