Search by Image in Flipkart Fashion
Flipkart Fashion is a section of key focus for the eCommerce goliath Flipkart. A massive inventory, an impressive set of features and aggressive marketing campaigns - remember those TV ads featuring cute kids as adults? - Flipkart is vying for the top spot in the fashion eCommerce industry.
Search by Image feature
As an individual in love with fashion and having a penchant to try the latest fashion trends, I look up to photos of stars wearing it or my ahead-of-time friends wearing it or even the photos in fashion magazines. The biggest challenge though, is the fact that I am pathetic at exactly defining what I am looking for in a 'search-friendly' language.
An attempt like this - 'yellow half-sleeve T-Shirt with something written in blue at the centre’, may cause the search engine to hang! Filters are an option but a little bit cumbersome and lazy beings like me may drop-off at this point.
For all the troubled souls like me, 'Search by Image' (SBI) feature would feel like a 'cheat code' helping you reach the destination faster.
In a nutshell -
Upload Image -> Search -> List of similar stuff
Design and flow
• Initially, to create awareness, the top banner in the fashion section will inform users about the new feature. On tapping the banner a popup or a 10-seconds video will educate the user about how-to-use-it.
• Post that the user would be redirected to 'Search' screen which will be redesigned to accommodate two action buttons for uploadng an image.
• Once user selects an image, a screen to crop the selected image appropriately and select a category of clothing would be shown. The identification of T-Shirt/Shirt/Jeans etc from the image could be accomplished by any of the three image processing algorithms namely - SIFT, PCA-SIFT or SURF.
• After user confirms, the cropped image is uploaded and sent to the image matching engine (more on it below) that returns a list of Tshirts/Shirts/Jeans/Trousers similar to what the user is looking for. Pure Magic !!
A bit about the image matching engine
Although I am not an AI/ML guy, I have done some research on how image matching algorithms work and would love to give you’ll a brief about it.
The image inventory is stored in the form of image fingerprints/hashes. This provides us a data-set of image hashes. Image hashing is the process of examining the contents of the image and then constructing a unique value that identifies the image. A hash function shall be applied to the image to generate the image hash. For generating the hash the dHash algorithm is the most preferred.
Now the hash values of the image uploaded and that of the inventory images are compared and a list of similar ones is returned where similar means having 'difference in hash value below a certain threshold’.
Important metrics to determine success or failure
Now comes the most exciting part. An aspect of product every PM is worried about.
Flipkart’s core business metrics to track
1. Percentage improvement in cart size - The linchpin of any eCommerce business is the cart size. A 2-5% increase in the cart size within 30 days of feature release is a good indicator. Note: cart size here is specific to Flipkart Fashion.
2. Percentage improvement in engagement - Overall improvement in the MAU and DAU.
Feature-specific metrics to track
1. Total number of times SBI used - Gives an idea of whether users find the feature interesting and believe it to be having value.
2. Percentage of successful SBIs - A successful image search would be if - user ends up purchasing at least one of returned list of products or if user adds a product from list to his/her wishlist or if he/she shares it on social media or if he/she views the details of any of the products shown to him. An initial figure of 50% for the first month and a gradual rise to 80% post 6 months would be a positive indicator of the image search engine's strength and adaptability.
3. Percentage SBIs of the total number of fashion searches - When it comes to clothing, more and more people opting for image search rather than the standard text search indicates adaptability and the value created by this feature. Starting off with 10% for the first month, the target should be 50% by the end of 6 months.
4. The number of failed SBIs - A failed SBI would be the one where the user drops off either from the 'Crop Image screen' - indicating a bug perhaps in the crop section or from the 'Results screen' indicating the matches were not as per user expectations.
5. Percentage improvement in recommendation-based purchases - As a user uploads an image of what he/she is looking after, it also proves to be a 'nitro-boost' for the recommendations engine and helps in improving it significantly. As the quality of recommendations improves, the purchases happening through it would also improve and this should be tracked to understand how well SBI is helping improve recommendations.
Tip: Notice that most of the metrics mentioned above are in the form of percentages. When calculated as ratios or percentages, these metrics become easily comparable and more actionable.
Thank you for reading. Note that I am in no way associated with Flipkart but do believe that such giants of the eCommerce game need to stay ahead of the curve with some awe-inspiring features like these.
You can access the prototype here
