Product Assignment : Smart Refrigerator

I recently worked on an assignment to design a smart refrigerator, which I thought I’d finish in a weekend. However, then I did some research about what’s already happening in the market, which obviously opened up a few more tabs in the browser and this weekend assignment turned into a project worth spending time on. So, I am sharing the notes I had prepared. Let me know your views and what you would like to add to your variant of a smart refrigerator. I would also enjoy some feedback from any commercial refrigeration services. Maybe you could find a way to adapt my refrigerator so that it could be used for grocery or retail purposes?

Also, as you can infer, this assignment is part of my self study and journey to become a better product manager. In the last couple of years, I have gained a broad perspective on this function. I’ve just tipped my toes, so to say, and there is so much more to learn. So, I’ll try to write more on product management as a theme. This also gives me a new chapter on the topics to write about on this blog, which otherwise had sort of taken a backseat in terms of relevance, as most of my life updates are now broadcasted on Instagram and Twitter (and increasingly on private watsapp chats) because who reads blog posts anymore?!


Problem Statement

Design a fridge for the millennials.

Market research has come back saying that millennials need to have an interactive smart fridge. IOT is the buzzword they want to capitalize on. Design the product to meet this requirement from purchase, initial configuration, features etc. Be as detailed as possible. Try to define metrics/KPIs to measure. Call out any assumptions. Estimate the ROI of the product you are making.

Assumptions on the key requirements of the users

  • Married Couple & Family
    • Both working – optimizing on time and coordination is the biggest need for this cohort, this couple would like to save time in ordering and preparing food, so that maximum time can be spent with each other.
    • One partner working – since one partner is stay-at-home parent/home-maker, considerable time is spent in the kitchen by that member; focus would be on entertainment and equipping her/him with data and content to improve family’s health and quality of life.
    • Families with kids and/or others livings at home – With kids and other people at home, the added features can be around their health and safety. Also shared notes and communication can be a potential need.
  • Single residents – Primary need is to optimise on time, help save time in ordering and preparing food. Since single residents would tend to have smaller houses, another dimension would be to offer auxiliary services such as a coffee maker within the fridge to save space

I am prioritising the family use-case, as the high cost of the refigertor right now makes it unviable for single home owners.

Model Unit

  • Samsung RF28N9780SG/TL
  • Capacity ~ 800 Litres
  • Retail Price ~ INR 2,50,000
  • Retail Price without touchscreen, internet connectivity ~ INR 1,50,000

Usecases –

  1. Food Catalog : The fridge should be able to catalog the food inside it for easy remote access via the door panel and mobile app without opening the fridge. This can be accomplished by scanning the barcodes on the food items to pick relevant info, plus clicking pictures. All this can be done by the phone app, no scanners or cameras needed in the fridge. Search post cataloging can be voice enabled for both the door panel and the phone app. Long range plan – retailers would start printing QR codes which capture all relevant info to automate this cataloging. This helps in 2 ways –
    1. Save time and effort in grocery shopping, can just check the app to recall what’s stocked in the fridge
    2. Helps save power consumption and maitains food freshness when the door is opened fewer times
  2. Expiry Date Alerts : Alerts can be set to trigger reminders X days before the expiry to help consume food when it is still healthy. The door panel & phone apps can have a dedicated section “Expiring Soon” to help monitor these food items
  3. Recipe Recommendations : The door panel app can pull up step-by-step video recipes, some of which can even be stored locally for availability without internet. Voice control to play and pause per step would help further. Recommendations basis the major ingridients available in the fridge can be a section for quick selection.
  4. Video Streaming : In addition to the recipes, the door panel app can pull up videos and music from popular OTT channels basis the subscription plans of the user. The door panel would effectively be the entertainment hub for the kitchen.
  5. Grocery Ordering : Basic feature set would be to run the grocery ordering app of choice on the door panel tab, but a smarter version of it could pickup the stock count and patterns from the food catalog to build the cart, maybe even order some of the common regularly ordered items such as milk, fruits, etc as they are about to get expired.
  6. Ice on-demand : The integrated water and ice dispenser unit in the fridge can now be connected to the door panel tab and phone app to switch on the freezer on-demand. Typical usecase being need of fresh ice before guests coming over, while there is nobody at home to freeze the ice.
  7. Defrosting on-demand : Having separate sections in the fridge can allow for maintaining different temperatures in the containers at any given time. The freezer iself can be split into 2 components – one section which can continue to be at required temperature and the other one which can hold items which need to be brought down to room temperature at a certain time by a remote trigger or pre-set config. This would help in natural thawing of the meat and other related items, which currently needs to be done manually.
  8. New integrations : These integrations are not digital in nature but hardware components that can be added to the fridge panels,similar to the water dispnser. Remote control by the door panel and the phone app further add to the utility which is difficult to achieve in standalone equipments.
    1. Coffee/Tea maker : The water dispenser unit can be extended to a coffee/tea make. On the inside of the fridge, a storage unit can made to hold the coffee/tea powder which can brewed on-demand using the same water supply as the water dispenser
    2. Water heater : The water cooling unit can do more than just cooling the water, it can also heat it to a pre-set temperature!
    3. Rice cooker : Similar to the coffee maker, a storage unit can hold the rice and mix it with the same supply as that of the water dispenser to prepare steamed rice
    4. Toaster : A section holding the bread slices can be connected to a toaster to have it ready on-demand. Phone app and Door panel can be used to trigger the loading of the toaster at a configured time.
  9. Smart Connectivity Hub : The door panel tab would support a pool of digital applications like a regular android tablet, supporting common usecases of shared notes, calendars, to-dos, mails, music, budgeting and other apps that are available on the user’s phone. The handsfree audio/video calling feature can be particularly useful while cooking.

Prototype of the new smart fridge

Key applications to be designed for the display tab and phone app

  • Food Catalog : Add new items, check for contents, expiry date alerts
  • Recipe Finder & Player : Find recipe by search, recommendations basis stock
  • Grocery Shopping : Buy new items manually, recommendations basis stock
  • Operate the fridge : on demand ice, defrosting, toasts, rice, tea/coffee, etc
  • Other apps : as per installation from playstore and sync with phone devices

Key metrics to measure product success, ROI

  • Purchase : measuring the sales funnel – lead > oppurtunity > open deal > closed deal across online, retail and wholesale channels. KPI here would be the % success rate for each stage of the funnel to track conversion and pipeline health. If you’re unfamiliar with a sales funnel, here are some great free sales funnel templates that could help.
  • Usage : Usage of the key 4 features (first 4 points of the section above) can be tracked by recording anonymous data from the devices directly. This data would help analyse if the features are being used as intented or not on parameters such as frequency, timing, duration. The variation from expected behavior would infact be the leading indicator for intervention and feedback on improvements.
  • Feature Support : Each application in the display tab and the connected phone app would feature a section for direct feedback and complaint registry which would help in understading the specific issues. This section would also be the input for all customer support and related follow-up conversations.
  • Repair & Replacement : Basis the feedback and support requests received, the support tasks would need to be monitored as well. The whole journey from support ticket issuance > assignment of a service personnel > actual service > closure of the support ticket can even be mapped on the door panel to increase transparency in the process. KPI here would be keep the service TAT as per SLA.
  • Referral : To foster high NPS and organic growth of the product, referrals can be incentivised. The door panel can have a special app to generate voucher codes to be used for referals, which upon successful redemption can provide benefits in terms of cashbacks and/or waivers on due EMI payments, complimentary service packs, etc.

GST simplified for a business owner

If you’ve been following the national news, you must be aware of the big tax reform coming up next month. The numerous existing taxes are getting consolidated into a single overarching tax – Goods and Services Tax. India is not the first country to implement the GST. For example, in Canada GST was introduced in 1991. You can learn more about GST in Canada by contacting some of the top lawyers in toronto. But no other country faces such complexities as India does. If managed successfully, it would become one of the biggest accomplishments of the current government as there is no doubt that this is the most ambitious GST reform ever attempted, both in scale of operations and the intricacies of execution.

There are lots of resources just a google search away, if you’d like to understand the background of this reform, i’d recommend this guide by Zoho to get started. Cleartax has launched an e-learning course for those who want to understand the concepts in a video format. I’ve even attended a couple of events to learn from the perspective of the finance community, the speaker from Capstone was amazingly talented at summarizing my views with an analogy –

What the current world looks like –

What the government promised when it promoted the slogan – One Country, One Tax

What we’re actually getting –

Apart from the rules that are changing, the modus operandi demands an upgrade as well. I believe that’s what is more relevant for people to understand, because you can still go to your CA for consultation on the rules and ensure compliance at all steps, but the CAs are not equipped right now to transform the back-office operations of their clients to ensure the work is happening as per the new world order. A lot of the large firms will get their processes upgraded by contracting Big4 consultants and Xero accountants, their mid-sized cousins will opt for smaller consulting firms. The bulk of the business owners would still attempt making these changes by themselves because that’s how they’ve been doing it. The local entrepreneurs are the real multi-tasking gurus who don’t mind getting their hands dirty.

I’m working on creating an ecosystem of connected google sheets that can be used as a make-shift processor for accounting and taxation needs of a small scale firm. This system would ingest monthly sales/purchase transactions, and create the tax sheets that can then be submitted on the GSP portals which inturn are nodal points that connect to the GST server. This process eventually would have to be automated as there 4 filings to be made each month, 36 per year; plus an annual return. However, if you find that your business becomes too advanced for this make-shift processor, you could always look for some outsourced accounting services who could help you with different parts of your financial work. It can be easy to fall behind on jobs like these, so sometimes getting help can be one of the best ways to keep up. Of course, you could always keep using the make-shift processor if you wanted to.

Anywa, let’s understand a bit more about these returns conceptually and the framework under which they would be filed, and then I’d talk more about the google sheets in a follow-up post..


Simply put, each month, the registered business owners would have to file the records of their sales, purchases and applicable tax for the previous month. One added monthly return would be for the TDS. This isn’t a new compliance expectation from the government, one just needs to think back to when we were all forced to apply for pan card seva, but when you look closely, there are 2 key features that really highlight the new approach adopted by the government –

  • Monthly frequency would make it further harder for business owners to evade taxes, people who used to fraudulently adjust their aggregate sales and purchase volumes would have lesser time to do so
  • The sales records that are to be uploaded will have to match the purchase records of the customers (as filed by them), thus interlinking the records at a digital level would make it impossible to get fake tax credits without involving multiple other parties

Let us now use an example to understand how this reconciliation process works:

Suppose MK Kitchen Knives (recipient) purchased 10 tons of steel from GH Steelware Inc. (supplier) which is also registered for GST. The two companies will reconcile their transactions, and the recipient will claim the input tax credit, as follows:

  • GH Steelware Inc. will file the GSTR-1 report (Details of outward supply).
  • The details furnished in the GSTR-1 will be auto-populated in the GSTR-2A (Details of inward supply) for MK Kitchen Knives, where they will be able to see the transaction details.
  • MK Kitchen Knives will then check the records and make any necessary modifications/additions. Once the changes are made, this information will be automatically pulled when they will file the GSTR-2. The correct input credit will then be credited to their electronic credit ledger.
  • GH Steelware Inc can then use the GSTR-1A form to view and accept the changes that MK Kitchen Knives made in the GSTR-2.
  • Finally, once GH Steelware Inc. has filed the monthly returns (GSTR-3), MK Kitchen Knives will be able to avail the input tax credit and apply it to future output tax liabilities.

Below is the application framework that’s been setup by the GSTN – the central body that essentially owns the GST server (in blue). It is expected that up to 3 billion rows worth of data (each invoice/bill being a row of its own) would be uploaded and downloaded each month. To manage this massive flow of data, the government has created clusters of nodes, the GSPs, to allow multiple people access portals where they can interact with the system in their own customized app without compromising on the security. Thus, the GSPs act as an intermediary between the government and the individual taxpayer, shown in the central box below. Some 200 companies have registered themselves as GSP already and soon more will follow. These include the Big4 and their smaller cousins, along with traditional IT companies and young fintech startups. GBox Capital is a good solution for many start-ups.

But as a business owner, the stick man on the left in the above diagram, what matters is that you’d have to ensure that you’re able to upload the GSTR-1/2/3/7 forms on a GSP portal of your choice each month. Since you have to do this exercise each month, the only sensible way is to ensure that you don’t simply rely on an accounting application but instead deploy an ERP that generates the GSTR return documents in the needed format automatically from the transactions data. These files can then be downloaded in CSV format for uploading on the GSP portal of your choice, and the rest of the data processing is done on the GSP-GST server.

The solution proposed above is the bare minimum you’d have to do, and this too would involve changes in mode of operations. This solution, however, is suitable only for small scale companies which can afford to manually track their filings. For large scale operations, the GSPs are expected to roll-out their own version of ERPs and tax applications that will allow users to file their tax returns automatically from their transaction data. This, however, comes at a cost of placing confidential data in the hand of a few GSPs but one way or not, any company can’t avoid this overhead going forward.

It’s amazing to see this level of the participatory and transparent approach of the government. It is the endeavor of GSTN to build the GSP ecosystem, ensure its success by putting in place an open, transparent and participatory framework for capable and motivated enterprises and entrepreneurs. In my last post, I spoke about designing new systems from bottom-up to keep up with the new delivery channels of optimum CX and through GSTN, the government has earned my respect in this regard!

I am motivated to participate in this ecosystem starting with helping small business owners migrate to GST world by digitizing their accounting, with a google sheet package to begin with and then as I get better understanding of the ecosystem, have an idea for a mobile ERP that would allow small business owners to evolve to a new world order where transactions are done and accounted for without fear of compliance and boring paper-work. It’s time finance became as fun as sales!

Thoughts on building customer experience

Customer Experience is the new buzzword in digital startup communities! A lot is getting written on inculcating it in the company DNA and consultants are advising companies on how they can improve their operations to create customer delight during their interactions. The challenge with CX is often it isn’t as refined as financial metrics in its measurement. I would say it is hard metric to define as it is still building up and wasn’t always as big a priority as it is in today’s world. But whichever way you want to focus on it, the opportunity lost is huge if you don’t focus on it. This survey estimated $62 billion is lost by U.S. businesses each year following bad customer experiences. In the a supply constrained world, there were 2 broad ways of surviving in the market –

  1. Create innovative products or service that nobody else (or a very few people) are offering and customers will flock to your stores
  2. Be efficient in operations and absorb good margins from business and thus fight the competition from other innovative companies

But in an overcrowded market for commodities where too many players are competing to meet the demand of a customer, the only way to shine bright in the crowd is by attracting him and then retaining him. Thus, the world saw the rise of online marketing teams for hire like SEOConsultantBanglore.in which were tasked with attracting new customers and maybe keep pulling back the dormant ones as well. However attracting new customers kept getting costly with time and there isn’t an infinite market to keep expanding to. Thus, retention takes over as the priority when acquisition starts showing signs of saturation.

Any effort that creates a positive experience is an effort towards retaining that customer and that’s what I would hold the CX team responsible for. It is my opinion that a lot of teams only expect CX teams to create wow moments for customers to delight them, but what’s actually needed is an audit of the interactions a customer has with the company and make sure there are no negatives. My intention for writing this post is not to reveal a hidden secret or to prove mathematically the ROI you can get over your investments in making your CX better, but to highlight some key philosophies that I believe in regarding CX..

A steady above average CX curve is better than a sine curve that gets a surge from freebies and dips when commitments are not honored

We’re at the inflexion point where the customer expects to see his touch points with the companies digitally as fluidly as he expects his personal touchpoints with his family and friends over a chat app. In the new digital world, the companies which are using technology as a lever to gain efficiency don’t have a very distinct offering for the customer, just that they are able to pocket better margins by playing smartly. I believe it becomes super-important for them to pay attention to customer experience early on as it can become an added value proposition they have at the front end for customers over their traditional once-innovative incumbent competitors. An ideal time series plot of the CX line should be a flat line slightly positive rather than a sine curve. That’s what builds transparency and trust in the system and cements the relationship that ensures the customer comes back again in future to transact.

Use technology to increase transparency in your operations, honor commitments and share updates as often as possible to build trust

However this process needs to be redesigned with a long term vision and integrated deep beneath the layers of processes that govern the day-to-day working of executives. You can not hack into this system by deploying last minute Jugaads. If we go back to CX curve, a momentary hack is only a crest following a trough and the CX curve will come down again due to a systemic fault elsewhere. Not all customers are patient enough to give you multiple chances, not all customer relationships are repaired by offering a freebie following a screw-up. Thus, we need to get out of the hustle mindset that has become ever so ingrained in the way young firms operate under the pretext of the urgency to outgrow competition. The shortlived growth brings upon bigger problems that later become 10x difficult to solve at bigger scale of operations.

Becoming digital is not same as creating an app that mimics the offline operations, redesign processes instead of hacking for short term

The only way to create sustainable business that retains its customer is by being transparent all the time, given the business is offering the right product/service at the right price point. Digitizing the interactions and sharing updates adds to the trust and prevents situations where a enraged customer needs to be pacified by personal assurances, all these last minute rescues can go worthless if a clear plan of action is not shared with the customer or is not adhered to after promising. Taking a specific example, banks today have created phone apps that allow the customers to transact online without having to visit the branches, but they haven’t replaced the old system yet. The internal systems are not tracking the interactions a customer has with the bank and thus the executives which are tasked with handling customer queries are often clueless about the problems of a customer even if all interactions were digital and could have been tracked by the bank. Thus, interactions right now still encounter troughs and crests in terms of experience. Compare this with the CX that Uber app delivers by designing their processes around the interactions and not merely treating it as a database they can refer when a customer calls up for a complaint.

Scale is not an excuse to go down on personalization, and offline retail still wins over big chains when it comes to customer experience.

The streamlined behavior then needs to be retained at the bigger scale as well, scaling up makes things complex – true, but that can’t be an excuse to make things less efficient than how they were previously. Banks today are spread geographically and between the online and offline world, they are a good example of how over-expansion and cold unemphatic control by processes alone can add to frustration when fires are breaking out, no one is willing to take the responsibility and there isn’t a clear system in place to identify non-adherence of processes and customer dissatisfaction. Things get done faster when you can hold someone accountable and blame bureaucracy without pointing fingers, thus digitization needs to be used as a glass pane that allows customers to see you’re working on their case and not as a shield to shy away from accountability in the name of complex processing..

One last note that I’d like to share is that as companies grow up, the interactions it has with its customers become increasingly standardized as processes set-in. In the trade-off of expansion and cost of operations, executives which are the face of the company for the customers are often compromised on quality and their sense ownership as it is expected that the set processes would guide them and situations where exceptional decision making is needed can be forwarded to managers which can specialize in this decision making for multiple locations. While all this makes sense from a financial perspective and companies do it to optimize their costs, but it should not come at a cost of alienating the customers and making them feel as if they dealing with a robotic teller instead of a business partner who understands and reciprocates. I personally am a big fan of unorganized retail segment in India which is a big contrast to the way the US operates in this segment. I think while it is important to optimize on cost, but personal touch can never be replaced. A dedicated small scale entrepreneur whose life is dependent on the sale he makes from his store cares much more about his customers, knows more about his products than any executive working at a big store. And the fact that small retailers still exist means that customers value experience over price. Maybe micro franchise is a model that’s building up because of this as it is a hybrid that has the needed intelligence and processes of a big retailer but the entrepreneurial spirit and personal vested interest for growth of a small retailer.

I’m excited about the digital transformation that’s reshaping a lot of business, especially retail sector, be it shopping stores or banks. I’ve heard that many companies have decided to implement digital transformation into their business through the use of event-based architecture. You can click here to learn about event based architecture to see if this is something that your business would like to introduce too. This will only start to improve the business in all areas, especially their productivity. The transformation projects can really outgrow expectations when the CX component is seen as an integral part of the central process design and not merely as a coating on the transaction layer between Us and Them..

Analytics Stack – Predicting the future of modern data teams

I see a systemic change in how we are leveraging the analytics teams at organizations and as a result a complete new analytics stack is getting developed where people with different skillsets are individually working on the different layers of this stack. This evolution path is similar to how software development teams came up over the last ~20 years. HBR has labelled Data Scientists as having the sexiest job in the world, and McKinsey recently has published a report calling the present times as the age of analytics.

While the companies are convinced that they need to invest in their data teams, but young professionals who would be taking up these roles are often unaware of what they are signing up for. So, a lot of articles exist for the CIOs and CDOs on how to levarage analytics, but nobody is telling the potential analysts how things are changing from their perspective and what new roles are available for them, so that they can hone their skills accordingly.

We are right now living in a world where most of us in this field are self-taught, so I feel it is becoming increasingly important to carve out time to understand the new oppurtunities rather than just following existing career paths. Everyone today wants to be a data-scientist but I believe soon there wouldn’t be any such generalist role. Mature analytics teams have such complex workflows that they have to break-down the work into multiple jobs and have analytics dedicated to a part of the problem. The middle-layer manager managing the analytics would be the only one with grip over the whole project but he’d spend little time in technicalities and a big part in people management. This is a classic individual contributor vs manager role conflict, managers often have a good grip over the existing tools and frameworks but find it hard to innovate because they are no longer directly connected with the technical world. Thus, although smaller teams right now have data scientists but as the teams would mature these folks would have to either pick a technical segment to specialize in or become people managers.

This post is an attempt to deconstruct the workflows and list the roles available in a typical analytical IT team. I’ll try to follow it up with a post on how to acquire the skills needed for these roles. I’m also learning this way!


The biggest reason we were not able to take all business decisions based on hard data was that it was too expensive to store data. So, we had to prioritize, important financial data was stored and archived digitally but transactional data was mostly still recorded on paper or it was just stored at an aggregated level instead of individual records. At best we were using data to publish financial health of companies. Trading firms had pioneered by creating quant jobs to predict outcomes and possibly alter decisions as there was a lot of potential to increase profits just by having faster access to data. For all other industries, standardized month-end reporting was enough. Not anymore! You’d be surprised to know that hard disk drive capacity increased 1 million times in last 5 decades!

As the data storage costs have come down drastically, bulky transactional data is now being stored and archived, it is also being parsed through models to identify patterns and trends as an additional source of insights for increasing profits or decreasing costs. Models and algorithms which were earlier limited to research are now being employed in daily routine and are even getting standardized into plug and play models, making it easier to skim through datasets storing these mundane daily transactions. This practice will only get deeper and sharper as we scale up things!

But, big data doesn’t always mean it is good data, a lot of the times the transaction data is just noise and not really insightful at a macro level to the human eye. A general business user can only do simple analytical operations in his mind like finding co-relations. Thus, a lot of times, the only way you can discover hidden insights is by assuming some co-relations and iterate through the standard templates known to us, in a way hit and trial to see if we can guess the complex relations. In my experience, often analysis at this level is equivalent of searching for a needle in a haystack. It involves working at such microscopic levels that to focus one has to disassociate with other aspects such as managing people who are executing strategies formed on the basis of these insights. Thus, often these roles are very technical and disconnected with their business impact and a company can afford to get to such analysis only when it has solved all easier problems!

So, although bigdata is a new found asset, regular business users’ life hasn’t changed a lot, they still use excel for most of their adhoc analysis. Thus, I believe the bigger impact will delivered by making it increasingly easy for an average user to store regular data in an online connected environment instead of isolated offline ms excel/access silos. I say so because majority of the businesses don’t really generate such huge amounts of data that a single hardrive is not enough for them. The problem that they really face is that it takes a lot of manual effort to convert the non-standardized raw data in their ERP, CRM and other systems into usable insights. Plus the data is hardly available at the instant you need it, some analyst needs to understand your request, and create an adhoc report to serve you, and this asynchronous approach translates into the overall slower speed of execution.

System interface are getting redesigned as well; monopoly of SAP for ERP and SalesForce for CRM will soon end. A lot many options are coming up which are mobile friendly for easier data entry and the database is accessible for connections with other dashboarding tools. To summarize, with the infrastructure changes in place, we have changed the game and the roles of players playing the game.

And, because systems are getting re-designed, I see a lot of scope for data analysts to get involved in the designing process of these new systems. Whoever is leading the analytics efforts at a firm, needs to manage 3 gears that in a way run in series – Webforms for users to enter data into databases, Database tables to store and process data and Dashboards to visualize and analyse data for users. These 3 fields are individually big enough by themselves for anyone new to specialize in!


Backend Data Engineers are responsible for ensuring forms are connected with correct fields in databases and tables are interconnected with each other. Bulk of traditional IT teams would soon get concentrated towards this role. We’d always want custom apps to be created for enterprise in Java but that industry has reached its saturation point, with hardly any innovation. I see much more effort being done to capture data from new kinds of input devices and then storing the big data in newer efficient ways. Once data pipelines are laid down, engineers are needed to fetch data and push it through other pipes. We live in an interconnected world, where not only websites but soon products would be communicating with each other via APIs, a backend engineer is also laying wires for these connections to make sure every data point needed for analysis internal or external is made available.

Front Data Analysts are responsible for pulling aggregated data from tables into BI dashboards. more often than not they need to enrich the data before publishing it. Because we keep adding the types of datatables available for storage and the technology on which they are based, a lot of upcoming IT roles are asking candidates to be just able to read data from these new kinds of tables. Another breed is focusing on the enrichment part. Often we’d simply want the dimensions to be grouped – eg – data providing us just pincodes but we want to compare performance of states. At times the end-result of the custom analysis results in a few extra columns storing these new ratios, coefficients, probabilities, relations, etc – eg – basis the historical shopping history of a customer, identifying 5 items on which he can be offered a discount. This post here explains the enrichment of data using machine learning technologies.

The third class is of designers who visualize the data in the form of charts and tables. No matter how rich the data is, if it is not represented correctly, not everyone would find it actionable. This subject can be more art than science. It may seem simple but being the last layer in architecture, this is where a lot of collaboration happens. And with the ever-growing features in dashboarding tools, business users will soon ditch powerpoint in favor of creating live dashboards. Which is the most commonly used form of visuals? It’s not a bar or a line chart, data-tables are by far the most commonly used visuals still and I don’t see that changing anytime soon. Why? because that’s how people consume data, only when they’ve got acquainted with the order of magnitude and relevance, they start comparing one item with another or look at the trend, and that’s where the graphs come into the picture. Thus excel or similar spreadsheet tools will remain strong, and the scripting languages like VBA and Google App Script won’t die. Their roles are surely changing though, instead of getting used for data transfer and manipulation, they are increasingly getting used for data representation.


You’d be surprised if I tell you that last year I had created this complete stack within G Suite and I was managing a multi-million dollar business on it.  It is the easiest way to get started with your business. Especially when you’re still prototyping and changing your tables frequently.  Within G Sute, you can use forms to allow users to enter new records from their mobile devices, these forms are connected with google sheets, thus creating new records in the sheets in realtime. Also, these sheets can then be interconnected with each other using importrange function which allows you to convert these sheets into mastertables. DataStudio can then display dashboards based on live data in google sheets which can be your first ERP and CRM. Thus, without any upfront investment, you can create a realtime workflow and build prototypes.

But I see these workflows getting further simpler when their makers put up a layer of GUI on top to allows non-technical users to drag-drop their customization instead of coding them. I love the work that AirTable and Podio have done in this area. There are so many other tools that more specific in their use-case like industry specific CRMs and workflow apps. They provide you the complete playground that eases the work of a back-end engineer to a great extent. Obviously when you scale up the project, you’d need your own custom databases, but let’s not go there for now.

Alteryx is a great tool that fits in the middle layer and helps the front-end engineer to breakdown and re-aggregate data as needed without coding. This is why I strongly believe soon there will be tools that will have in-built drag-drop modules to predict complex co-relations between variables to create new labels/fields within the table, feature engineering is what this process is called. Maybe Alteryx will up-scale in this area or maybe newer tools like SparkBeyond will get better and become part of the toolkit. Those who can’t afford Alteryx or would like to have a no-constraint playground, only option left is to build a custom script over R or Python. It is the preferred path right now due to lack of alternatives but soon only specialists would need code to manipulate data. This is why I’m not too worried about learning every other new coding language, it is more important to know the difference between different techniques, and which one is to be applied in a given problem set.

Lastly, the layer that is already part of mainstream analytics workflow will only get better with time. Tableau and PowerBI are 2 popular tools here and it’s important to have a good command over either of them.

Excel and Google Sheets will soon be used at this last step of data representation only instead of the current approach where it is being used at all 3 steps. But still I would say it would be important to have a good command over VBA and Google App Script. Their relevance will not be diminished by all the other tools that I mentioned above.

Once you move over to enterprise level tools instead of these starter ones, you’d be seeing html/android forms on your devices for data entry, MySQL/Mongo databases storing your data, might as well have hadoop to manage the bigdata, php/python to connect the two, tableau/powerBI to pull data from databases with maybe some addon-ons depending on your usecases. So, essentially the workflow remains the same, but because of added complexity due to scale, you might end up working on only one of the pieces.


The best (or worst) thing is that none of this knowledge exists in our coursework anywhere! So, the field is pretty much open to anyone irrespective of their field of study. It’s best to get some hands-on experience through internships and see what you like best and then go deeper into that area.

Hopefully, we’ll see changes in the curriculum of schools and colleges, and these fields would become stand-alone subjects. We’re seeing these distinctions at graduate level but I believe we don’t want as many researchers here. More kids need to take up these jobs straight out of college but they need to be better prepared. Just an aptitude and will to learn is not good enough as it favors self learners. This field is application driven, we’re modern age engineers who need to need to manage the flow of data and make sure everyone is fed well 🙂

 

Community Outreach

A happy new year to all the readers, ours is a small community but growing with time! 🙂

2014 was a good year, it was the first full-year of work for me, as I had shifted to Bangalore in 2013 for work after graduating from college. Through this post, I would like to highlight 2 NGO campaigns being driven in India that I had the chance to work with and I really believe they should be getting more visibility.

Not Just a Piece of Cloth is a campaign by Goonj which focuses on opening up the most taboo and ignored subject of menstrual hygiene: a female health hazard by involving the masses in generating an affordable cloth napkin. The NGO from India promoted by Anshu Gupta, aims at re-positioning cloth and other underutilized material, beyond charity, adding dignity and turning it into a big resource for development. NSS, the community service team at the BITS Pilani campus had partnered with Goonj in the past for collection drives and from what I’ve read about them, they are really doing a great job with not just clothing but also in the field of calamity relief.

BMST India has been providing blood banking & transfusion services to hospitals & patients in Bangalore and surrounding districts since 1984 but they’ve recently extended their program to setup a Stem Cell Registry Drive. It is a new drive that they have started and are still in the process of setting up operations at full scale. Mentioning bone marrow transplant and leukaemia raises eyebrows and suspicion. To know more, I’d encourage you to follow the stories of Amit Gupta and Nalini Ambady if you have some time.

I had a good experience working with the teams that were working on these projects. Feel free to contact me if you’d like to know more about them. Cheers to a good start for the new year!

Learning continues after college

Today, I wrote the final exam of my second course on Coursera. The course was titled – “A Brief History of Humankind” and I don’t think I’ve ever taken a more important course than this one, so much so, that I am almost compelled to write this post so that anyone who might be interested in the course but doesn’t know about it can be helped in a way if this post reach him/her.

The first course that I had completed earlier was titled – “An Introduction to Operations Management” and was equally interesting for me. It was very specific to what I wanted to in future. After completing it, I really felt encouraged to learn as much as I can from the site, so I signed up for many other courses, but then it wasn’t easy to take out time for more than 1 course at a time. I really wanted to do things properly and didn’t want to rush through them for the sake of completing the course. So, I un-enrolled out of all the courses except for the course on History. It was a right decision as this second course on history was altogether a different adventure.

Before I go any further, if you’ve not heard about Coursera, this is what they have to say about themselves –

We believe in connecting people to a great education so that anyone around the world can learn without limits.

Coursera is an education company that partners with the top universities and organizations in the world to offer courses online for anyone to take, for free. Our technology enables our partners to teach millions of students rather than hundreds.

We envision a future where everyone has access to a world-class education that has so far been available to a select few. We aim to empower people with education that will improve their lives, the lives of their families, and the communities they live in.

Coursera hit a milestone of 100 partner institutions, 500 courses, and 5 million students in 2013. You can read the founder’s views on future of online education on this blog post and their journey of last 2 years here on this post.

I felt really good about the fact that I was able to take these online courses without having to pay a single coin and are still these courses are taught by some of the best professors available in the world on the respective topics. I don’t think anyone can deny the fact that MOOCs are one of the best gifts of internet to the world. What’s a MOOC, you ask? MOOC stands for a Massive Open Online Course. O’yes, Coursera is not the only place where you can learn almost anything for free, there are more sites and the numbers are growing. Although, Coursera is really the biggest thing that’s there right now.

I will now briefly share the details of the two courses I’ve completed so far –

coursera_operations

An Introduction to Operations Management – This course was taken by Prof. Christian Terwiesch. He teaches Operations and Information Management at the Wharton School, University of Pennsylvania.  The course had 90,000 student enrollment. The keywords that were discussed during the course were – Bottleneck, Little’s Law, Inventory, Waiting Time, Six Sigma. I ended up scoring high on the course, so it was a very good experience for me, though I’ve forgotten the formulas now after 6 months, but I can always refer to the notes and video lectures from the course. Here is my Statement of Accomplishment for the course which is also visible in my linkedin profile, nicely done by Coursera team!

 

History of Humankind

A Brief History of Humankind – This course was taken by Dr. Yuval Noah Harari. He lectures at the Department of History, the Hebrew University of Jerusalem. The course starts off from about 200 thousand years ago when humans had just started evolving from apes and there were atleast 6 different human species including the sapiens. From that point which arguably is the oldest point in human history, the course moves forward covering every possible milestone we’ve had till present day. What’s more interesting is that the lectures are not just about listing the facts and dates of important events, they make you question things that most of us had taken for granted. When you are covering so much of history, there can be parts which you’d like to skip but Dr. Harrari is such a brilliant storyteller, he would keep you attentive all the time. Also, I really appreciate the depth and wide scope of his knowledge, during his explanation of historical cases/events, he could link them with scientific and psychological theories almost as if he’s an expert in all the 3 areas. I even ended up scoring a perfect score in all the exams/quizzes, but the questions were very easy, you wouldn’t have any problems with them as well. Here is the course syllabus briefing about all the lectures, this one comes highly recommended!

 

Well, that’s it from my side. You should now be heading over to the Coursera site to choose what you’d like to learn and enroll for..

Happy Learning!

 

Home Expenses Template

It’s been exactly 100  days that I last posted on the blog and this last post of mine was on shifting to Bangalore and starting with my first job! I shifted here in June, but even after 4+ months, the city is yet to impress me. Bangalore has little charm. Everyone here seems to be stuck in a town that’s away from home, the usual pursuit of happiness, you know the rest of the story…

Anyways, let’s not get overwhelmed with the sad life of Bangalore, this is not why I’m back on the blog. Like I said, I haven’t posted anything on the blog in the last 3+ months, it was mostly because I had been catching up with my new job. College did very little to equip me with what was needed at the job. So, you can say, I was busy filling that gap. Finally I have some control over things, so here I am back on the blog!

One thing that I am doing a lot these days is Excel. (I know it is a clever sentence! but, I’m just referring to the Microsoft tool). Probably this is true for all fresh graduates who are currently working as an analyst.

I might not have shared this but I have taken up a flat with 2 of my friends. We shifted to the apartment in September and it has been a really good experience so far. I had created an excel workbook to track our expenses and to solve our fundamental problem – who owes how much to whom.

You might find it useful and it is ideal for people who are living with friends and are spending money on a shared basis. The workbook captures the following metrics –

  • Monthly expenditure for each person
  • Monthly combined and average expenditure
  • Monthly delta between the average expenditure and expenditure of each individual
  • YTD delta (based on the same logic as above) telling you who is to pay whom and how much

 

Below is the screenshot of the summary tab of the workbook –

Summary Tab

 

 

Each monthly tab is the sheet where you would enter the details –

monthly expense

 

The bigger table on left is meant to have all the expenses which cover all of the flatmates. You shouldn’t be entering any value doesn’t cover all of the flatmates in this table as this table is used to calculate the average expense which in turn is used in calculating who owes how much to whom.

The smaller table on right is an auxiliary table to track transactions between any 2 flatmates, and these transactions are not used for calculations in the Summary tab for obvious reasons that this amount shouldn’t contribute to calculation of average expenditure. The table has just 1 cell for each pair of flatmates, so you’d have to use a sum formula incase of multiple entries for a particular pair. There is a similar table on the summary page that shows the YTD pair-wise transaction.

Update – I’m working on expanding the side table as it’s better to have the details like the date and description for these pair-wise interaction as well. The homepage would have an sumif formula to give a 1 cell summary of who is to pay whom and how much.

I have parked the file in google drive. I hope everything is simple enough to be self-explanatory.

[google-drive-embed url=”https://drive.google.com/uc?id=1GNmVVivhwGd_Y6ylEGMoUaW2O19xZp6esSUQb1Wt2uk&export=download” title=”Home Budget Template.xlsx” icon=”https://drive-thirdparty.googleusercontent.com/16/type/application/vnd.google-apps.spreadsheet” style=”download”]

It’s not something out of the world but still solves the purpose without any frills. We’ve not had any problems with the tool so far, we’ve just hosted the file on google drive and thus all the flatmates get a notification whenever there is any update in the file, isn’t that convenient!

All you have to do to start using the file is put the correct names of the flatmates in the summary tab. The monthly tabs use Data Validation for which the list got updated when you replaced the dummy names with the actual names. However, you’d have to update the formulas and the auxiliary table if the number of flatmates is >3.

Let me know in the comments below if you found it useful and are using it or have any suggestions for improvements!

 

 

 

1 year of mapping my music

I have been conducting an experiment since last 1 year by mapping all the music I have been listening. I learned to play many instruments with music lessons in home and I’ve been getting better since then. I started scrobbling my songs on 2nd June 2012 and today it’s exactly 1 year past that date – 2nd June 2013. Well, I was playing against the odds that I’d be able to complete this test according to the Mayans, but I guess that’s an old joke now! The only thing I needed to conduct this experiment was last.fm and its scrobbler. If you don’t know what last.fm is, you should check it out. Last.fm is a music recommendation service. You use Last.fm by signing up and downloading The Scrobbler, which helps you discover more music based on the songs you play.

last_fm_logo

I’ve got some interesting results to share. I spent 750 hours or an entire month’s time in just listening songs in the last 1 year. That’s  certainly the most amount of time I have spent on music in a single year! To put it in perspective, I spent about 100 days of the 365 days I had in sleeping and 30 of the remaining 265 days on music. I don’t know what I did in the rest of the time, should’ve mapped everything!

The whole time of the ‘experiment’ was divided into two parts –  In first 3 quarters of the year of listening to music, all the exploration I did was based on the recommendation of last.fm and by own efforts. I usually listen to the top songs of any band and judge them based on those 6-7 songs, if I really like them, I explore their other songs further, otherwise I move on and put the songs I liked (from those 6-7 songs) into my phone’s library. So most of the times, my laptop is a testing ground for exploring new music and my phone has a collection of the songs I really love to listen.

One more reason behind this habit is I’m sure all my songs on my android phone are scroblled but I’m not sure of that for my laptop. It’s really a slow way of exploring new music because I generally have to put in a lot of effort in getting the songs to my phone. Music is good but it’s not my only love or the top one for me, so maybe I don’t spend a lot of time on improving my listening habits, but that’s the way things have been…

Now coming back to the experiment, a few months ago, say 5 months ago, I was recommended to check out 8tracks by a friend, so I did and it completely changed my workflow of how I listen music. In the last 4 months, I’ve discovered not only lot more new music but also have been listening to it more often. The analysis graphs blew my mind! Now that I compare my data from 3 months back and 6 months back, I think I had been on a loop of same 100-150 songs for a good part of my year, no points for guessing which half. 8tracks is the answer to my problem of discovering new music, the results are really impressive and I love their apps. It’s going to be really big soon and will catch on with public. I’m sure of that. You can check out the playlists I’ve favorited so far by going on to my 8tracks profile.

8tracks

I have the following graph below, it’s not the best thing as I don’t have a snapshot of how the graphs looked 6 months earlier when there was no 8tracks, but I guess you’ll get the idea I want to present – I’m listening to so much more music these days as compared to the past, all thanks to 8tracks. In the pie chart below, the bigger-darker slice of the pie represents the number of unique songs I’ve listened to in the last 3 months and the smaller-lighter slice represents the songs listened to in earlier time (6 months, 12 months and overall).

3months_3

Last.fm was a brilliant idea and it enabled so many people to actually measure their music habits and make sense out of it. One unique thing I inferred from the data on last.fm is that I really really love the hard rock bands that came out Britain in the sixties, some of my personal favorites are Deep Purple, Pink Floyd, Led Zeppelin and The Who. You can add me as a friend on last.fm if you also like the bands I just mentioned, here is the link to my profile. I’d  love to hear your suggestions if you have any for me..

8tracks on the other hand, takes less of a technical route by making their playlists handmade. They have no algorithms to determine how you are thinking, people don’t want that logically determined suggestions,  people want to explore based on what they feel at the moment. Here is a mixtape I had made when I was missing BITS.

8tracks is much more effective that many other alternatives that let you discover new music because it doesn’t forces these new upcoming indie songs onto you but instead the new songs are sandwiched between the popular ones, so that the whole playlist is interesting. It truly is handcrafted radio because there are always songs with which you can sing along, and you want to hear those songs as well while discovering new ones. For me this maintains the balance between the thrill of discovering your next best song and the joy of singing along an old song you’ve already heard hundreds of times. It’s such a simple thing to create playlists of mixed songs and so much more effective for some one like me.

What do you think about my analysis? I’ll try to post this again next year in June 2014.

why instagramming is not that bad

I love the Instagram app, there, I said it! but before you hit the close button thinking this post to be a joke , hear me out on why Instagram is the best thing that happened to phone cameras!

Before Instagram, not many people used to take their phone cameras seriously, it was hardly used to take creative photos, and whatever photos were clicked using a phone cam used to just wither away in the phone memory cards, they hardly got shared on social networks. But, Instagram changed all this! and perhaps that’s why everyone knows about their rise to fame at the Apple app-store and then the overly-hyped-and-reported billion dollar acquisition by Facebook.

The app made me realize that the megapixel count of the camera doesn’t make a difference, all one needs to do to be a good photographer is be creative! Very few apps have an impact level similar to that of Instagram.

Here are some fun facts about Instagram –

  • Instagram launched as an iPhone app in Apple’s App Store on October 6, 2010. By December 2010, Instagram had one million registered users (it took foursquare a year to reach this milestone and Twitter two years). By July 2011, Instagram users had shared over 100 million photographs. It reached 10 million registered users in September that year. The number has hit 100 million by September 2012.
  • The Android version received over 430.000 pre-registrations and when it finally released on 3rd April 2012, it was downloaded over a million times in less than 12 hours.
  • Apple named Instagram app of the year for 2011
  • In April 2012 Instagram was aquired by Facebook for about 1 billion $ in cash and stock
  • More than 1 billion Instagram photos have been uploaded so far
  • roughly 58 photographs are being uploaded each second
  • Instagram gains 1 new user every second.

These are some great numbers! and the good thing is in one survey it was observed many people use the app to just relax and take a creative break! The app is hugely used by people who wants to make their own thing and make a blog creation out of it. The whole process of taking a photo, editing it and sharing it takes less than 5 minutes so you can snap a pic whenever you see something worth clicking because you always have your camera in your pocket now!

This whole experience of taking a good photo which you would have missed earlier is satisfying for every photographer. You don’t even need to do the whole process in a go, you can just click a photo and when you are waiting in a line or traveling you can edit n share by importing the photo from your gallery, so it’s pretty flexible. The app even lets you do all this without a net connection and when you get a network later, the app will share the pending uploads. Kevin Systrom and Mike Krieger, the founders have done an excellent job in designing the UI and UX of the app and thus it is such a hit with both the nerds and not-so-tech-savvy folks alike!

I can say from my experience of using the app from last 1 year that I’ve been able to snap shots which I wouldn’t have taken usually because I believed that I had left my camera at home and there is nothing I can do right now. Infact when I had posted my best shots of 2012, I realized that out of 40 shots that I think were the better than the rest, around 10 were from my phone but the credit goes to Instagram for making me realize the power of the camera that I had always been carrying in my pocket.

So, I would suggest you to give the app a spin once more, you’ll enjoy trying the different filters, it won’t make you a teenage girl if you’re thinking in that direction! So chill and make some good shots!

Another cool thing that is trending these days is #instawalk – it is just like photowalk, but people use instagrams on their phones instead of regular cameras! Instagram blog is a great place to know more about these events. Here is one group of photographers who made a video of their recent instawalk, cool, right! You can check out the Instagram Blog for updates on such events and weekly contests based on themes

However, like myself there are lot of people who are bored of the usual Instagram shots like the food-pic, mirror-self-shot and others. You should totally avoid these ones! 😀

Let me know what your experience has been with the app and I’d love to know why you love/hate it ? For me Instagram is the easiest way to share photos from your camera to your social media channels. Cheers to (phone) photography!

If you want a professional to take pictures of you, visit PMAI Photography NYC.