Question: Thank you so much for making yourself available on such a short notice Satish. As we all know the big data fever is really catching up the technology industry and looks like it is bound to play a very crucial role in business intelligence. What do you really feel? Is Big Data really here to stay?
Satish: Ofcourse! Big data is here to stay big time! I won’t really call this as just another fever but it is more like the ‘next best thing to do’ for Business Intelligence. However, there are various aspects of Big data that organizations need to understand before taking a call. Yes, it is a great tool that lets you understand the business, customers, products, competitors via capturing, processing and analyzing bulks and bulks of data that usually would make no sense at all but infact would actually be the most essential elements in building a systematic roadmap. Big data has the power and ability of literally helping businesses in increasing sales, minimizing the cost and providing a better customer service or improved quality products.
Question: So what is it that Big data offers that traditional systems could not?
Satish: See, if you ask me if this was possible with even traditional systems like relational databases(RDBMS) I would still say yes, it was, but the real question is ‘At what cost?’. RDBMS technology has been around since 70’s and the only reason why it still exists is because RDBMS knows one job too well and that is thinking in rows and columns and fetching data quickly and rapidly with super normalized forms. But as you can understand, with the advent of time the traditional systems went piling new datatypes and that is where they started failing grossly. There was a need for a new system where virtualization/cloud could do its magic. Yes RDBMS scales too, but only with an expensive hardware and ofcourse they are powered by legacy software's that are not only expensive but were also designed for a much primitive era. Big data has allowed to accumulate every type of data for analyzing and making good use of virtualization.
Question: Great, can you throw some light as to how a banking applications can leverage its data capabilities with Big data?
Satish: Sure. A bank records GB and TB’s of transactional data of their customers that include their info, history of transactions and so on. This data however till date has been sufficing their basic intention of analyzing the credit and debit history of each customer, however where Big data can make a big difference is that the same data can be used in analyzing the records to such an extent that even minutest details like the customers favorite drinks, his favorite hangouts, the car he drives, the service station he prefers, the gas he fills, the online shopping websites he shops on, the malls he frequently visits and even the most probable missing piece of electronic devices in his collection would be analyzable. This data can be used for either predicting where he is going to shop next with reference to his buying trends or someone else's buying trends that look similar to his OR to generate loyalty by rewarding him with something that he likes rather than something that he won’t really use. There can be 'n' other use case stories that can bring the best in the bank for better and quality business. Needless to say, in order to keep track of these details of massive data, you need big data on your table because though a singular tupple may carry no value in driving decisions however when you put it in a massive data flock, it starts making the perfect sense.
Question: So, when do you think would be the right time for any organization to venture into big data?
Satish: Well, its never a wrong time to do the right thing. One thing that organizations need to realize at this moment is that instead of just sustaining and surviving in this competitive and contested markets, the fun is in wiping the competition clean by making them irrelevant by hitting the right nerves. All in all, all I would say is that they need to start preparing, and the time is now! For this preparation, the data can certainly be in megabytes, gigabytes, petabytes or zettabytes – in increasing order for better success.
Question: How to get there? Can you give us a brief intro to the complete process?
Satish: See, it is actually a simple 3 step approach. First step is to acquire the data which can be in any form. All you need is data, data and more data. Second step is to transform this data only to be stored in multiple versions in order to make it more structured. Also there is need to make atleast 2 copies of this data for putting a business continuity plan. Honestly saying it’s all about capacity, cost and reliability. At Equations Work we have lately tried the MapReduce technique for slicing-dicing of the data and our experience so far has been fantastic. Last step is analytics where companies actually make sense out of this data. Platforms like Tableau and Cognos can prove to be the best tools for making the perfect charts that would allow in bringing the best interactive visual representations out of your data.
Question: Equations Work has been a direct or an indirect part of over 4+ Big data projects till date. Can you shortly brief us about any one of them?
Satish: Sure. Initially, a retail giant approached us for providing a predictive analysis solution for one of their existing Big data setups. By the time we signed up the project, we had already started analyzing the data that would assist in showing the shopping patterns of the customers. We categorized these customers into various categories and then there were sub categories and observation charts over general buying trends. We started testing our assumptions on these categories by giving a recommended product. For eg. People who were interested in wine, were also distinctly shown offers on cheese and chips. These kind of analytic helped the giant in improving the sales by atleast 17%. Going a step further, the data was increased yet further and these analytic now spawned the season wise buying patterns, average consumption and many more insights. The results have been phenomenal. With the kind of potential Big Data has, one day even your local grocery store will have the potential of recommending the best products for you based on your current purchases over the past 3-4 months and not just your current cart.
Question: Lastly Satish, can you give the decision makers from our readers a quick five pointer considerations for moving on to the right Big data technology?
Satish: Yes, whereas most of the technologies are matured enough to handle most of the production use cases, decision makers should consider various dimensions while selecting the best for themselves. They go as follows.
- Software Licensing Models.
- Offline Support
- Development tools & Community
- Agility
- Purpose
Well, Thank you Satish for your time and inputs. We are sure it will help our readers in generating the best out of this technology.
(In case if you have any queries or want to know more, feel free to get in touch with blogs@equationswork.com and we will be happy to get in touch with you.)
Tags:
BigData
,
Warehousing
Thanks Vishal for sharing this post. It is very useful and gives adequate knowledge about Big Data.
ReplyDeleteSatish sir can you give some inforemation about server/infra requirement for Big Data project.
ReplyDeleteThis is very nice Post.
ReplyDelete