Analysing Data Storage: Data Lakes Vs Data Warehouses

-


Big data and analytics are now at the forefront of digital marketing strategies, as Martin (2021)  explains “Influential marketing ideas are now determined by analytics and big data. By utilizing past data and predictive analytics, businesses can now generate a better return on investment (ROI)”. For businesses to be able to utilise this data they must have the infrastructure and procedures in place to handle and store this data correctly. The importance of data in marketing is not to be underestimated.


There are two main types of data storage solutions: Data Lakes and Data Warehouses. 


Data lakes contain large amounts of unfiltered, raw data that a business collects from interaction with its customers. While data lakes are a helpful way to store raw data and this method is highly accessible and easy to update, it is an inefficient way for a business to process its data for digital marketing purposes. Mainly used by data scientists it is useful for machine learning, however not easily applicable for marketers as the vast amount of data might not be relevant to the business and could be of poor quality without data governance in place. However, data lakes are helpful if a company wants to analyse and review a more thorough and deeper level of data than they would find in a data warehouse. Artificial intelligence is widely used in data lakes to help characterise and format this data. The vast quantities of data make it difficult to apply this data but the processes for this technology are always improving. An example of a business moving to a data lake model is Amazon. The sheer amount of data they process is too vast for a warehouse, so as their CTO Werner Vogels explains “If you wanted to combine all of this data in a traditional data warehouse without a data lake, it would require a lot of data preparation and export, transform, and load (ETL). You would have to make trade-offs on what to keep and what to lose and continually change the structure of a rigid system.” (Vogels, 2021)


Data Warehouses only store processed, structured data meaning that the data held has a specific purpose for the business. This method is growing more popular with business as, in the current climate, the drive to have a personalised user interface and experience for customers is paramount. The fact that data warehouses have specific, relevant data makes them a great option for a business’s data storage needs. Having a data warehouse in place is not enough however, as George Fraser explains, “To get the most out of data in today’s world, you need a data warehouse that can handle the three V’s of big data: volume, variety and velocity. More specifically, the technology must be able to store millions of pieces of data from multiple — and frequently changing — data sources.” (Fraser, 2021). One of the leader’s in data warehouse tech is Oracle. Their platform allows companies to personalise marketing programmes and build customer profiles from third party sources. The ability to use these sources quickly and effectively make data warehouses an attractive option for businesses.


In conclusion, there seem to be endless possibilities when it comes to data storage and this technology is constantly evolving. Currently, in my opinion, data lakes seem more suited to larger corporations (Amazon, Google, etc.) who collect vast amounts of data and the loss of some of this raw data would give them an inaccurate picture for their marketing tactics. While data warehouses seem to suit SME’s with a more focused target audience and the processed nature of the data collected will give them specific results which will inform them of the correct marketing strategy to adopt. Will we see the development of a hybrid Lake/Warehouse in the future? A Lake House if you will? Perhaps


Author: James White


#dataprocessing #datalakes #datawarehouses #datastorage #SME #digitalmarketing


References


Fraser, G., 2021. Council Post: The Importance Of A Data Warehouse In Marketing. [online] Forbes. Available at: https://www.forbes.com/sites/forbestechcouncil/2020/10/30/the-importance-of-a-data-warehouse-in-marketing/?sh=179b5cac4139 [Accessed 25 February 2021].


Martin, N., 2021. Why Utilizing Data In Your Digital Marketing Strategy Is So Essential. [online] Forbes. Available at: https://www.forbes.com/sites/nicolemartin1/2019/10/22/why-utilizing-data-in-your-digital-marketing-strategy-is-so-essential/?sh=4a1fe34cd525 [Accessed 23 February 2021].


Vogels, W., 2021. How Amazon is solving big-data challenges with data lakes - All Things Distributed. [online] Allthingsdistributed.com. Available at: https://www.allthingsdistributed.com/2020/01/aws-datalake.html [Accessed 25 February 2021].

 

 


Comments

  1. Udit Shokeen2 March 2021 at 09:29

    In my opinion too, data warehouses are a better choice for small businesses as data lakes seem to be a hassle and more work needs to be put into it due to its still improving technology in relation to inputs of large data. Data warehouses on the other hand provides a personalized user interface and is much easier to use. SMEs as it is wont have much data as compared to large organizations. Data warehouse seems to be a suitable option for SMEs.

    ReplyDelete
  2. Sergio Augusto12 March 2021 at 01:57

    Great content James! The explanation about storage is clear and objective. Thanks for that. I think small and medium businesses owners should be attentive to these facts, critically thinking in the best way of accessing insights about their customers and behaviours. In my opinion, both ways have their own benefits and disadvantages. If I were a company who makes decisions based on trends of fashion, for example, unstructured data would be the best fit for me. This means data lakes would be the best option for me. On the other hand, if I just want to analyse the number of sales from a specific branch of my company, structured data would apply and, also, data warehouses storage would suit me better.

    Always nice to understand a little bit of data analytics to absorb the best out of it.

    ReplyDelete
  3. Natassya Coelho14 March 2021 at 13:08

    An excellent view, James! Due to the current moment that we are experiencing, we can observe how data has become the most important asset for an organization, and understanding how it should be stored is the key component to the success of any company. From my point of view, the data warehouse is a type of system that helps companies to analyse data from different sources, separate analytical data from transactional data and put the information together in a certain way. I would say that I agree with your point of view that it is recommended for small and medium-sized organizations.
    Your content brought me clarity on the importance of data storage for business. Thank you so much for that.

    ReplyDelete

Post a Comment

Popular posts from this blog

The importance of Knowledge Management and how data analysis informs marketing strategies

-
Knowledge management is crucial for companies to succeed. Knowing your business and your customer’s needs, wants and behaviours help businesses to inform their marketing strategies, increase their revenue and their market share. In the digital age, the majority of this knowledge comes in the form of data. However, data gathered by businesses is unrefined and difficult to read. A large company would have thousands of rows and columns of raw data on spreadsheets which would be extremely difficult to read and impossible to draw insights from. As Steve Denning stated “Nothing can be managed unless and until it is measured.   Because knowledge per se has no value in itself, it has to be measured in relation to the strategy it is intended to accomplish. ” (Denning, 2021) . This blog will outline the knowledge management and analytics processes which transform this raw data into actionable information. The two main analytic types that companies use to analyse their data are information ...
Read more

The Importance of Data Governance and how it can help small and medium businesses succeed.

-
As we know data is a valuable asset to businesses and needs to be governed and secured in an efficient manner while complying with regulations. Firstly, we must understand and define what data governance means. “ CIO.com  put forth an excellent definition earlier this year: “Data governance defines roles, responsibilities, and processes for ensuring accountability for and ownership of data assets across the enterprise.” (Iversen, 2021) . While the primary function of data governance is to comply with regulations and address security concerns, a company can use this to improve decision making and optimise their products and services. Data governance systems can be used to organise data for specific uses and target essential KPI’s if utilised correctly. This blog will outline the benefits of an effective data governance system and how it can help businesses be successful.   A strong data governance system can benefit a business in a number of different ways. Firstly, puttin...
Read more

Two ways that data analytics can satisfy customer needs

-
Image
Even before considering using the power of data analytics for small or medium companies, the main questions should be how those analytics are going to help businesses to know their consumers, and how it can be useful and consistent to leaders and managers to make an effective application of them. This is considered one of the main challenges to independent and established owners, the value challenge. The value challenge in data analytics is relatable to the extraction of insights from the data collection and how we can interpret them. (AWS, no date)  In this case, we have two main approaches to read, comprehend and take useful outcomes from it: Prescriptive and Predictive. Both are distinct by default and serve different purposes that can enhance knowledge management within small and medium businesses. Credit: Georgia de Lotz (Unsplash) Beforehand, it is important to highlight that there are some important aspects of how giving value and generating management knowledge, as it was d...
Read more