Data Analytics x Velocity: How fast are you processing data?

-


A road full of car lights shoot in high velocity. The image illustrates the word "fast" relating to data collection and processing.
Credit: Jason Corey (Unsplash).


A range of companies in the industry uses data analytics as a source of reading insights from their customers, comprehending external reality and creating new assets and products directly driven to their customers. They have been key elements for businesses that operate either in the digital and physical world, making it possible to forecast trends and consumers change of habits. (Djafri, Bensaber, Adjoudji, 2018) Collection and processing of data demand systems to deal with five main challenges to analysts and scientists. Velocity is one of them and is defined as the time taken to collect and process data. (AWS, no date) In this post, we will be covering two distinct approaches to process data, their pros and cons, and business solutions for different cases.

 

There are two categories of data processing: Batch process and stream process. Both have their advantages and disadvantages in the lights of what businesses need to process and get insights into. The batch process is the process of a large volume of data. For instance, analytics of point-of-sales in a retail chain or payroll processes. (Shiff, 2020) Users who analyse the data collected can schedule the collection and processing time of the data. Also, they can receive it on a periodic base at irregular times once a certain amount of data is collected. (AWS, no date) Businesses have to critically assess their needs concerning the pros and cons of batch processing. Advantages include low-cost workload, and managers have complete control of when they want this data process. On the other hand, the data will be complex to analyse by its large size, and it demands staff to know how to handle any issues that might appear.

 

The data stream process is described as the income of small pieces of data in near real-time and real-time. This specific process is executed within minutes or seconds as soon as data is collected. Differently from the first approach, now data is collected and processed in different steps, being gathered from multiple sources and in parallel with other kinds of data. Users can work simultaneously in different streams of data, interpreting insights separately and, because of the near-real-time or real-time factor, providing businesses with up-to-date analytics. (AWS, no date) For example, a company that develops products based on social media trends or solutions for what people are searching through their smart devices, like Echo Dot or Google Voice Assistant. Some challenges are the veracity of the data coming in and how the output of analysis has to be faster as much the input of raw data. (Shiff, 2020) Otherwise, there might be issues with the volume of data and the storage of it.

 

To conclude, the velocity of data processing is the time taken to collect and process the data. Batch and stream processes are the two characteristics of this challenge, having pros and cons in the appliance on data analytics. Businesses owners have to evaluate the procedure that fits them better, minding that analysis is important to understand the impact of their brand or product in the external environment.

 

Author: Sergio Augusto dos Santos Rabelo Silva

 

#DataAnalytics #DataCollection #DataProcessing #BusinessSolution

 

References

 

Djafri, Laouni; Djamel Amar Bensaber; Adjoudj, Reda. (2018) ‘Big Data analytics for prediction: parallel processing of the big learning base with the possibility of improving the final result of the prediction', Information Discovery and Delivery, 46(3), pp. 147-160. 

 

'Data Analytics Fundamentals' (no date). Available at https://www.aws.training/Details/eLearning?id=35364 (Accessed: 18 February 2021)

 

Shiff, L. (2020) ‘Real Time vs Batch Processing vs Stream Processing’, BMC Blogs. Available at: https://www.bmc.com/blogs/batch-processing-stream-processing-real-time/ (Accessed: 25 February 2021)

 

Comments

  1. Udit Shokeen2 March 2021 at 09:18

    I have to say that this post would be a great help for SMEs looking to analyze their data. Reading this post would definitely enlighten their knowledge on both batch process and stream process making it easy for them to make an apt choice that goes well with their business.

    ReplyDelete
  2. Natassya Coelho8 March 2021 at 08:13

    Velocity is one of the big challenges of Big data analytics. In addition to your text, from a business perspective, I would say that batch processing and stream process are the case of efficiency and speed respectively. Batch processing is better utilised in situations where organisations don't need real-time results, and when processing a large volume of data to get more detail to generate accurate insights is important than the speed of processing. On the other hand, the stream process is the golden key to analytics results in real-time. It allows the system process data immediately as soon as the data arrives. For example, it is a perfect tool to detect fraudulent actions in real-time.
    The decision to utilize one of those systems in business is related to the goals of the organisation.
    Thank you for your point of view, Sergio!

    ReplyDelete
  3. Unknown8 March 2021 at 08:23

    This is a great post about data velocity Sergio. Understanding the processing of data is really important for businesses. The differences in batch and stream processes is explained here very well. I suppose the size of the business would come into play when deciding which process suits their needs better. Moreover, it seems that the type of data collected really determines which process to go for and because of this, businesses must research this and also develop methods to process this data efficiently. Looking forward to your next blog Sergio.

    James

    ReplyDelete

Post a Comment

Popular posts from this blog

The importance of Knowledge Management and how data analysis informs marketing strategies

-
Knowledge management is crucial for companies to succeed. Knowing your business and your customer’s needs, wants and behaviours help businesses to inform their marketing strategies, increase their revenue and their market share. In the digital age, the majority of this knowledge comes in the form of data. However, data gathered by businesses is unrefined and difficult to read. A large company would have thousands of rows and columns of raw data on spreadsheets which would be extremely difficult to read and impossible to draw insights from. As Steve Denning stated “Nothing can be managed unless and until it is measured.   Because knowledge per se has no value in itself, it has to be measured in relation to the strategy it is intended to accomplish. ” (Denning, 2021) . This blog will outline the knowledge management and analytics processes which transform this raw data into actionable information. The two main analytic types that companies use to analyse their data are information ...
Read more

The Importance of Data Governance and how it can help small and medium businesses succeed.

-
As we know data is a valuable asset to businesses and needs to be governed and secured in an efficient manner while complying with regulations. Firstly, we must understand and define what data governance means. “ CIO.com  put forth an excellent definition earlier this year: “Data governance defines roles, responsibilities, and processes for ensuring accountability for and ownership of data assets across the enterprise.” (Iversen, 2021) . While the primary function of data governance is to comply with regulations and address security concerns, a company can use this to improve decision making and optimise their products and services. Data governance systems can be used to organise data for specific uses and target essential KPI’s if utilised correctly. This blog will outline the benefits of an effective data governance system and how it can help businesses be successful.   A strong data governance system can benefit a business in a number of different ways. Firstly, puttin...
Read more

Two ways that data analytics can satisfy customer needs

-
Image
Even before considering using the power of data analytics for small or medium companies, the main questions should be how those analytics are going to help businesses to know their consumers, and how it can be useful and consistent to leaders and managers to make an effective application of them. This is considered one of the main challenges to independent and established owners, the value challenge. The value challenge in data analytics is relatable to the extraction of insights from the data collection and how we can interpret them. (AWS, no date)  In this case, we have two main approaches to read, comprehend and take useful outcomes from it: Prescriptive and Predictive. Both are distinct by default and serve different purposes that can enhance knowledge management within small and medium businesses. Credit: Georgia de Lotz (Unsplash) Beforehand, it is important to highlight that there are some important aspects of how giving value and generating management knowledge, as it was d...
Read more