- \n
- How will AI affect our lives? <\/li>\n\n\n\n
- Will it really take my job? <\/li>\n\n\n\n
- How can I participate in the AI revolution?<\/li>\n\n\n\n
- How can my company operate more efficiently with the help of AI? <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
Meanwhile, it is also widely discussed that AI did not start with chatGPT. We have talked about this in our podcast<\/mark><\/a>, presented at the University of P\u00e9cs' Artificial Intelligence Professional Day,<\/mark><\/a>and at various corporate events upon request from our interested clients. <\/p>\n\n\n\n
can easily lead to misunderstandings. That's why we are launching our article series, in which we will try to compile and explain the most important professional terms in this field from a business perspective. <\/p>\n\n\n\n
![\"A](\"https:\/\/dev.test-it.hu\/wp-content\/uploads\/2024\/07\/data-mining-1-1024x512.jpg\")
<\/figure>\n\n\n\nWhat is Data Mining? <\/h2>\n\n\n\n
In our first article, we will explore data mining. Many of today's artificial intelligence experts started their careers as data miners, and there are many overlaps between the various concepts. <\/p>\n\n\n\n
The goal of data mining is to <\/strong>extract valuable information and patterns from large datasets. In the business world, this means finding something in the data that can be used to support business objectives, such as: <\/p>\n\n\n\n
- \n
- We gain a better understanding of our customers and can communicate with them more effectively. <\/li>\n<\/ul>\n\n\n\n
- \n
- We understand customer behavior and can respond to their needs in a timely manner. <\/li>\n<\/ul>\n\n\n\n
- \n
- We identify suspicious credit card transactions and prevent potential fraud. <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
Unlike traditional statistical methods, the size of the dataset is a crucial aspect here. Data mining techniques and algorithms are characterized by their efficiency with large volumes of data. For example, they can analyze millions of purchase transactions or interactions between customers without needing to take smaller samples and potentially losing relevant information due to sampling. <\/p>\n\n\n\n
How Can I Use Data Mining in Business? <\/h2>\n\n\n\n
Data mining has proven its value in almost every sector of business: <\/p>\n\n\n\n
- \n
- Optimizing Existing Processes <\/li>\n<\/ul>\n\n\n\n
- \n
- Developing New Processes <\/li>\n<\/ul>\n\n\n\n
- \n
- Establishing New Revenue Streams <\/li>\n<\/ul>\n\n\n\n
Characteristics of Data Mining Projects <\/h2>\n\n\n\n
It is also important to highlight another characteristic of data mining: sometimes, we work with data that, due to its size, was not previously used by the organization, making it difficult to predict whether it contains valuable information. This is particularly significant during the preliminary return-on-investment analysis of the project, as we cannot always estimate the expected profit, while the costs are immediately apparent.
Thus, the ROI (Return on Investment) calculation cannot be performed with high quality. Fortunately, there is now a wealth of experience in data mining projects, so with the right expertise and experience, we can provide some estimate that is sufficiently accurate for a project's approval process. However, it is important to emphasize that the organization must understand the characteristics of data mining projects, and the processes must be prepared to handle them appropriately. <\/p>\n\n\n\n![\"Az](\"https:\/\/dev.test-it.hu\/wp-content\/uploads\/2024\/07\/adattarhaz-data-warehouse-1-1024x512.jpg\")
<\/figure>\n\n\n\nBut where do I store all this data? <\/h3>\n\n\n\n
The prerequisite for data mining activities is having access to data. There are several approaches to this: <\/p>\n\n\n\n
- \n
- Data Lake: the large amount of collected data (e.g., from source systems, the web) can be loaded into a data storage layer where it can be managed efficiently and cost-effectively. <\/li>\n<\/ul>\n\n\n\n
- \n
- Data Warehouse: Generally, large enterprises create data warehouses to systematically and structurally store essential data available within the company. <\/li>\n<\/ul>\n\n\n\n
- \n
- Data Mart: A data storage layer that supports a specific business function of the company, where data is specifically prepared for reporting, analysis, and data mining purposes for that particular business area. <\/li>\n<\/ul>\n\n\n\n
The relationship between data mining and project management<\/h2>\n\n\n\n
There are several approaches to managing data mining projects. One of the most widespread is the CRISP-DM<\/strong> (Cross-Industry Standard Process for Data Mining) methodology, which many organizations follow with minor to significant modifications. <\/p>\n\n\n\n
![\"Cross](\"https:\/\/dev.test-it.hu\/wp-content\/uploads\/2024\/07\/CRISP-DM-Cross-Industry-Standard-Process-for-Data-Mining.jpg\")
<\/figure>\n\n\n\n<\/p>\n\n\n\n
<\/p>\n\n\n\n
One of the main advantages of this methodology is that it provides a framework for managing data mining projects that can be applied regardless of the area of application. <\/p>\n\n\n\n
The methodology consists of six phases, through which the project progresses iteratively. This means that a phase may conclude with results that necessitate returning to a previous phase and proceeding again through the appropriate phases: <\/p>\n\n\n\n
- \n
- Identifying business objectives (Business Understanding) <\/li>\n<\/ol>\n\n\n\n
- \n
- Data Understanding <\/li>\n<\/ol>\n\n\n\n
- \n
- Data Preparation <\/li>\n<\/ol>\n\n\n\n
- \n
- Modeling <\/li>\n<\/ol>\n\n\n\n
- \n
- Evaluation <\/li>\n<\/ol>\n\n\n\n
- \n
- Deployment <\/li>\n<\/ol>\n\n\n\n
How can one verify that complex logic is functioning correctly? <\/h3>\n\n\n\n
Data mining is characterized by processing large volumes of data that cannot be easily reviewed in tools like Excel. Therefore, it is often necessary to use Big Data solutions for effective data management. Additionally, data mining procedures often involve complex mathematical algorithms that may not always be easily interpreted from a business perspective. <\/p>\n\n\n\n
Equally important\u2014yet often neglected\u2014is the task of verifying and testing data mining models and the data assets they process. <\/p>\n\n\n\n
- \n
- Have all available and relevant resources been utilized? <\/li>\n<\/ul>\n\n\n\n
- \n
- Is the quality of the data used adequate? <\/li>\n<\/ul>\n\n\n\n
- \n
- Is the model's result relevant from the perspective of the actual business objectives? <\/li>\n<\/ul>\n\n\n\n
- \n
- Could the application of the model have unintended negative consequences for the business operations? <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
By applying appropriate testing and validation methodologies, unintended negative consequences of data mining results can be avoided. <\/p>\n\n\n\n
![\"Adatok](\"https:\/\/dev.test-it.hu\/wp-content\/uploads\/2024\/07\/adatbanyaszat-1024x512.jpg\")
<\/figure>\n\n\n\nWhat tools do data miners use? <\/h2>\n\n\n\n
During data mining, a variety of tools can be used; however, some are more widely adopted. <\/p>\n\n\n\n
Among programming languages <\/h3>\n\n\n\n
- \n
- SQL: Provides excellent capabilities for data extraction and potential transformation. <\/li>\n<\/ul>\n\n\n\n
- \n
- Python and\/or R: These can cover the entire data mining process, but many use them alongside SQL for data manipulation, model development, and validation. In recent years, Python has started to dominate over the two languages. <\/li>\n<\/ul>\n\n\n\n
Low code \/ No code tools: <\/h3>\n\n\n\n
- \n
- KNIME<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- \n
- RapidMiner (Altair)<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- \n
- SAS enterprise Miner<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
<\/p>\n\n\n\n
In addition, all relevant tools for performing data mining activities are also available in the service offerings of major cloud providers: <\/p>\n\n\n\n
- \n
- Google Cloud Platform<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- \n
- Microsoft Azure<\/mark> <\/a><\/li>\n<\/ul>\n\n\n\n
- \n
- AWS<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- \n
- Alibaba Cloud<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
How does data mining relate to Generative Artificial Intelligence? <\/h2>\n\n\n\n
We often tell business representatives that they don't need to worry too much about whether a solution to a business problem falls under data mining, machine learning, or artificial intelligence. There are overlaps between these areas, and it is rare for someone to be concerned about achieving business value with only one approach or the other.
<\/p>\n\n\n\nGenerative artificial intelligence (e.g., ChatGPT) has, however, heavily benefited from both text mining and machine learning. From text mining, Natural Language Processing (NLP) has provided essential foundations, while from machine learning, neural network approaches have laid much of the groundwork. This combination enables engaging conversations today with systems like ChatGPT (OpenAI) or Gemini<\/mark><\/a> <\/mark>(Google). <\/p>","protected":false},"excerpt":{"rendered":"
The goal of data mining is to extract valuable information and patterns from large datasets... <\/p>","protected":false},"author":4,"featured_media":6761,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[63],"tags":[64,67,66,65],"class_list":["post-6749","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-aidata","tag-adatbanyaszat","tag-data-mining","tag-mesterseges-intelligencia","tag-projektmenedzsment"],"acf":[],"_links":{"self":[{"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/posts\/6749","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/comments?post=6749"}],"version-history":[{"count":32,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/posts\/6749\/revisions"}],"predecessor-version":[{"id":6821,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/posts\/6749\/revisions\/6821"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/media\/6761"}],"wp:attachment":[{"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/media?parent=6749"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/categories?post=6749"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev.test-it.hu\/en\/wp-json\/wp\/v2\/tags?post=6749"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- Alibaba Cloud<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- AWS<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- Microsoft Azure<\/mark> <\/a><\/li>\n<\/ul>\n\n\n\n
- Google Cloud Platform<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- SAS enterprise Miner<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- RapidMiner (Altair)<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- KNIME<\/mark><\/a> <\/li>\n<\/ul>\n\n\n\n
- Python and\/or R: These can cover the entire data mining process, but many use them alongside SQL for data manipulation, model development, and validation. In recent years, Python has started to dominate over the two languages. <\/li>\n<\/ul>\n\n\n\n
- SQL: Provides excellent capabilities for data extraction and potential transformation. <\/li>\n<\/ul>\n\n\n\n
- Could the application of the model have unintended negative consequences for the business operations? <\/li>\n<\/ul>\n\n\n\n
- Is the model's result relevant from the perspective of the actual business objectives? <\/li>\n<\/ul>\n\n\n\n
- Is the quality of the data used adequate? <\/li>\n<\/ul>\n\n\n\n
- Have all available and relevant resources been utilized? <\/li>\n<\/ul>\n\n\n\n
- Deployment <\/li>\n<\/ol>\n\n\n\n
- Evaluation <\/li>\n<\/ol>\n\n\n\n
- Modeling <\/li>\n<\/ol>\n\n\n\n
- Data Preparation <\/li>\n<\/ol>\n\n\n\n
- Data Understanding <\/li>\n<\/ol>\n\n\n\n
- Identifying business objectives (Business Understanding) <\/li>\n<\/ol>\n\n\n\n
- Data Mart: A data storage layer that supports a specific business function of the company, where data is specifically prepared for reporting, analysis, and data mining purposes for that particular business area. <\/li>\n<\/ul>\n\n\n\n
- Data Warehouse: Generally, large enterprises create data warehouses to systematically and structurally store essential data available within the company. <\/li>\n<\/ul>\n\n\n\n
- Data Lake: the large amount of collected data (e.g., from source systems, the web) can be loaded into a data storage layer where it can be managed efficiently and cost-effectively. <\/li>\n<\/ul>\n\n\n\n
- Establishing New Revenue Streams <\/li>\n<\/ul>\n\n\n\n
- Developing New Processes <\/li>\n<\/ul>\n\n\n\n
- Optimizing Existing Processes <\/li>\n<\/ul>\n\n\n\n
- We identify suspicious credit card transactions and prevent potential fraud. <\/li>\n<\/ul>\n\n\n\n
- We understand customer behavior and can respond to their needs in a timely manner. <\/li>\n<\/ul>\n\n\n\n
- We gain a better understanding of our customers and can communicate with them more effectively. <\/li>\n<\/ul>\n\n\n\n