建模数据的准备，是指在未经过整合处理的原始数据中，通过整合处理形成后续

The preparation of modeling data refers to the process of forming various data sets required for subsequent modeling through integration processing from the original data that has not been integrated. The variables of these data sets are the independent variables required for subsequent modeling. . The data preparation process may be processed repeatedly due to the needs of the model, and there is no order regulation. The main work of data preparation includes the selection of data variables and data volume, standardized processing of related variables according to model requirements, and processing of missing values ​​and outliers at the same time. The preparation of modeling data may be the most time-consuming step in the data mining process, and may even account for more than half of the entire data mining process. The main work flow of data preparation is: selecting data, cleaning data, building data, Consolidate data, format data, and prepare data sets. <br>In the establishment of the data model, the first step is to select the variables involved in the data modeling. The selection of variables is very important. If too many variables are selected and there are many irrelevant variables, the effect of the model will be weakened. On the contrary, if there are too few variables, it will not be comprehensive. Reflects the various attributes that affect the dependent variable, and also affects the validity of the model. The second is to analyze the correlation between the independent variable and the dependent variable, as well as the correlation between the independent variables, to ensure that the independent variable and the dependent variable have a correlation, if there is no correlation, the irrelevant variable can be directly eliminated; There is a strong correlation between variables, and direct modeling is prone to overfitting. Therefore, if there is a strong correlation between independent variables, the principal component analysis method can be used to reduce the dimensionality to generate new ones with little correlation. The comprehensive index can also be processed by eliminating similar variables. Finally, one or more data mining techniques are selected, and the parameters are continuously adjusted in the process of data modeling, in order to optimize the model. It is usually necessary to use a variety of data mining algorithms to model the data, and finally compare the accuracy and effectiveness of the training results of various algorithms, so as to select the most suitable algorithm.

The preparation of modeling data refers to the process of forming the various data sets required for subsequent modeling through integration processing in raw data that have not been consolidated, i.e. the variables required for subsequent modeling. The process of data preparation can be repeated due to the needs of the model and is not sequential. The main task of data preparation involves selecting data variables and amounts of data, standardizing the processing of related variables according to model requirements, and processing both missing and outliers. The preparation of modeling data can be one of the most time-consuming steps in the data mining process, and may even account for more than half of the overall data mining process, whose main workflows for data preparation are selecting data, cleaning data, building data, consolidating data, formatting data, and data set preparation.<br>The establishment stage of data model, first of all, to choose the variables involved in data modeling, variable selection is very critical If too many variables are selected and there are many unrelated variables, it will weaken the effect of the model, on the contrary, if there are too few variables, it can not fully reflect the impact of various attributes of the dependent variables, the same impact on the effectiveness of the model. The second is to analyze the correlation between the argument and the dependent variable, as well as the correlation between the argument, to ensure that the argument and the dependent variable are related, if there is no correlation is an unrelated variable can be directly eliminated; If there is a strong correlation between the arguments, direct modeling is prone to overfitting, so for the strong correlation between the arguments, the main component analysis method can be reduced, resulting in a new comprehensive index with little correlation, or can be processed by eliminating similar variables. Finally, one or more data mining techniques are selected, and the parameters are constantly adjusted in the process of data modeling, with the aim of bringing the model to the best possible position. It is often necessary to model the data using a variety of data mining algorithms, and finally compare the results trained by various algorithms with accuracy and effectiveness, so as to select the most suitable algorithm.

The preparation of modeling data refers to the process of forming various data sets for subsequent modeling through integration in the original data without integration. The variables of these data sets are the independent variables for subsequent modeling. The process of data preparation may be repeated due to the needs of the model, and there is no order. The main work of data preparation includes the selection of data variables and data volume, the standardized processing of related variables according to the model requirements, and the processing of missing values and outliers. The preparation of modeling data may be the most time-consuming step in the process of data mining, and may even account for more than half of the workload of the whole data mining process. The main workflow of data preparation is: selecting data, cleaning data, building data, integrating data, formatting data and data set preparation.<br>In the data model building stage, the first step is to select the variables that participate in the data modeling. The selection of variables is very critical. If too many variables are selected and there are many irrelevant variables, the effect of the model will be weakened. On the contrary, if too few variables are selected, they can not fully reflect the attributes of the dependent variables, which also affects the effectiveness of the model. Secondly, it is necessary to analyze the correlation between independent variables and dependent variables, as well as the correlation between independent variables, so as to ensure that the independent variables and dependent variables are related. If there is no correlation, it belongs to irrelevant variables and can be directly eliminated; If there is a strong correlation between the independent variables, direct modeling is easy to produce over fitting phenomenon. Therefore, if there is a strong correlation between the independent variables, we can use the principal component analysis to reduce the dimension and generate new comprehensive indicators with little correlation, or eliminate similar variables. Finally, one or more kinds of data mining technologies are selected to adjust the parameters continuously in the process of data modeling, so as to optimize the model. Usually, we need to use a variety of data mining algorithms to model the data, and finally compare the accuracy and effectiveness of the training results of various algorithms, so as to select the most suitable algorithm.<br>