Machine Learning Classifier for Seismic Liquefaction Potential Evaluation

 

Sudhirkumar Vinayakbhai Barai

Assistant Professor, Department of Civil Engineering,
Indian Institute of Technology, Kharagpur, Kharagpur, India
Email: skbarai@civil.iitkgp.ernet.in

ABSTRACT

Liquefaction potential assessment has been a very important problem from the point of view of geotechnical engineering. It is well known that many factors such as soil parameters and seismic characteristics influence this problem. Various researchers have attempted to solve this problem using artificial neural networks (ANN), a sub-branch of machine learning (ML). However, many authors have missed important issues such as proper data modeling, ANN model selection, and performance evaluation of ANN for liquefaction potential assessment. Covering these aspects, the present paper intends to provide systematic steps to model liquefaction potential data using a ML classifier.

Keywords: Data collection, Data Modeling, Neural Networks, Liquefaction Potential Assessment, Performance evaluation

INTRODUCTION

Liquefaction is a phenomenon in which the strength and stiffness of a soil is reduced by earthquake shaking or other rapid loading. Liquefaction and related phenomena have been responsible for tremendous amounts of damage in historical earthquakes around the world. Determination of liquefaction potential due to an earthquake is a complex problem from geotechnical engineering field. It is well known that factors such as soil parameters and seismic characteristics influence this problem. Many papers have appeared in the literature to solve this problem mathematically. However, recently such phenomenon has been modeled by various researchers using artificial neural networks (ANN), a sub-branch of machine learning (ML). This modeling was feasible since ANNs can successfully replace existing equation-based models. For seismic liquefaction potential, ANN models provide significant improvements in prediction accuracy over their statistical counter part.

Tung et al. (1993) carried out study using back propagation based neural networks with input as ground shaking intensity, ground water level, depth of liquefiable soil deposit and soil penetration resistance and output as liquefaction occurrence. The study was trained with a selected set of data and tested on the same domain test data and other city test data.

Goh (1994) has used neural networks to model the complex relationship between seismic and soil parameters in order to investigate liquefaction potential. The network uses the standard penetration test (SPT) value, fines content, grain size, dynamic shear stress, overburden stress, earthquake magnitude, and horizontal acceleration at the ground surface as inputs. Goh (1996) has also extended neural network study to assess liquefaction potential from cone penetration test (CPT) data.

Ural and Saka (1998) used back-propagation learning algorithm to train network using actual field soil records. The performance of the network models was investigated by changing the soil and seismic variables including earthquake magnitude, initial confining pressure, seismic coefficient, relative density, shear modulus, friction angle, shear wave velocity and electrical characteristics of the soil. The most efficient and global model for assessing liquefaction potential and the most significant input parameters affecting liquefaction were summarized. A forecast study was performed for the city of Izmir, Turkey. Comparisons between the artificial neural network results and conventional dynamic stress methods were made.

Above-mentioned papers have missed some important issues from civil engineers' perspective. Firstly, proper data modeling issue was not well addressed. Secondly, the basis for selection of ANN model was not clearly defined. Last but not the least, the reliability of performance of ANN model was not well discussed. The aim of this paper is to discuses these aspects and to provide systematic steps to model seismic liquefactions potential data using a ML classifier. These steps can be used as guidelines for applying ML classifier for civil engineering problems.

Remainder of the paper is organized as follows: The text of the paper will briefly review issues such as proper data modeling, ML classifiers selection and their parameters and proper performance evaluation. Finally these issues have been demonstrated with an example of liquefaction potential assessment and results are discussed.

DATA MODELING FOR MACHINE LEARNING CLASSIFIER

In this phase following two steps are involved.

Data collection

‘Raw’ data consist of the collected (measured, sensed, polled, observed, etc.) attribute values describing objects and relations between objects in the application domain. Data coming from systems include the results of manipulating independent or control variables. These data need to be modeled properly for ML classifier. The data model represents a set of data in mathematical form. The model behaves in similar ways as the system. The data model helps in making predictions, classifying new data, trying out “What if” situations, learning about relationships in the data and optimizing the system from which it came. In achieving these goals ‘raw’ data must be processed before training the network.

Preliminary Data Analysis

Preliminary data analysis is one of the essential parts in the ML data modeling. One should not forget the old saying ‘Garbage in – Garbage out’. Data can be online or offline, continuous or discrete, coming from static or dynamic systems. Great care is expected in presenting inputs, removing outliers from the data and use of prior knowledge in finding relevant inputs. The step of preliminary data analysis involves identification of methods of pre-processing and post-processing. It is general practice in ML applications to present pre-processed input data to a ML model and to obtain the required output values from post-processed outputs of a ML model. The pre-processing is the selection of appropriate data subsets for performance as well as consistency reason and the transformation of complex data for better meaningful representation. The post-processing is the sub-selection of voluminous results and application of visualization techniques to understand the data better.

NEURAL NETWORKS AS A MACHINE LEARNING CLASSIFIER

In this phase, varieties of neural network models, such as Hopfield net, Hamming net, Carpenter/Grossberg net, single-layer perceptron, multiplayer network etc. can be used as ML model. The single-layer Hopfield and Hamming nets are normally used with binary input and output under supervised learning. The Carpenter/Grossberg net, however, uses unsupervised learning. The single-layer perceptron can be used with multi-value input and output in addition to binary data. A serious disadvantage of the single-layer network is hyperplane boundary whereas those of two-layer networks may have open or closed convex decision regions (Lippman, 1987). One can select the model depending upon the application domain. The multiplayer network is very popular artificial neural network architecture and has performed well in a variety of applications in several domains including engineering applications. Hence, in the present study, we have used back-propagation based multilayer perceptron.

In multilayer perceptron, the training input vectors are presented to the input layer. Then the intermediate weights of all the connections are adjusted to make the output layer best represent the desired outputs. The process is repeated many times until a predefined error is obtained. By iterating through the training data many times, the neural networks are able to generalize the rules implicit in the data. The method of adjusting weights is called learning rule. The weights are adjusted to minimize the error between input and output vectors and the process is known as delta-rule learning. The most commonly used type of neural network is back-propagation.

MACHINE LEARNING MODEL PERFORMANCE EVALUATION

Both qualitative and statistical criteria can be used for reliable ML model evaluation. There are several methods that have been used to evaluate the performance of ML models such as NN models include: Resubstituion (R), Hold-out (H), Cross-validation such as leave-one-out (L) and k-fold cross validation (K), and Bootstrap (B). Bootstrap being computationally exhaustive, in the present paper, it will not be addressed. Reich and Barai (1999) have given qualitative nature of these basic evaluation methods. They give an idea about the reliability of these evaluation methods for given data in particular NN model. However, they do not include any distinction between different types of errors (e.g., false positive or negative in classification, or positive vs. negative errors in function prediction), although different types of errors might be very different in terms of cost or severeness for the particular engineering applications. However, there is growing interest in the ML community in understanding the properties of these tests. Such properties are derived empirically from many tests on artificial and real databases.

In general, it is common to evaluate the predictive accuracy of models by cross-validation. Either K or L is acceptable. The selection of cross-validation method is based on the size of the data. This method provides reliable estimates of the true error rate, as nearly all the cases are used for training, and all the cases are used for testing. The estimates obtained from this test are nearly unbiased. In general, K was found to be more stable than L, and given its reasonable computational requirements; K is the recommended test for absolute evaluation. For small databases one has to use L or B although there are cases where both fail. When better estimations are required and if computational resources are available one may carry out K exercise I times, which can be denoted as KI. It is sufficient to use I = 10 for this test. In order to get most out of the evaluation process, Reich and Barai (1999) recommend executing all the evaluation methods.

An important aspect when solving practical problems is obtaining the best possible performance out of the data. Therefore it is natural to wish to optimize ML program parameters. However, this requires special attention related to evaluation. Fig. 1 illustrates an estimation method called training-testing-validation (TTV) that can be used for tuning operational parameters and options of programs, and finally estimating the performance of the ML model. In the first step, the data is subdivided into data for model learning and model testing. In the second step, the data for model learning is used to select the best model (i.e. learning approach) and operational parameters. In this step, evaluation is done by K in order to generate better ground for selecting between the different parameters. In the third step, a model is created from the complete model learning set by the best approach and best operational parameters. This model is validated on the testing set. Obviously, the final experiment is an H estimation method with all its limitations.

 


Figure 1. Relative performance of different ML model evaluation methods

LIQUEFACTION POTENTIAL ASSESSMENT EXAMPLE AND DISCUSSION

Problem Definition

The goal of the study is to develop ML classifier for mapping seismic and geological attributes (intensity, depth of liquefiable soil, water level etc.) to assessment of liquefaction potential.

Data Modeling

For the present studies, two data sets were collected from the paper (Tung et al., 1993).

Data set 1: Tangshan earthquake data – The 1976 Tangshan earthquake occurred in the Fengman area of Hebei Province of China. The data set contained information from 81 sites. For each site the data was compiled for local intensity, ground water lever, depth of liquefiable soil layer, soil penetration resistance and liquefaction occurrence. The data set contains the data observed at the site about liquefaction occurrence, and also obtained using Chinese Seismic Code.

Data set 2: Xing Xiang earthquake data – The data were collected on seismicity and local site conditions for 69 sites of the Xing Xiang city of China. The sites data were compiled for local intensity, ground water level, depth of liquefiable soil layer, soil penetration and liquefaction potential based on Chinese Seismic code.

The data needed some preliminary analysis. Both data sets needed conversion. The Chinese intensity scale is divided into 12 levels, which are more or less same as Mercalli Intensity Scale. Standard penetration test (SPT) has been used as an index of soil liquefaction potential for many years. The SPT data has been converted by suitable approach by Tung et al. (1993) and has been reported in their paper.

The attributes of both sets – Ground shaking intensity (MMI), ground water level (m), depth of liquefiable soil deposit (m), and soil penetration resistance (blow count/ft) is used as input attributes for ML classifier. The liquefaction occurrence (0 = no liquefaction, 1=liquefaction) is used as output attribute for these data sets.

Neural Networks Modeling

The ML classifier – Multilayer perceptron was selected owing to its recognized ability to perform regression and classification. The neural network architecture had two hidden layers with 12 and 2 hidden units having sigmoid activation function in each layer. The program was implemented using improved backpropagation in MATLAB Neural Networks Toolbox (Demuth and Beale, 1994). After several exercises, sum square error was selected as 0.0001 and learning rate 0.02, keeping compromise between the accuracy and the computational time. Note that no optimization of the architecture or training parameters was performed. In contrast, common architecture and parameters were selected so that the time consuming exercise will take reasonable time.

Performance Evaluation and Discussion

In this section a comparison between the results reported in Tung et al. (1993) and results obtained from the present study will be made. Thereafter results based on proposed approach for evaluating ML model will be discussed.

From Table 1, it has been observed that for H evaluation method after randomizing the order of the data set the performance of ML classifier worsened substantially (47.5 % error) in comparison to the results reported (2.5 %) by Tung et al. (1993) for observed liquefaction potential assessment. Further, it would be worthwhile to mention that the results reported by Tung et al. (1993) have been for different neural network architecture. However, for R evaluation method, the results have not changed much. Results reported by Tung et al. (1993) can be misleading in the case of ordered data set.

 

Table 1. Comparison of ML classifier performance for observed liquefaction
Study Data set 1
Training Examples 41 R
Data set 1
Testing Examples 40 H
Remarks
Tung et al. (1993) Correct prediction = 40
% pred. error = 2.43
Correct Prediction = 39
% pred. error = 2.5
The data used for training
and testing were ordered
(Refer Tables 2 and 3 of Tung et al. (1993))
Present
Study
Correct prediction = 41
% Ave. pred. error = 0.0
Correct prediction = 21
% Ave. pred. error = 47.5
The data set was randomly ordered
for training and testing and exercise
was carried out for 20 times

 

Table 2. Comparison of ML classifier performance for Chinese seismic code
Study Data set 1
Training Examples
41 R
Data set 1
Testing Examples
40 H
Data set 2
Validation Examples
69
Remarks
Tung et al. (1993) Correct prediction = 37
% pred. error = 9.75
Correct Prediction = 33
% pred. error = 17.5
Correct Prediction= 60 
% pred. error = 13.0
The data used for training
and testing were ordered
and validated on data set 2
(Refer Tables 2, 3 and 4 of Tung et al. (1993))
Present
Study
Correct prediction = 41
% Ave. pred. error = 12.2
Correct prediction = 21
% Ave. pred. error = 47.5
Correct prediction = 63
% Ave. pred. error = 8.69
The data were randomly ordered
for training and testing
and exercise was carried out for 20 times

 

Table 3. Cross-validation results of present study
Evaluation Method Data set 1
ML performance for Liquefaction Observed
(81 examples)
Data set 1
ML performance for Chinese Seismic Code Prediction
(81 examples)
Data set 2
ML performance for Chinese Code Prediction
(69 examples)
Remarks
L Correct prediction = 76
% Ave. pred. error = 6.1
Correct prediction = 69
% Ave. pred. error = 14.81
- Ordered data set was used for training and testing
KI Correct prediction = 78
% Ave. pred. error = 3.70
Correct prediction = 69
% Ave. pred. error = 14.81
Correct prediction = 64
% Ave. pred. error = 7.24
Data set 1 was randomly ordered and 10 times K exercise were carried for 10-folds

 


Figure 2. Comparative study of evaluation method for Observed Liquefaction Potential Assessment

 


Figure 3. Comparative study of evaluation method for Chinese Seismic Code Prediction

It is worthwhile to comment from the above results that without proper evaluation there is no meaning of results. Therefore, appropriate report of the evaluation performed in a study must be a prerequisite. Such report should include explanation of the nature of the data and ML programs, the reason for selecting an evaluation method, the data necessary for checking the evaluation, the data needed to verify statistical tests that are reported, as well as explanation of deviations from anticipated results.

CONCLUSIONS

Paper discussed about the present practice of ML classifier in the field of liquefaction potential assessment. The important issues such as data modeling, ML model selection and ML model evaluation for the problem context were discussed. Finally paper demonstrated the systematic approach for the example domain and gave comparison of ML classifier with the existing published results. It was found that proper evaluation of ML classifier is must before putting into practical use as a reliable ML model. The steps discussed in the paper will form the guidelines for future researchers, who are interested in applying ML classifier for various civil engineering applications.

ACKNOWLEDGEMENT

The work was carried out by author as a BOYSCAST Fellow at Department of Civil and Environmental Engineering, Carnegie Mellon University, Pittsburgh PA 15213, USA.

references

  1. Demuth, H. and Beale, M. (1994). Neural Networks Toolbox-For Use with MATLAB, The Mathworks Inc. Natick, MA.
  2. Goh, A.T.C. (1994). “Seismic liquefaction potential assessed by neural networks.” Journal of Geotechnical Engineering, ASCE. 120(9): 1467-1480.
  3. Goh, A.T.C. (1996). “Neural network modeling of CPT seismic liquefaction data.” Journal of Geotechnical Engineering, ASCE. 122(1): 70-73.
  4. Lippman, R. P. (1987). “An introduction to computing with neural nets.” IEEE ASSP Magazine, 4:6-22
  5. Tung, A. T. Y., Wang, Y. Y. and Wong, F. S. (1993). “Assessment of liquefaction potential using neural networks.” Soil Dynamics and Earthquake Engineering, 12: 325-335.
  6. Ural, D. N. and Saka, H. (1998). “Liquefaction assessment by artificial neural networks.” http://www.ejge.com/1998/Ppr9803/Ppr9803.htm
  7. Reich, Y. and Barai, S. V. (1999). “Evaluating machine learning models for engineering problems.” Artificial Intelligence in Engineering, 13:257-272

 

© 2003 ejge