The structural health monitoring model provides decision-making basis for structural safety control by extracting data information from the characteristic space of various influencing factors of dam. However, during the service of dam, it may be necessary to add measurement points for specific parts and update the original system, which will cause the sample data of new measurement points to be insufficient to support the training model and have a large difference in the distribution of monitoring data before and after the system replacement. Both of them will pose a great challenge to the development of accurate dam behavior prediction models. This paper proposes a hybrid deep transfer learning framework, which can use the knowledge learned from sufficient dam deformation sample data to assist in the deformation prediction of target dams with limited measurement data and different data distributions. The feature extractor based on convolutional neural network (CNN) is used to extract spatial features across source and target dams. The attention mechanism and long short-term memory network (LSTM) search for the domain invariant features between source dam and target dam from the perspective of temporal features. Then, the technology of transfer learning can assist in predicting the deformation of target dam without degrading the performance of deformation prediction model due to domain transformation. Through experiments, it can be found that the framework on same types of target dams can not only significantly improve the deformation prediction performance of target dams, but also provide guidance on how to effectively use the observed deformation data that is not enough to train the prediction model.