 Original Article
 Open Access
 Published:
Nonrigid registration of medical images based on \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensor product Bspline
Visual Computing for Industry, Biomedicine, and Art volume 5, Article number: 5 (2022)
Abstract
In this study, a nontensor product Bspline algorithm is applied to the search space of the registration process, and a new method of image nonrigid registration is proposed. The tensor product Bspline is a function defined in the two directions of x and y, while the nontensor product Bspline \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) is defined in four directions on the 2type triangulation. For certain problems, using nontensor product Bsplines to describe the nonrigid deformation of an image can more accurately extract the fourdirectional information of the image, thereby describing the global or local nonrigid deformation of the image in more directions. Indeed, it provides a method to solve the problem of image deformation in multiple directions. In addition, the region of interest of medical images is irregular, and usually no value exists on the boundary triangle. The value of the basis function of the nontensor product Bspline on the boundary triangle is only 0. The algorithm process is optimized. The algorithm performs completely automatic nonrigid registration of computed tomography and magnetic resonance imaging images of patients. In particular, this study compares the performance of the proposed algorithm with the tensor product Bspline registration algorithm. The results elucidate that the proposed algorithm clearly improves the accuracy.
Introduction
Medical imaging technologies are essential in modern medical diagnosis, and provides indispensable help for doctors to accurately determine the condition of patients’ lesions. At present, the most commonly used medical imaging techniques include computed tomography (CT), magnetic resonance imaging (MRI), positron emission computed tomography (PET), and ultrasound (US). However, in the treatment cycle of each patient, the human tissue inevitably undergoes a local deformation, and doctors cannot accurately judge the changes in the lesions using medical images. The medical image nonrigid registration algorithm can solve this problem to a certain extent, which is a search for the transformation relationship between images collected at different times or using different instruments [1]. According to the existing threedimensional (3D) modeling technologies [2, 3], Fig. 1 illustrates a 3D model of hematoma and nerve bundle before and after registration, respectively.
In the 1980s, the medical image scanning technology was relatively backward, and most registrations were only applied to rigid registrations in the same modalities. In 1992, Brown [4] classified registration methods according to the complexity of the transformation model and proposed four components of registration, including the feature space, similarity measure, search space, and optimization strategy. In the mid to the late 1990s, with the application of CT, MRI, PET, US, and other medical imaging equipment in clinical diagnosis, doctors and scientific researchers realized that medical images of different modalities can provide various information and a solid basis for diagnosis. Maes et al. [5] applied the concept of mutual information to multimodal medical image registration and used the mutual information between the reference and moving images as a similarity measure for image registration. In 1999, Rueckert et al. [6] proposed a free deformation model based on a Bspline. Subsequently, they applied the uniform multilevel Bspline method of discrete data interpolation, proposed by Lee et al. [7], to the nonrigid registration of breast MRI, which could restore the motion and deformation of the breast [8]. Subsequent methods attempted to improve this method. In the twentyfirst century, the rapid development of computer hardware has promoted studies on the image registration in 3D and even fourdimensional fields, particularly on the nonrigid registration between different images of a certain patient. In 2001, Schnabel et al. [9] proposed a nonuniform multilayer Bspline to improve the registration efficiency. In 2013, Oliveira and Tavares [10] introduced a new enhanced Bspline method to register the plantar pressure image sequences in time and space simultaneously. Although the traditional Bspline registration algorithm can achieve reasonable results, it is difficult to obtain accurate results for images with large local and global distortions. The accuracy of the algorithm is low when the smoothing term in the cost function is large. When the smoothing term is considerably small or close to 0, overregistration occurs, severely destroying the image topology. Therefore, some areas cannot be registered. In view of this problem, Ji et al. [11] presented a nonrigid registration algorithm based on a multilevel Bspline, and checked the influence of the balance item using the L2 regularization term, which improved the registration accuracy.
In 1975, Wang [12] established the basic theoretical framework of multivariate splines on arbitrary subdivisions, proposed the smoothing cofactor conformality method, and pioneered the algebraic geometry method for studying multivariate splines. Subsequently, many types of nontensor product expressions for binary Bspline spaces and basic functions were introduced. In particular, the binary Bspline was on the 1type and 2type triangulations. In 1984, Chui and Wang [13] provided a bivariate quadratic firstorder smoothness Bspline basis with a minimum symmetric support under the uniform division and constructed an \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensor product Bspline theoretical framework. In 2001, the theory of multivariate spline was organized, while the theory of smooth cofactor coordination method and its application in multivariate spline function were introduced in detail [14]. In addition, some considerable results were achieved in various applications [15,16,17]. To the best of our knowledge, the application of the abovementioned splines and registration of medical images has not been reported yet.
The tensor product Bspline in the rectangular domain can only express the information in the horizontal and vertical directions. The direction information is crucially important in the registration of images. The nontensor product Bspline may contain more directional information, which can display more feature information. In this study, a new registration algorithm is proposed that uses nontensor product Bspline \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) on the 2type triangulation in search space to describe the global or local motion of the image in more directions [18]. The brain CT and MRI were selected for the registration test. In addition, there are basis functions that can describe the deformation of an image in four directions, which compensates for the lack of tensor product Bspline functions to a certain extent and achieves accurate results.
The remainder of this paper is organized as follows. First, the basic framework of image registration is introduced, and explicit expressions and images of nontensor product Bspline basis functions in \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) space are provided, followed by the registration experiments and a detailed analysis.
Methods
Nontensor product Bspline in the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) space
Uniform 2type triangulation
Uniform 2type triangulation is a triangulation formed by connecting two diagonal lines of each small rectangle on the basis of a rectangular division. If the rectangular division is uniformly divided, the formation of a uniform 2type triangulation is also achieved using the division.
The uniform 2type triangulation \( {\Delta}_{mn}^{(2)} \) on area D is generated by the following division lines, as shown in Fig. 2.
Basis function of the spline space \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) [14]
The dimension of the spline space \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) can be expressed as
\( \dim {S}_2^1\left({\Delta}_{mn}^{(2)}\right)=\left(m+2\right)\left(n+2\right)1 \) (2)
A local support spline function B(x, y) exists in the space \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \). Its support is the area Q in Fig. 3, where the support function is centered at the origin.
According to the theory of polynomial interpolation, the seconddegree polynomial on a triangle can be uniquely determined using the value of the polynomial on the three vertices of the triangle and midpoints of the three sides. In fact, these six points are simply a set of wellposed node groups for the seconddegree polynomial interpolation, as shown in Fig. 4.
Therefore, it is only necessary to indicate the values of the corresponding six points on the cavities to represent the spline B(x, y) of the abovementioned local support.
As shown in Fig. 3, for each i = 1, …, 28, consider the polynomials as
where other polynomials can be obtained using the principle of symmetry.
According to Eq. (4), all polynomials on the cavity are obtained as follows:
Here, the Bspline basis is \( {B}_{ij}\left(x,y\right)=B\left( mxi+\frac{1}{2}, nyj+\frac{1}{2}\right) \). Figure 5 demonstrates a 3D diagram of the local support of a bivariate quadratic spline curve, where the provided equidistant grid points are located at x_{i − 1}, …, x_{i + 2} = − 1, 0, 1, 2 and y_{i − 1}, …, y_{i + 2} = − 1, 0, 1, 2. The bivariate Bspline basis function was connected to all 28 nonzero polynomial surfaces. In addition, it is continuous and has a continuous firstorder partial derivative. The following properties were established:
The smoothness and local characteristics of the bivariate Bspline make it an ideal candidate as the basis function of the approximation or interpolation kernel.
Normalized crosscorrelation algorithm
Common measures of image similarity include peak signaltonoise ratio, structural similarity, normalized crosscorrelation (NCC), normalized mutual information, and mean square error. Because the calculation of the NCC coefficient is relatively simple, and the reasonable concaveconvex characteristic is conducive to solving the optimal parameters, the NCC method was chosen as the similarity measurement algorithm. The NCC similarity metric measures the similarity of the images to be registered by calculating the crosscorrelation value between the reference image and the moving image. The NCC of a twodimensional (2D) image can be expressed as
where A(i, j) and B(i, j) represent the gray values of the moving image and the reference image in m rows and n columns, respectively. When the NCC value between the reference and moving images is the largest, the two images are fully registered. Our chosen optimization strategy adopts the gradient descent method to solve the minimum value of the objective function. Therefore, this study selects the reciprocal of Eq. (7).
Image registration framework
The 2D arrays I_{1}(x, y) and I_{2}(x, y) of a known size respectively represent the gray value of the moving and reference images at the point (x, y). Subsequently, the registration relationship between the images can be expressed as
where f represents a 2D geometric transformation function, and g represents a onedimensional grayscale interpolation function.
The geometric transformation function f in the nonrigid registration algorithm consists of three parts, including the search space (the nontensor product Bspline model), similarity measure (the NCC algorithm), and optimization strategy (the gradient descent algorithm). The graylevel interpolation relationship g applies a bilinear interpolation algorithm. First, convert the pixel coordinates of the image into parameter grid coordinates, perform the uniform 2type triangulation on the parameter grid, and calculate the value of the nontensor product Bspline model transformation. This value is the offset of the image coordinates, each of which is affected by nine control point parameters. Provided that the pixel coordinates of a moving image I(x, y) in row a and column b are (x, y) and the parameter coordinates are (u, v), convert it into a parameter grid with m rows and n columns, respectively.
The coordinate offset is as follows:
Algorithm 1
Pseudocode implementation of nontensor product Bspline transformation
Input:Moving_Image (x, y), Control_vertex_parameter[2 * m * n]  

Output:Deformation_Image(New_x, New_y)  
1  while x < a and y < b do 
2  delta_x = a / m; 
3  delta_y = b / n; 
4  x_block = x / delta_x; 
5  y_block = y / delta_y; 
6  w = floor(x_block); 
7  s = floor(y_block); 
8  u = x_block  w; 
9  v = y_block  s; 
10  i = 0; 
11  j = 0; 
12  if (u, v) in the interval of the basis function p(u, v) then 
13  While i < 3 do 
14  While j < 3 do 
15  Tx + = B(u, v) * Control_vertex_parameter[(s + i) * n + w + j]; 
16  Ty + = B(u, v) * Control_vertex_parameter[(s + i) * n + w + j + m * n]; 
17  end while 
18  end while 
19  end if 
20  end while 
21  New_x = x + Tx; 
22  New_y = y + Ty; 
In pseudocode implementations, such as Algorithm 1, the control vertex parameter sequence is first initialized, the parameter sequence and moving image are input into the nontensor product Bspline model and the pixel coordinates of the moving image are transformed to obtain the deformation image. Subsequently, the NCC algorithm is used to calculate the similarity measure between the deformation and reference images, and then it is judged whether the similarity measure satisfies a certain threshold. If not, the gradient descent algorithm is used to update the control vertex parameters, find the optimal solution using the continuous iteration, and finally export the registration image, as shown in Fig. 6.
To perform the experiments in this study, an Intel(R) Core (TM) i78750H CPU and NVIDIA GeForce GTX 1050 GPU hardware configuration was employed. The graphics processing unit with Compute Unified Device Architecture was used to accelerate the calculation of the nontensor product Bspline algorithm.
Results and Discussion
33× 33 parameter grid
This group of experiments used MRI and CT images of a patient with a brain hematoma to test the effectiveness of the algorithm in correcting the nonrigid motion of medical images. The size of the images was 496 × 472, and the parameter grid specification was set to 33 × 33, as shown in Fig. 7. The initial similarity measure value of the MRI and CT reference and moving images was 1.01348 and 1.01908, respectively.
Tensor product quadratic uniform Bspline
In this experiment, a twovariable quadratic uniform Bspline tensor product was selected as the deformation function of the registration, in which 3 × 3 control vertices controlled the offset of each pixel. Thirteen valid iterations were performed for the registration experiment of the MRI images. The similarity measure reached 1.00338, where the running time was 5 min. Figure 8 illustrates the MRI experimental results. Eighteen valid iterations were carried out for the registration experiment of the CT images. The final similarity measure, for a running time of 4 min, was 1.00569. Figure 9 demonstrates the CT experimental results.
Tensor product cubic uniform Bspline
In this experiment, a twovariable cubic uniform Bspline tensor product was selected as the deformation function of the registration, in which 4 × 4 control vertices controlled each pixel. Sixteen valid iterations were performed for the registration experiment of the MRI images. The similarity measure reached 1.00327, where the running time was 11 min. Figure 10 presents the MRI experimental results. Eighteen valid iterations were carried out for the registration experiment of the CT images. The final similarity measure, for a running time of 10 min, was 1.00559. Figure 11 shows the CT experimental results.
Nontensor product uniform Bspline
In this experiment, a Bspline uniformly divided in the space \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) was used, where the node vector was \( 0,\frac{1}{33},\frac{2}{33},\dots, \frac{31}{33},\frac{32}{33},1 \). Here, m = 33, n = 33; thus, the dimension of \( {S}_2^1\left({\Delta}_{33\times 33}^{(2)}\right) \) was 1224, and the set of basis functions to be subtracted was as follows:
Among them, 3 × 3 control vertices controlled the offset of each pixel, where 17 valid iterations were carried out for the registration experiment of MRI images. The similarity measure reached 1.00274, where the running time was 49 min. Figure 12 presents the MRI experimental results. Twentyeight valid iterations were carried out for the registration experiment of the CT images. The final similarity measure was 1.00457, where the running time was 45 min. Figure 13 illustrates the CT experimental results.
Nontensor product nonuniform Bspline
In this experiment, the nonuniformly divided Bspline in the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) space was used as the deformation function, where the node vector was \( 0,\frac{1}{33},\frac{3}{33},\frac{4}{33},\dots, \frac{29}{33},\frac{31}{33},\frac{32}{33},1. \) Among them, the 4 × 3 control vertices controlled the pixel offset, and the subtracted set of basis functions was as follows:
The MRI registration experiment carried out 19 effective iterations and reached a similarity measure of 1.00537 with a running time of 30 min. Figure 14 presents the MRI experimental results. The CT registration experiment carried out 25 effective iterations, and reached a similarity measure value of 1.00808, where the running time was 28 min. Figure 15 demonstrates the CT experimental results.
Figures 16 and 17 present the fitting curves of the number of iterations and the registration accuracy of different methods in the experimental process, respectively. As shown in Tables 1 and 2, the uniform Bspline method in the nontensor \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) space has the highest registration accuracy.
66× 66 parameter grid
The experimental data from this group used CT images of a patient. Its size was 835 × 835, the parameter grid size was 66 × 66, and the node vector was \( 0,\frac{1}{66},\frac{2}{66},\dots, \frac{64}{66},\frac{65}{66},1 \), as shown in Fig. 18. The initial similarity measure of the CT images was 1.01562.
Tensor product cubic uniform Bspline
This experiment used a tensor product twovariable cubic Bspline as the deformation function. The CT registration experiment carried out 35 effective iterations, and reached a similarity measure value of 1.00671, where the running time was 3 h. Figure 19 presents the experimental results.
Nontensor product uniform Bspline
In this experiment, a Bspline uniformly divided in the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) space was used, and the subtracted set of basis functions was as follows:
The CT registration experiment carried out 36 effective iterations, and reached a similarity measure value of 1.0067, where the running time was 6 h. Figure 20 shows the CT experimental results.
Figure 21 illustrates the fitting curves between the number of iterations and the registration accuracy in this set of experiments. Table 3 proves that as the image size and specification of the parameter grid increase, the time cost of the two algorithms increases. Consequently, the experimental results proved that the nontensor product algorithm obtained a higher accuracy.
Conclusions
In this study, a nonrigid registration algorithm based on the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensortype Bsplines was introduced. Applying the proposed algorithm to the search space algorithm could satisfactorily simulate the nonrigid deformation of medical images and describe the dynamic motion of medical images. Indeed, the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensor product Bspline algorithm is a function defined in four directions. Compared with the tensor product spline function, it can describe the deformation of the image in more directions. Simultaneously, the \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensor product Bspline algorithm is highly flexible in the processing of boundary triangles. By comparing the performance of different methods, the nontensor product uniform Bspline algorithm yielded the highest accuracy. The errors in this study might have been caused by the slicing technology or tissue effects, which will be attempted to be reduced in the future studies. Although the accuracy could be improved, the required time was increased owing to the increased complexity of the algorithm. This problem can be addressed by changing the optimization algorithm of the search space and increasing the computing power of the device. However, further study is required to investigate the model’s parametric mesh subdivision and convergence.
Availability of data and materials
The datasets used and/or analyzed during the current study are not publicly available due to personal privacy, but are available from the corresponding author upon reasonable request.
Abbreviations
 CT:

Computed tomography
 MRI:

Magnetic resonance imaging
 3D:

Threedimensional
 PET:

Positron emission computed tomography
 US:

Ultrasound
 2D:

Twodimensional
 NCC:

Normalized crosscorrelation
References
Crum WR, Hartkens T, Hill DLG (2004) Nonrigid image registration: theory and practice. Br J Radiol 77(S2):S140–S153. https://doi.org/10.1259/bjr/25329214
Pan QL, Zhu W, Zhang XL, Chang JC, Cui JZ (2020) Research on a bifurcation location algorithm of a drainage tube based on 3D medical images. Vis Comput Ind Biomed Art 3(1):2. https://doi.org/10.1186/s4249201900390
Zhu YF, Bao MY, Jiang MM, Chang JC, Cui JZ (2021) Visual analysis of flow and diffusion of hemolytic agents and hematomas. Vis Comput Ind Biomed Art 4(1):3. https://doi.org/10.1186/s42492020000684
Brown LG (1992) A survey of image registration techniques. ACM Comput Surv 24(4):325–376. https://doi.org/10.1145/146370.146374
Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187–198. https://doi.org/10.1109/42.563664
Rueckert D, Sonoda LI, Hayes C, Hill DLG, Leach MO, Hawkes DJ (1999) Nonrigid registration using freeform deformations: application to breast MR images. IEEE Trans Med Imaging 18(8):712–721. https://doi.org/10.1109/42.796284
Lee S, Wolberg G, Shin SY (1997) Scattered data interpolation with multilevel Bsplines. IEEE Trans Vis Comput Graph 3(3):228–244. https://doi.org/10.1109/2945.620490
Wei DY (2016) Image superresolution reconstruction using the highorder derivative interpolation associated with fractional filter functions. IET Signal Process 10(9):1052–1061. https://doi.org/10.1049/ietspr.2015.0444
Schnabel JA, Rueckert D, Quist M, Blackall JM, CastellanoSmith AD, Hartkens T et al (2001) A generic framework for nonrigid registration based on nonuniform multilevel freeform deformations. In: Niessen WJ, Viergever MA (eds) International conference on medical image computing and computerassisted interventionMICCAI 2001. MICCAI 2001. Lecture notes in computer science, vol 2208. Springer, Heidelberg, pp 573–581. https://doi.org/10.1007/3540454683_69
Oliveira FPM, Tavares JMRS (2013) Enhanced spatiotemporal alignment of plantar pressure image sequences using Bsplines. Med Biol Eng Comput 51(3):267–276. https://doi.org/10.1007/s1151701209883
Ji HZ, Li YS, Dong EQ, Xue P, Xiong WS, Sun WY et al (2018) A nonrigid image registration method based on multilevel Bspline and L2regularization. Signal Image Video Process 12(6):1217–1225. https://doi.org/10.1007/s1176001812740
Wang RH (1975) The structural characterization and interpolation for multivariate splines. Acta Math Sinica 18(2):91–106
Chui CK, Wang RH (1984) On a bivariate Bspline basis. Sci China Ser A 27(11):1129–1142
Wang RH (2001) Multivariate spline functions and their applications. Science Press, Beijing https://doi.org/10.1007/9789401723787
Wang RH, Chang JC (2007) A kind of bivariate spline space over rectangular partition and pure bending of thin plate. Appl Math Mech 28(7):963–971. https://doi.org/10.1007/s104830070713y
Wang RH, Chang JC (2007) Mechanical background of bivariate spline space \( {S}_3^{1,0}\left({\Delta}_c^{(1)}\right) \). J Informat Comput Sci 4(1):299–307
Wang RH, Chang JC (2011) Bivariate splines and golden section based on theory of elasticity. J Math Res Exposition 31(1):1–11
Zhang XX, Gilliam C, Blu T (2020) Allpass parametric image registration. IEEE Trans Image Process 29:5625–5640. https://doi.org/10.1109/TIP.2020.2984897
Acknowledgments
The authors thank Tangshan Gongren Hospital for their support with the original data.
Funding
This research was funded by National Natural Science Foundation of China, No. 61702184; Ministry of Education Production University Cooperation Education Project, No. 201802305012; and Tangshan Innovation Team Project, No. 18130209 B.
Author information
Authors and Affiliations
Contributions
All authors contributed to the data curation; QZ and CL contributed to the visualization; CL summarized and reviewed the literature and pointed out the prospects and deficiencies of the research; JC contributed to the supervision, the data curation and the funding acquisition. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zheng, Q., Liu, C. & Chang, J. Nonrigid registration of medical images based on \( {S}_2^1\left({\Delta}_{mn}^{(2)}\right) \) nontensor product Bspline. Vis. Comput. Ind. Biomed. Art 5, 5 (2022). https://doi.org/10.1186/s42492022001018
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s42492022001018
Keywords
 Nonrigid registration
 Diagonal direction
 Nontensor product type Bspline
 Boundary triangle domain