An efficient Dai-Yuan CG method based on structured secant conditions for NLS problems and its application
DOI:
https://doi.org/10.64497/jssci.5Keywords:
Keywords: Conjugate gradient; Least-squares; quasi-Newton; Inverse kinematicsAbstract
Nonlinear least squares (NLS) problems are commonly encountered in a wide range of scientific and engineering fields, often demanding robust and efficient optimization methods for their solution. Traditional approaches for solving NLS problems frequently encounter difficulties related to high computational costs and memory usage, particularly in large-scale scenarios. This study introduces a novel structured Dai-Yuan two-term conjugate gradient (CG) method to address the minimization of NLS problems. By leveraging a Taylor series expansion of the objective function's Hessian, a structured vector approximation, representing a matrix-vector interaction, is constructed. This formulation adheres to a quasi-Newton condition. The method utilizes this structured approximation to enrich the conventional search direction with additional curvature information from the Hessian. The resulting search direction satisfies the required descent condition. Furthermore, numerical experiments on a range of benchmark problems demonstrate that the proposed method offers superior performance, outperforming several existing techniques.
Downloads
References
[1] M. M. Yahaya, P. Kumam, A. M. Awwal, and S. Aji, "A structured quasi-Newton algorithm with nonmonotone search strategy for structured NLS problems and its application in robotic motion control," J. Comput. Appl. Math., vol. 395, pp. 113553, 2021. DOI: https://doi.org/10.1016/j.cam.2021.113582
[2] J. Iwaniec and T. Uhl, The application of the nonlinear least squares frequency domain method to estimation of the modal model parameters. Cracow, Poland: AGH Univ. of Sci. and Technol., Dept. of Robotics and Machine Dynamics, n.d.
[3] N. Salihu, P. Kumam, I. M. Sulaiman, and T. Seangwattana, “An efficient spectral minimization of the Dai–Yuan method with application to image reconstruction,” AIMS Mathematics, vol. 8, pp. 30940–30962, 2023. DOI: https://doi.org/10.3934/math.20231583
[4] F. Saâdaoui, H. Saadaoui, and H. Rabbouch, “Hybrid feedforward ANN with NLS-based regression curve fitting for US air traffic forecasting,” Neural Computing and Applications, vol. 32, no. 14, pp. 10073–10085, 2020. DOI: https://doi.org/10.1007/s00521-019-04539-5
[5] R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, M. M. Yahaya, and A. Al-Yaari, "An improved accelerated 3-term conjugate gradient algorithm with second-order Hessian approximation for nonlinear least-squares optimization," J. Math. Comput. Sci., vol. 36, pp. 263–274, 2025. https://doi.org/10.22436/jmcs.036.03.02 DOI: https://doi.org/10.22436/jmcs.036.03.02
[6] W. W. Hager and H. Zhang, "A survey of nonlinear conjugate gradient methods," Pac. J. Optim., vol. 2, pp. 35–58, 2006.
[7] M. Al-Baali and R. Fletcher, "Variational methods for non-linear least-squares," J. Oper. Res. Soc., vol. 36, pp. 405–421, 1985. DOI: https://doi.org/10.1057/jors.1985.68
[8] J. E. Dennis, Jr., H. J. Martínez, and R. A. Tapia, "Convergence theory for the structured BFGS secant method with an application to nonlinear least squares," J. Optim. Theory Appl., vol. 61, pp. 161–178, 1989. DOI: https://doi.org/10.1007/BF00962795
[9] F. Wang, D.-H. Li, and L. Qi, "Global convergence of Gauss-Newton-MBFGS method for solving the nonlinear least squares problem," Adv. Model. Optim., vol. 12, pp. 1–19, 2010.
[10] M. Kobayashi, Y. Narushima, and H. Yabe, "Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems," J. Comput. Appl. Math., vol. 234, pp. 375–397, 2010. DOI: https://doi.org/10.1016/j.cam.2009.12.031
[11] H. Mohammad and S. A. Santos, "A structured diagonal Hessian approximation method with evaluation complexity analysis for nonlinear least squares," Comput. Appl. Math., vol. 37, pp. 6619–6653, 2018. DOI: https://doi.org/10.1007/s40314-018-0696-1
[12] R. Dehghani and N. Mahdavi-Amiri, "Scaled nonlinear conjugate gradient methods for nonlinear least squares problems," Numer. Algorithms, vol. 82, pp. 1–20, 2019. DOI: https://doi.org/10.1007/s11075-018-0591-2
[13] R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, S. A. A. Karim, and M. M. Yahaya, "A modified structured spectral HS method for nonlinear least squares problems and applications in robot arm control," Mathematics, vol. 11, pp. 1–17, 2023. DOI: https://doi.org/10.3390/math11143215
[14] M. M. Yahaya, P. Kumam, P. Chaipunya, A. M. Awwal, S. Aji, and S. Salisu, ``Structured adaptive spectral-based algorithms for nonlinear least squares problems with robotic arm modelling applications,'' Comput. Appl. Math., vol. 42, no. 320, 2023. https://doi.org/10.1007/s40314-023-02452-1. DOI: https://doi.org/10.1007/s40314-023-02452-1
[15] J. Barzilai and J. M. Borwein, "Two-point step size gradient methods," IMA J. Numer. Anal., vol. 8, pp. 141–148, 1988. DOI: https://doi.org/10.1093/imanum/8.1.141
[16] K. Sugiki, Y. Narushima, and H. Yabe, "Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization," J. Optim. Theory Appl., vol. 153, pp. 733–757, 2012. DOI: https://doi.org/10.1007/s10957-011-9960-x
[17] Y.-H. Dai and Y. Yuan, "A nonlinear conjugate gradient method with a strong global convergence property," SIAM J. Optim., vol. 10, no. 1, pp. 177–182, 1999. DOI: https://doi.org/10.1137/S1052623497318992
[18] [38] H. Zhang and W. W. Hager, "A nonmonotone line search technique for unconstrained optimization," *SIAM Journal on Optimization*, vol. 14, no. 4, pp. 1043–1056, 2004. DOI: https://doi.org/10.1137/S1052623403428208
[19] W. L. Cruz, J. Martínez, and M. Raydan, "Spectral residual method without gradient information for solving large-scale nonlinear systems of equations," *Mathematics of Computation*, vol. 75, no. 255, pp. 1429–1448, 2006. DOI: https://doi.org/10.1090/S0025-5718-06-01840-0
[20] M. Jamil and X.-S. Yang, "A literature survey of benchmark functions for global optimisation problems," *International Journal of Mathematical Modelling and Numerical Optimisation*, vol. 4, no. 2, pp. 150–194, 2013. DOI: https://doi.org/10.1504/IJMMNO.2013.055204
[21] L. Lukšan and J. Vlček, *Test problems for unconstrained optimization*, Tech. Rep. 897, Institute of Computer Science, Academy of Sciences of the Czech Republic, 2003.
[22] J. J. Moré, B. S. Garbow, and K. E. Hillstrom, "Testing unconstrained optimization software," *ACM Trans. Math. Softw.*, vol. 7, no. 1, pp. 17–41, 1981. DOI: https://doi.org/10.1145/355934.355936
[23] R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, M. Muhammad Yahaya and S. Ariffin Abdul Karim, "New CG-Based Algorithms with Second-Order Curvature Information for NLS Problems and a 4DOF Arm Robot Model," in IEEE Access, vol. 12, pp. 61086-61103, 2024, doi: 10.1109/ACCESS.2024.3393555. DOI: https://doi.org/10.1109/ACCESS.2024.3393555
Downloads
Published
Versions
- 2025-06-02 (2)
- 2025-07-01 (1)
How to Cite
Issue
Section
License
Copyright (c) 2025 RABIU Bashir Yunus, Dr. Nooraini Zainuddin, Dr. Kamilu kamfa, Dr. Bashir Danladi Garba, Dr. Muhammad Auwal Lawan, Dr. Sulaiman I. Mohammed
This work is licensed under a Creative Commons Attribution 4.0 International License.
- Abstract 467
- PDF 133
