The proposed design of neural network in this article is based on new accurate approach for training by unconstrained optimization, especially update quasi-Newton methods are perhaps the most popular general-purpose algorithms. A limited memory BFGS algorithm is presented for solving large-scale symmetric nonlinear equations, where a line search technique without derivative information is used. On each iteration, the updated approximations of Hessian matrix satisfy the quasi-Newton form, which traditionally served as the basis for quasi-Newton methods. On the basis of the quadratic model used in this article, we add a new update of quasi-Newton form. One innovative features of this form's is its ability to estimate the energy function's or performance function with high order precision with second-order curvature while employ the given function value data and gradient. The global convergence of the proposed algorithm is established under some suitable conditions. Under some hypothesis the approach is established to be globally convergent. The updated approaches can be numerical and more efficient than the existing comparable traditional methods, as illustrated by numerical trials. Numerical results show that the given method is competitive to those of the normal BFGS methods. We show that solving a partial differential equation can be formulated as a multi-objective optimization problem, and use this formulation to propose several modifications to existing methods. Also the proposed algorithm is used to approximate the optimal scaling parameter, which can be used to eliminate the need to optimize this parameter. Our proposed update is tested on a variety of partial differential equations and compared to existing methods. These partial differential equations include the fourth order three dimensions nonlinear equation, which we solve in up to four dimensions, the convection-diffusion equation, all of which show that our proposed update lead to enhanced accuracy.
