Share E-Book
Scan to open this page

Scan with your phone to open this page

Author: Mykel J. Kochenderfer, Tim A. Wheeler

Rating 4.0 AI initial rating

The extensively updated second edition of this popular textbook provides a comprehensive introduction to optimization with a focus on practical algorithms. Mykel Kochenderfer and Tim Wheeler approach optimization from an engineering perspective, where the objective is to design a system that optimizes a set of metrics subject to constraints. Readers will learn about computational approaches for a range of challenges, including searching high-dimensional spaces, handling problems where there are multiple competing objectives, and accommodating uncertainty in the metrics. Figures, examples, and exercises convey the intuition behind the mathematical approaches. Suitable for advanced undergraduates and graduate students in mathematics, statistics, computer science, engineering, and operations research and as a reference for professionals, the text provides concrete implementations in the Julia programming language.

AI Reading Assistant

Whole-book reading guide from stratified index samples; jump to passages in the text

AI guide
【One-Line Pitch】 A practical, engineering-first tour of optimization algorithms—from bracketing and line search to constraints, duality, and population methods—that turns mathematical intuition into runnable Julia code. Best for advanced undergraduates, graduate students, and practitioners who want to actually implement optimizers rather than only prove theorems. 【Book Arc】 - **Opening (~0%–10%)**: Frames optimization as a design problem—choosing system parameters to optimize metrics under constraints—and introduces the algorithmic mindset plus computational tools like automatic differentiation (forward and reverse accumulation) that later methods rely on. - **Early (~10%–32%)**: Builds the univariate and local-search foundation: bracketing a minimum, line search (including approximate/Armijo backtracking and Wolfe conditions), first-order descent methods with accelerated variants such as Adam, and second-order methods (Newton, secant) with convergence comparisons. - **Early–Middle (~32%–48%)**: Moves into derivative-free and stochastic search—pattern search, Nelder–Mead simplex, DIRECT-style partition search, simultaneous-perturbation stochastic gradients, simulated annealing, information-geometric optimization, and evolutionary schemes like differential evolution. - **Middle (~48%–60%)**: Introduces constraints and how to handle them: constraint types, transformations that remove constraints, projected iterative methods, Lagrange multipliers, slack variables, penalty methods, the method of multipliers, and interior-point methods. - **Late (~60%–75%)**: Develops duality—dual problems, primal-dual methods, dual ascent, ADMM and its applications, and distributed optimization—then linear programming with the simplex algorithm and dual certificates. - **Ending (beyond ~75%)**: Excerpts do not cover the remaining chapters in detail; the table of contents suggests further treatment of multi-objective and uncertainty-aware optimization, but this guide cannot confirm their content. 【Key Takeaways】 - **Optimization is framed as engineering design, not pure math** (Opening): the goal is to search a design space to optimize metrics like efficiency, safety, or accuracy under constraints such as cost and weight—this framing motivates every algorithm that follows. - **Automatic differentiation is the computational backbone** (Opening): forward accumulation needs one pass per input dimension, while reverse accumulation computes a full gradient in a forward-plus-backward pass and underpins backpropagation—key for anyone implementing gradient-based methods. - **Line search is where descent methods live or die** (Early): exact line search is often wasteful; approximate search using the Wolfe/Armijo sufficient-decrease condition and backtracking gives a practical step-size rule. - **Method choice depends on available derivatives and conditioning** (Early): Newton converges in one step on quadratics but needs second derivatives; the secant method trades speed for only requiring first derivatives; gradient descent can diverge on badly scaled problems where conjugate gradient and Newton succeed. - **Derivative-free methods cover non-smooth and black-box objectives** (Early–Middle): pattern search, Nelder–Mead, and DIRECT-style partitioning search the space without gradients, at the cost of scalability and tuning of step sizes and tolerances. - **Stochasticity is a tool, not just noise** (Middle): simultaneous-perturbation estimates approximate gradients from function evaluations alone, and simulated annealing uses a temperature schedule to escape local minima—useful when evaluations are expensive or noisy. - **Constraints can be removed or handled analytically** (Middle): transformations, projections, Lagrange multipliers, penalties, and interior-point methods each convert constrained problems into tractable ones, with different trade-offs in feasibility and convergence. - **Duality and decomposition enable scale** (Late): dual formulations, ADMM, and distributed methods let large or separable problems be solved in pieces, while linear programming gets the specialized simplex treatment. 【Reading Tips】 - Deep-read Chapters 2–6 (automatic differentiation through second-order methods) if you implement optimizers; this is the conceptual spine and the Julia code is most instructive there. - Skim the derivative-free and stochastic chapters (7–9) on a first pass, then return when you face black-box or noisy objectives; note the hyperparameter defaults and when they fail. - Treat the constraint and duality chapters (10–11) as a pair—understand transformations and penalties before ADMM, or the primal-dual machinery will feel unmotivated. - Work the exercises with solutions where provided; the Newton-vs-gradient-descent-vs-conjugate-gradient comparison is a compact lesson in conditioning. - Keep the Julia implementations open alongside the math; the algorithms are short enough to read line by line. 【Coverage Limits】 This guide is based on stratified excerpts covering roughly the first three-quarters of the book; later chapters on multi-objective and uncertainty-aware optimization are not represented, so claims about them are deliberately omitted.
Page 12
lgorithm 255 12.3 Dual Certificates 266 12.4 Summary 269 12.5 Exercises 269 © 2026 Massachusetts Institute of Technology, shared under a Creative Commons CC-...
View in text
Excerpt 2
urns a tuple containing while true the new interval [a, b]. c, yc = b + s, f(b + s) if yc > yb return a < c ? (a, c) : (c, a) end a, ya, b, yb = b, yb, c, yc...
View in text
Excerpt 3
uses the same initial search direction as gradient descent and converges to the minimum in the second step because the optimization objective is quadratic. E...
View in text
Excerpt 4
For a multivariate normal N (µ, Σ), the log-density is 1( ) log p(x | µ, Σ) = − n log 2π + log|Σ|+ (x− µ)>Σ−1(x− µ) (8.20) 2 The gradients with respect to th...
View in text
Excerpt 5
Dual Ascent Recall that the dual problem from equation (11.4) is always a concave optimization problem in µ and λ: maximize minimizeL(x,µ,λ) (11.21) µ≥0,λ x...
View in text
Excerpt 6
, b − = 2 1 0 0 1 0 c 7 = 0 , −1 1 0 0 0 1 1 0 0 0 and the initial vertex defined by B = {1, 2, 3, 4}. Verify that B defines a feasible vertex, and then exec...
View in text
Excerpt 7
er the in chapter 9. Pareto frontier. 15.4.1 Subpopulations Population methods can divide their attention over several potentially competing objectives. The...
View in text
Excerpt 8
estimating the generalization error is the holdout method, which partitions the available data into a test set Dh with h samples and a training set Dt consis...
View in text
Tags
AI categories
MathematicsProgramming Language
ISBN: 0262058162
Publisher: MIT Press
Publish Year: 2027
Language: English
File Format: PDF
File Size: 13.9 MB
Text Preview (First 20 pages)
Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

Generating text preview…