Adaptive Markov Control Processes

1 Controlled Markov Processes. - 1. 1 Introduction. - 1. 2 Stochastic Control Problems. - 1. 3 Examples. - 1. 4 Further Comments. - 2 Discounted Reward Criterion. - 2. 1 Introduction. - 2. 2 Optimality Conditions. - 2. 3 Asymptotic Discount Optimality. - 2. 4 Approximation of MCM's. - 2. 5 Adaptive Control Models. - 2. 6 Nonparametric Adaptive Control. - 2. 7 Comments and References. - 3 Average Reward Criterion. - 3. 1 Introduction. - 3. 2 The Optimality Equation. - 3. 3 Ergodicity Conditions. - 3. 4 Value Iteration. - 3. 5 Approximating Models. - 3. 6 Nonstationary Value Iteration. - 3. 7 Adaptive Control Models. - 3. 8 Comments and References. - 4 Partially Observable Control Models. - 4. 1 Introduction. - 4. 2 PO-CM: Case of Known Parameters. - 4. 3 Transformation into a CO Control Problem. - 4. 4 Optimal I-Policies. - 4. 5 PO-CM's with Unknown Parameters. - 4. 6 Comments and References. - 5 Parameter Estimation in MCM's. - 5. 1 Introduction. - 5. 2 Contrast Functions. - 5. 3 Minimum Contrast Estimators. - 5. 4 Comments and References. - 6 Discretization Procedures. - 6. 1 Introduction. - 6. 2 Preliminaries. - 6. 3 The Non-Adaptive Case. - 6. 4 Adaptive Control Problems. - 6. 5 Proofs. - 6. 6 Comments and References. - Appendix A. Contraction Operators. - Appendix B. Probability Measures. - Total Variation Norm. - Weak Convergence. - Appendix C. Stochastic Kernels. - Appendix D. Multifunctions and Measurable Selectors. - The Hausdorff Metric. - Multifunctions. - References. - Author Index. Language: English