Instructor

Dr. Aranya Chakrabortty

Dr. Aranya Chakrabortty

Electrical and Computer Engineering

Phone: 919-513-3529
Email: aranya.chakrabortty@ncsu.edu
Instructor Website
Research Website

ECE 792-602 Adaptive Control and Reinforcement Learning

3 Credit Hours

Advanced topics in adaptive control, parameter estimation, reinforcement learning and Q-learning based control. Current research and recent developments in the field.

ECE 792 Adaptive Control and Reinforcement Learning Course Flier

Prerequisite

ECE 516 – System Control Engineering

First-level knowledge of nonlinear systems is encouraged, but not necessary.

Course Objectives

Upon completion of this course, students will be able to:

  1. Combine ideas of adaptation and control theory for linear and nonlinear control systems
  2. Formulate adaptive control problems using both direct and indirect methods
  3. Solve indirect adaptive control using least-squares based parameter estimation
  4. Solve direct adaptive control using Lyapunov stability theory
  5. Solve optimal adaptive control using reinforcement learning, Q-learning, actor-critic policy
  6. Develop simulation skills for online and offline learning
  7. Apply adaptive control to practical systems such as power systems, mechatronics, process control, aircraft control, biomedical systems, and manufacturing.

Course Requirements

Grading Policy:
There will be 7 or 8 homework assignments throughout the semester, a midterm, a course project, and a final exam. The weight for each is as follows:

Homeworks: 20%
Midterm: 30%
Project: 10%
Final Exam: 40%

Homework assignments will be uploaded to the ‘Assignment’ section on the Moodle course website. All homework will be due in a week from the day the assignment is handed out. There will be 20% penalty for each session late. Submission will not be accepted if more than two sessions late.

Distance learning students are requested to scan their completed homeworks and email it to the instructor at achakra2@ncsu.edu by the submission deadlines stated on the homework.

Solutions:
Solutions to homework and tests will be uploaded in Moodle.

Software needed: MatLab and Simulink
Other related simulation packages will be provided by the instructor.

Textbook

Gang Tao, Adaptive Control Design and Analysis, Wiley, 2003.

Notes from Hassan Khalil’s book on nonlinear systems (item number 4 in the list below) will also be used from time to time. The instructor will upload these book notes on Moodle.

Other reference books of interest (On reserve in library):

  1. P. Ioannou and B. Fidan, Adaptive Control Tutorial, SIAM 2006.
  2. K. Narendra and A. Annaswamy, Stable Adaptive Systems, Prentice Hall, 1995.
  3. Astrom and Wittenmark, Adaptive Control, Addison-Wesley, 1995.
  4. H. Khalil, Nonlinear Systems, Prentice Hall, 2003.
  5. B.D.O. Anderson and J.B. Moore, Optimal Filtering, Prentice-Hall, 1979.
  6. F.L. Lewis and V.L. Syrmos, Optimal Control, 2nd Ed.,Wiley Interscience, 1995.
  7. G.E. Dullerud and F. G. Paganini, A course in robust control theory, Springer, 2000.