Stanford CS234: Reinforcement Learning | Winter 2019 | Lecture 9 - Policy Gr...

Professor Emma Brunskill

Assistant Professor, Computer Science

Stanford AI for Human Impact Lab

Stanford Artificial Intelligence Lab

Statistical Machine Learning Group

See More

SVD Method of Snapshots

This video describes how to compute the singular value decomposition (SVD) using the method of snapshots, by Sirovich 1987.

See More

Particle Filter Explained without Equations

This video provides a quick graphical introduction to the particle filter. It does a good job building some intuition behind the filter without ever touching on any mathematics. It's worth a...

See More

RL Course by David Silver - Lecture 5: Model Free Control

Dives into On Policy Monte-Carlo Control and Temporal Difference Learning, as well as Off-Policy Learning.

See More

Control Bootcamp: Loop shaping

This video explores shaping the loop transfer function to have desirable sensitivity and complementary sensitivity.

See More

Relative Gain Array RGA Analysis

I cover how you can find the relative gain array from the steady state gain array, and interpret the results to determine which input to pair with which outp...

See More

Constraint Enforcement for Improved Safety | Learning-Based Control

Learn about the constraints of your system. Then see a how to enforce those constraints so the system does not violate them. Constraint enforcement is important for safety-critical...

See More

Sparse Identification of Nonlinear Dynamics for Model Predictive Control

This lecture shows how to use sparse identification of nonlinear dynamics with control (SINDYc) with model predictive control to control nonlinear systems purely from data.

See More

State Space in Process Control

An overview on how we can derive a state space model from a given set of state variables and inputs, as well as an intro to deviation variables. This is part...

See More

Data Preprocessing and the Short-Time Fourier Transform | Deep Learning for ...

Data in its raw form might not be ideal for training a network. There are some changes we can make to the data that are often desired or sometimes necessary in order to make training faster...

See More

Controllability [Control Bootcamp]

This lecture explores when a linear system is controllable. We begin with the simple test in terms of the rank of the controllability matrix on a few intuitive examples.

See More

Data-Driven Control: BPOD and Output Projection

In this lecture, we introduce the output projection for balancing proper orthogonal decomposition (BPOD), to reduce the number of adjoint simulations required when the number of measurements...

See More

Numerically Linearizing a Dynamic System

In this video we show how to linearize a dynamic system using numerical techniques. In other words, the linearization process does not require an analytical description of the system. This...

See More

Sketching Root Locus Part 2

This is the second part of how to sketch a root locus by hand. However instead of following the normal rules for sketching a locus that you'd see in a book, I decided to explain the rules...

See More

Peter Ponders PID. Second Order Plus Dead Time , SOPDT, Temperature Control,...

In this video I derive the equations for the controller gains and a low pass filter for a SOPDT system with a very long dead time To make the simulation mo...

See More

Designing a PID Controller Using the Ziegler-Nichols Method

In this video we discuss how to use the Ziegler-Nichols method to choose PID controller gains. In addition to discussing the method and providing a Matlab i...

See More

Drone Simulation and Control, Part 2: How Do You Get a Drone to Hover?

In the last video, we showed we can manipulate the four motors of a quadcopter to maneuver it in 3D space by getting it to roll, pitch, yaw, and change its thrust. We also covered the four...

See More

Derivation and Solution of Laplace’s Equation

In this video we show how the heat equation can be simplified to obtain Laplace’s equation. We investigate how to solve Laplace’s equation using separation ...

See More

Peter Ponders PID - Why PID with 2nd Derivative Gain?

If you have ever tuned a hydraulic system and wondered why PID control didn't work better than PI control the answer is here. Since the 1980s people have kn...

See More

Solving the 2D Wave Equation

In this video, we solve the 2D wave equation. We utilize two successive separation of variables to solve this partial differential equation. Topics discuss...

See More

Fourier Analysis: Overview

This video presents an overview of the Fourier Transform, which is one of the most important transformations in all of mathematical physics and engineering. This series will introduce the...

See More

Frequency domain – tutorial 1: concept of frequency (with Chinese subtitle)

In this video, the following materials are covered:1) intuitive explanation on the frequency concept 2) what is the relation between time and frequency domai...

See More

Koopman Spectral Analysis (Multiscale systems)

In this video, we discuss recent applications of data-driven Koopman theory to multi-scale systems.

See More

Lecture 4: Electromechanical system Transfer functions and Analogous circuit...

The transfer function of a dc motor driving a mechanical load [an electromechanical system] has been derived following which a quarter car model is discussed... See More

Frequency domain – tutorial 5: Fourier transform

In this video, we learn about Fourier transform which enables us to travel from time to frequency domain when a signal is not periodic. The learning objectiv...

See More