This work studies a beam tracking problem in a reconfigurable intelligent surface (RIS)-assisted communication system, in which a single antenna access point (AP) tracks a single-antenna mobile user equipment (UE) through actively reconfiguring the RIS. To maintain beam alignment over time, the mobile UE periodically sends a sequence of pilots to the AP in the uplink, and the AP updates the RIS reflection coefficients for both the subsequent downlink data transmission and uplink pilot reception stages in a sequential fashion. This is an active sensing problem which is analytically intractable. This work proposes a deep learning framework to solve this problem. We use a neural network architecture based on long short-term memory (LSTM) in which the LSTM cell automatically summarizes the time-varying channel information based on periodically received pilots into a state vector, and the state vector is mapped to the RIS reflection coefficients for subsequent downlink data transmission and uplink pilot reception using two additional deep neural networks (DNNs). We then consider an RIS-assisted multiuser multiple-input single-output (MU-MISO) system where K single-antenna UEs are served by an AP with M antennas. To jointly and adaptively design the RIS reflection coefficients for pilot/data transmissions and the AP beamforming strategies for data transmissions, we propose an end-to-end neural network that contains two main building blocks: a graph neural network (GNN) for equalizing the SINR performance of all UEs, and a LSTM for summarizing the temporal channel correlations. Simulation results show that this proposed active sensing approach is able to maintain beam alignment much more efficiently than traditional data-driven methods based only on channel statistics.