Television white spaces (TVWSs) refer to the unused part of the spectrum under the very high frequency (VHF) and ultra-high frequency (UHF) bands. TVWS are frequencies under licenced primary users (PUs) that are not being used and are available for secondary users (SUs). There are several ways of implementing TVWS in communications, one of which is the use of TVWS database (TVWSDB). The primary purpose of TVWSDB is to protect PUs from interference with SUs. There are several geolocation databases available for this purpose. However, it is unclear if those databases have the prediction feature that gives TVWSDB the capability of decreasing the number of inquiries from SUs. With this in mind, the authors present a reinforcement learning-based TVWSDB. Reinforcement learning (RL) is a machine learning technique that focuses on what has been done based on mapping situations to actions to obtain the highest reward. The learning process was conducted by trying out the actions to gain the reward instead of being told what to do. The actions may directly affect the rewards and future rewards. Based on the results, this algorithm effectively searched the most optimal channel for the SUs in query with the minimum search duration. This paper presents the advantage of using a machine learning approach in TVWSDB with an accurate and faster-searching capability for the available TVWS channels intended for SUs.