While reinforcement learning (RL) is gaining popularity in gameplay, creating a generalized RL model is still challenging. This study presents BaziGooshi, a generalized RL solution for games, focusing on two different types of games: 1) a puzzle game Candy Crush Friends Saga and 2) a platform game Sonic the Hedgehog Genesis. BaziGooshi rewards RL agents for mastering a set of intrinsic basic skills as well as achieving the game objectives. The solution includes a hybrid model that takes advantage of a combination of several agents pretrained using intrinsic or extrinsic rewards to determine the actions. We propose an RL-based method for assigning weights to the pretrained agents. Through experiments, we show that the RL-based approach improves generalization to unseen levels, and BaziGooshi surpasses the performance of most of the defined baselines in both games. Also, we perform additional experiments to investigate further the impacts of using intrinsic rewards and the effects of using different combinations in the proposed hybrid models.