During the past decade, there was an exponential growth in the number of webpages available. Automatic webpage categorization systems can help to manage these immense amounts of content, making search tasks and recommendation easier. However, most webpages have a significant proportion of visual content that conventional, text-based web mining systems can not handle. In this paper, we present a novel hybrid CBR framework designed to perform imagebased webpage categorization. Our system incorporates stateof-the-art deep learning techniques which help attain high accuracy rates. In addition, the system was designed with the goal of minimizing computational costs.