Classification of Yelp Photos

    Image recognition has seen breakthroughs in recent years, especially because of rapid growth in deep learning. The applications of image recognition is limitless. The task of the project is to classify photos of businesses from Yelp Dataset Challenge to appropriate categories, such as “Chinese”, “Lifestyle” and “Nightlife”. A successful classification model could easily be expanded to add new features such as auto-tagging of photos or error detection for tags.

    Yelp Dataset Challenge provides access to 200,000 photos and 77,000 business information. The dataset contains a total of 897 possible categories; to simplify our dataset, we preprocessed the data to include businesses and photos that belong to top 20 most frequently appearing categories. For feature detection from photos, we created two different datasets: raw pixels and SIFT (Scale-Invariant Feature Transform) features.

     For training, various machine learning algorithms were explored. Using the Scikit-learn library in Python, we trained the model using classification techniques such as Logistic Regression, Naive Bayes, Support Vector Machine and Random Forest. Also, using Google’s Tensorflow library to train the model, we implemented a deep Convolutional Neural Network, which has been proven to be effective in image recognition.  

    The best F1 score that we achieved for the 20-class classification problem is 0.14. The accuracy was 0.17. Combination of 5-fold cross validation and grid search was used to fine-tune the parameters of the classifiers. Random Forest Classifier using SIFT features was the best-performing classifier for the task.