Recognize Trading Cards With AI: Industry Project for ALT

The sports trading card industry refers to the market for trading cards featuring professional athletes and teams. These cards can be collected, traded, and sold by fans, and they can also be used to play various games and simulations.The popularity of sports trading cards saw a resurgence following the COVID-19 pandemic, as many people turned to collecting as a hobby while stuck at home. With sports events and games being cancelled or played without live audiences, fans turned to collecting cards as a way to stay connected to their favorite teams and athletes. Additionally, the rise of e-commerce and social media made it easier for people to buy and sell cards online, further fueling the growth of the industry.

Please visit the Project GitHub repository for the full project.

Introduction

This project is a collaborative effort among students from Data Science, UX Design, and Web Development, working to develop digital solutions for one of BrainStation’s industry partners, ALT (a leading sports trading card marketplace).

Problem space: How can we help ALT’s users navigate the world of trading goods and make more informed trading and investment decisions?

Proposed Solution: integrate chatrooms into the existing web platform in order to foster communication and interactions between collectors. This will serve as a social platform where enthusiasts can connect, showcase their collections, and expand their knowledge of trading cards, ultimately increasing engagement with ALT.

Objective

Implement a deep learning-based image classifier to accurately identify the sport depicted on sport trading cards and facilitate assigning users to appropriate chatrooms based on the cards in the user’s vault.

About ALT

ALT is a sports trading card marketplace that allows users to buy, sell, and trade cards from various sports like football, basketball, baseball and hockey. It also offers a community for enthusiasts to connect and access historical card pricing and trends. The platform aims to provide a safe, secure and transparent environment for sports trading card collectors worldwide.

Image Data

Collection

For this project, Sports card images were obtained via ALT, using the Download All Images chrome extension, a bulk image downloader that enables downloading the images from a web page and saving them as a .zip file. The images were collected from four major sporting categories: Baseball, Basketball, Football, and Pokemon.

Below is an image downloaded from ALT.

Building a Data Pipeline

Tensorflow has an API tf.data.Dataset that can be used to build a data pipeline rather than loading all the data into memory at once. This is useful when you have a large dataset that cannot fit into memory. Instead of accessing the API directly, I used Keras utility function tf.keras.utils.image_dataset_from_directory that builds an image dataset from a directory by applying a number of preprocessing steps such as rescaling images to 256x256, generating labels, changing color to RGB, dividing to batch sizes of 32, shuffle and more.

The following code was used to create a data pipeline. ‘data’ here corresponds to the local directory where collected images were saved.

# Build the dataset
data = tf.keras.utils.image_dataset_from_directory('data')

... Found 845 files belonging to 4 classes.

However, this was not a dataset that was loaded into our memory. Instead, it was a dataset object that is a generator which meant I had to iterate over it and get batches of images.

# Convert data into a numpy iterator
data_iterator = data.as_numpy_iterator()

# Get the first batch
batch = data_iterator.next()

Corrupted images were removed prior to building the data pipeline.

Preprocessing

The numeric representation of an image from our pipeline is between 0 and 255. For a deep learning model to optimize faster, we need to normalize the data to between 0 and 1 by dividing the data by 255. This was done most efficiently using the map function as i loaded the data using the data pipeline.

Inorder to prevent data leakage to the model, we must split the data into training, validation and test sets and only train the model using the training set. Validation set is used to tune the model and the test set to evaluate the model.There were 27 batches of images in total that were split into a 7:2:1 ratio.

# Define size of each batch
train_size = int(len(data)*.7)
val_size = int(len(data)*.2)+1
test_size = int(len(data)*.1)+1

print(f'Train batches {train_size}')
print(f'Validation batches {val_size}')
print(f'Test batches {test_size}')

... Train batches 18
... Validation batches 6
... Test batches 3

To allocate data according to the split ratios, I used .take and .skip methods available in our pipeline.

# Assing train data
train = data.take(train_size)

# Skip already assinged train data and assign validation data
val = data.skip(train_size).take(val_size)

# Skip already assinged train and validation data and assign test data
test = data.skip(train_size+val_size).take(test_size)

Model Development

I created a Convolutional neural network (CNN) model to classify the images using Sequential API. The Sequential API in Keras is a simple way to create a linear stack of layers in a neural network model. Conv2D and MaxPooling2D layers were used in the CNN as these layers work well together to extract features from the input image for image classification.

Conv2D applied a convolution operation to the input data, where a set of learnable filters were used to extract features from the input while MaxPooling2D reduced its spatial size and helped to reduce overfitting and computational cost.

Relu was used as the activation function as it is the most commonly used activation, while for the output, softmax activation was used as we have a multi-class classification problem.

Below figure shows a summary of the CNN model

I took advantage of CUDA GPUs, which helped us optimize the model by using callbacks like EarlyStopping, ModelCheckpoint, and TensorBoard. Training was a breeze and only took 29min 35s.

Following plot shows the training history of the model.

Training Observations

  • No indications of overfitting as both training and validation losses and accuracies are improving with each epoch and are closer to each other.
  • Accuracy was improved, and loss was signally dropped, indicating a good performance by the model.
  • Due to the diminished performance of the model on validation data, training was terminated at the 25th epoch.

Model Evaluation

Model was evaluated using the test dataset. As accuracy alone is not always a good metric to base our evaluation on, I employed a confusion matrix for better evaluation and interpretation of the model as well as Precision and Recall as additional metrics.

  • Precision - The number of true positives divided by the number of true positives plus the number of false positives. Precision is a measure of a classifier’s exactness.
  • Recall - The number of true positives divided by the number of true positives plus the number of false negatives. The recall is a measure of a classifier’s completeness.

Following is the coconfusion matrix heatmap for the test data.

Result was a perfect confusion matrix meaning no confusion between the classes. I managed to classify all the images correctly with near perfect precision and recall scores.

Actual and predicted labels are shown below

Conclusion

I was successful in designing a deep-learning image recognition model that can classify sports cards with 100% accuracy and evaluated the model’s capabilities by running it through an original test set in addition to a new set of images. Though the results accurately demonstrate the model’s effectiveness, 100% accuracy is not likely attainable in reality. This result of perfection may be attributed to the limited dataset or superior quality images used for training. To make the model more reliable, I can introduce random noise, use image augmentation, incorporate poor-quality images into the model, and retrain. I can also incorporate more training classes really elevate the model. Nevertheless, at its current development, it can be implemented into the ALT website to allocate users accurately into the correct social group.