Using Deep Learning in Real-Time for Clothing Classification with Connected Thermostats

Thermal comfort is associated with clothing insulation, conveying a level of satisfaction
with the thermal surroundings. Besides, clothing insulation is commonly associated with indoor
thermal comfort. However, clothing classification in smart homes might save energy when the end-
user wears appropriate clothes to save energy and obtain thermal comfort. Furthermore, object
detection and classification through Convolutional Neural Networks have increased over the last
decade. There are real-time clothing garment classifiers, but these are oriented towards single
garment recognition for texture, fabric, shape, or style. Consequently, this paper proposes a CNN
model classification for the implementation of these classifiers on cameras. First, the Fashion
MNIST was analyzed and compared with the VGG16, Inceptionvv4, TinyYOLOv3, and ResNet18
classification algorithms to determine the best clo classifier. Then, for real-time analysis, a new
dataset with 12,000 images was created and analyzed with the YOLOv3 and TinyYOLO. Finally,
an Azure Kinect DT was employed to analyze the clo value in real-time. Moreover, real-time
analysis can be employed with any other webcam. The model recognizes at least three garments
of a clothing ensemble, proving that it identifies more than a single clothing garment. Besides, the
model has at least 90% accuracy in the test dataset, ensuring that it can be generalized and is not
overfitting.