To establish more economical and efficient non-destructive intelligent recognition technology for tobacco leaf maturity, a lightweight network model MobileViT-CBAM on mobile devices was constructed. Firstly, a dataset was built by collecting images of the middle and upper leaves of ‘Yunyan87’ with different maturity. The CBAM attention mechanism module was introduced into the MobileViT structure to enhance the feature expression ability of fresh tobacco leaf maturity images. Secondly, the original activation function Swish was replaced with the smoother SMU function to help the model converge faster. Finally, transfer learning was employed to improve the training efficiency and generalization ability of the model and achieve the classification of fresh tobacco leaf maturity in complex field environment. Results showed that MobileViT-CBAM exhibited an accuracy of 92.81% in maturity classification of fresh tobacco leaves, which is significantly superior to the models of VGG16, ResNet34, Vision Transformer, Swin Transformer, MobileNetV2, and MobileViT. The proposed MobileViT-CBAM model can effectively identify the maturity degree of tobacco leaves, providing technical support for the visual system of intelligent tobacco harvesting equipment.