Counterclockwise block-by-block knowledge distillation for neural network compression
Abstract Model compression is a technique for transforming large neural network models into smaller ones.Knowledge distillation (KD) is a crucial model compression technique that involves transferring knowledge from a large teacher model to a lightweight student model.Existing knowledge distillation methods typically facilitate the knowledge transf