In this work, we propose HC-SMoE (Hierarchical Clustering for Sparsely activated Mixture of Experts), a task-agnostic expert merging framework that reduces SMoE model parameters without retraining.

This project applies hierarchical clustering to group car models by attributes like horsepower, price, and fuel efficiency. It involves data preprocessing, cluster analysis, and visualization to ...

Abstract: The study applies a dataset comprising of more than 20,000 documents to a comprehensive comparative analysis of clustering algorithms: Mean Shift, Hierarchical and K-Means. The study's main ...

Abstract: We propose the Compact Clustering Attention (COCA) layer, an effective building block that introduces a hierarchical strategy for object-centric representation learning, while solving the ...