Browsing by Author "Xie, Siwei"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • No Thumbnail Available
    ItemOpen Access
    Accelerating segment anything models via token merging : a comparative study and a spectrum preservation-based approach
    (2025) Xie, Siwei
    The Segment Anything Model (SAM) has emerged as a significant advancement in image segmentation, demonstrating exceptional generalization across diverse datasets with minimal task-specific tuning. However, its computational demands, inherited from Vision Transformers (ViTs), pose considerable challenges for deployment in resource-constrained environments. This thesis addresses these challenges by integrating token merging strategies, which have proven effective in enhancing the efficiency of ViTs without additional training. Specifically, we conduct a comprehensive analysis of SAM’s architecture and adapt existing token merging techniques to reduce computational overhead while maintaining high segmentation accuracy. We propose an architecture for SAM that incorporates these strategies and evaluate its performance and computational efficiency across various datasets, showing that our approach effectively accelerates SAM’s inference speed while preserving segmentation quality. Furthermore, we propose GradToMe based on PiToMe, an innovative method that leverages gradient approximation and grid-based sampling to combine similar tokens. This approach emphasizes spectrum preservation to retain critical information during the token reduction process, thereby improving the effectiveness of token merging and further saving computational costs. Consequently, our results demonstrate that this approach enhances the feasibility of deploying SAM in real-time applications, making it more suitable for use in resource-limited environments without compromising performance. Code is available at: https://github.com/xxjsw/tome_sam.