Academic Journal
Controllability Analysis for Vision State Space Models: A Structural Interpretability Framework.
| Title: | Controllability Analysis for Vision State Space Models: A Structural Interpretability Framework. |
|---|---|
| Authors: | Mabrok MA, Zafari Y |
| Source: | IEEE transactions on neural networks and learning systems [IEEE Trans Neural Netw Learn Syst] 2026 Aug 18; Vol. PP. Date of Electronic Publication: 2026 Aug 18. |
| Publication Model: | Ahead of Print |
| Publication Type: | Journal Article |
| Language: | English |
| Journal Info: | Publisher: Institute of Electrical and Electronics Engineeers Country of Publication: United States NLM ID: 101616214 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 2162-2388 (Electronic) Linking ISSN: 2162237X NLM ISO Abbreviation: IEEE Trans Neural Netw Learn Syst Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Piscataway, NJ : Institute of Electrical and Electronics Engineeers |
| Abstract: | State space models (SSMs) have emerged as a compelling alternative to transformers for visual recognition, offering linear computational complexity while maintaining competitive accuracy. However, the lack of interpretability tools designed specifically for the recurrent dynamics of these models remains a significant gap: existing saliency methods originate from convolutional or attention-based architectures and do not account for the sequential state propagation that governs information flow in SSMs. We introduce controllability analysis, a framework grounded in control theory that quantifies the structural influence of each input position on the internal state dynamics of vision SSMs. We derive two complementary indices: a Jacobian-based measure that tracks sensitivity propagation through the full sequence via an efficient $O(LN)$ backward recursion, and a Gramian-based measure that admits a closed-form, fully parallelizable solution for diagonal state matrices. Unlike gradient-based attribution methods that produce class-specific explanations, controllability indices measure intrinsic model properties that are invariant to the target class. Experiments on seven image classification benchmarks spanning diverse visual domains (microscopy, optical coherence tomography (OCT), dermatoscopy, radiography, natural images, apparel, and remote sensing) demonstrate that controllability analysis produces perfectly class-agnostic explanations (cross-class correlation of 1.0 compared to -0.45 to 0.18 for Grad-CAM across datasets) and reveals layerwise information flow patterns that vary across imaging domains. On the dataset where the model is best trained (BloodMNIST, 98% test accuracy), the proposed Jacobian index also outperforms Grad-CAM on standard insertion/deletion faithfulness (Jacobian $0.63 {\,}\pm {\,}0.02$ versus Grad-CAM $0.47 {\,}\pm {\,}0.03$ , mean ± standard deviation across three training seeds, with nonoverlapping per-seed 95% bootstrap CIs); on the lower accuracy datasets Grad-CAM remains stronger on this class-specific metric, a gap we discuss as a diagnostic of incomplete alignment between the model's internal dynamics and the classification task rather than a failure of the structural framework. A complementary internal-attention IoU metric, measuring alignment between each method's saliency and the model's own L |
| Entry Date(s): | Date Created: 20260818 Latest Revision: 20260929 |
| Update Code: | 20260930 |
| DOI: | 10.1109/TNNLS.2026.3718747 |
| PMID: | 42611646 |
| Database: | MEDLINE |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=21622388&ISBN=&volume=PP&issue=&date=20260818&spage=&pages=&title=IEEE transactions on neural networks and learning systems&atitle=Controllability%20Analysis%20for%20Vision%20State%20Space%20Models%3A%20A%20Structural%20Interpretability%20Framework.&aulast=Mabrok%20MA&id=DOI:10.1109/TNNLS.2026.3718747 Name: Full Text Finder (for New FTF UI) (ns324271) Category: fullText Text: Full Text Finder MouseOverText: Full Text Finder |
|---|---|
| Header | DbId: cmedm DbLabel: MEDLINE An: 42611646 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Controllability Analysis for Vision State Space Models: A Structural Interpretability Framework. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AU" term="%22Mabrok+MA%22">Mabrok MA</searchLink><br /><searchLink fieldCode="AU" term="%22Zafari+Y%22">Zafari Y</searchLink> – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22101616214%22">IEEE transactions on neural networks and learning systems</searchLink> [IEEE Trans Neural Netw Learn Syst] 2026 Aug 18; Vol. PP. <i>Date of Electronic Publication: </i>2026 Aug 18. – Name: Publisher Label: Publication Model Group: PubInfo Data: Ahead of Print – Name: TypePub Label: Publication Type Group: TypPub Data: Journal Article – Name: Language Label: Language Group: Lang Data: English – Name: TitleSource Label: Journal Info Group: Src Data: <i>Publisher: </i><searchLink fieldCode="PB" term="%22Institute+of+Electrical+and+Electronics+Engineeers%22">Institute of Electrical and Electronics Engineeers </searchLink><i>Country of Publication: </i>United States <i>NLM ID: </i>101616214 <i>Publication Model: </i>Print-Electronic <i>Cited Medium: </i>Internet <i>ISSN: </i>2162-2388 (Electronic) <i>Linking ISSN: </i><searchLink fieldCode="IS" term="%222162237X%22">2162237X </searchLink><i>NLM ISO Abbreviation: </i>IEEE Trans Neural Netw Learn Syst <i>Subsets: </i>MEDLINE – Name: PublisherInfo Label: Imprint Name(s) Group: PubInfo Data: <i>Original Publication</i>: Piscataway, NJ : Institute of Electrical and Electronics Engineeers – Name: Abstract Label: Abstract Group: Ab Data: State space models (SSMs) have emerged as a compelling alternative to transformers for visual recognition, offering linear computational complexity while maintaining competitive accuracy. However, the lack of interpretability tools designed specifically for the recurrent dynamics of these models remains a significant gap: existing saliency methods originate from convolutional or attention-based architectures and do not account for the sequential state propagation that governs information flow in SSMs. We introduce controllability analysis, a framework grounded in control theory that quantifies the structural influence of each input position on the internal state dynamics of vision SSMs. We derive two complementary indices: a Jacobian-based measure that tracks sensitivity propagation through the full sequence via an efficient $O(LN)$ backward recursion, and a Gramian-based measure that admits a closed-form, fully parallelizable solution for diagonal state matrices. Unlike gradient-based attribution methods that produce class-specific explanations, controllability indices measure intrinsic model properties that are invariant to the target class. Experiments on seven image classification benchmarks spanning diverse visual domains (microscopy, optical coherence tomography (OCT), dermatoscopy, radiography, natural images, apparel, and remote sensing) demonstrate that controllability analysis produces perfectly class-agnostic explanations (cross-class correlation of 1.0 compared to -0.45 to 0.18 for Grad-CAM across datasets) and reveals layerwise information flow patterns that vary across imaging domains. On the dataset where the model is best trained (BloodMNIST, 98% test accuracy), the proposed Jacobian index also outperforms Grad-CAM on standard insertion/deletion faithfulness (Jacobian $0.63 {\,}\pm {\,}0.02$ versus Grad-CAM $0.47 {\,}\pm {\,}0.03$ , mean ± standard deviation across three training seeds, with nonoverlapping per-seed 95% bootstrap CIs); on the lower accuracy datasets Grad-CAM remains stronger on this class-specific metric, a gap we discuss as a diagnostic of incomplete alignment between the model's internal dynamics and the classification task rather than a failure of the structural framework. A complementary internal-attention IoU metric, measuring alignment between each method's saliency and the model's own L<subscript>2</subscript>-magnitude attention pattern, shows controllability outperforming Grad-CAM on six of the seven datasets. Directional decomposition of the controllability profiles further shows that influence progressively sharpens with depth, transitioning from diffuse responses in early layers to sparse, high-magnitude peaks in deep layers that align with semantically meaningful structures. The proposed framework provides a principled foundation for understanding, debugging, and improving vision SSMs across application domains. – Name: DateEntry Label: Entry Date(s) Group: Date Data: <i>Date Created: </i>20260818 <i>Latest Revision: </i>20260929 – Name: DateUpdate Label: Update Code Group: Date Data: 20260930 – Name: DOI Label: DOI Group: ID Data: 10.1109/TNNLS.2026.3718747 – Name: AN Label: PMID Group: ID Data: 42611646 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42611646 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1109/TNNLS.2026.3718747 Languages: – Code: eng Text: English Titles: – TitleFull: Controllability Analysis for Vision State Space Models: A Structural Interpretability Framework. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Mabrok MA – PersonEntity: Name: NameFull: Zafari Y IsPartOfRelationships: – BibEntity: Dates: – D: 18 M: 08 Text: 2026 Aug 18 Type: published Y: 2026 Identifiers: – Type: issn-electronic Value: 2162-2388 Numbering: – Type: volume Value: PP Titles: – TitleFull: IEEE transactions on neural networks and learning systems Type: main |
| ResultId | 1 |