I investigate the intelligence encoded inside a deep neural network.
Department of Electrical and Computer Engineering, Boston University
407-07, Photonics Center, 8 St Mary's St, Boston, MA 02215
shawn24 [at] bu [dot] edu

I am a lifelong proud gator and a PhD candidate in Electrical Engineering at Boston University, advised by Prof. Kayhan Batmanghelich at Batman Lab. I collaborate closely with Dr. Clare B. Poynton from Boston University Medical Campus. Before our lab moved to Boston, I was a PhD student in the Intelligent Systems Program (ISP) at the University of Pittsburgh. While at Pitt, I used to collaborate with Dr. Forough Arabshahi from Meta, Inc. At Pitt, I was also a cross-registered student at Carnegie Mellon University, where I registered for the courses Foundations of Causation and Machine Learning (PHI 80625) and Visual Learning and Recognition (RI 16-824). My current research interest lies in explainable AI by leverageing vision language representations to understand, explain and audit any pre-trained deep neural network. I believe that understanding a deep model's behavior is essential to mitigating bias and engendering trust in AI.

Prior to that, I graduated with a Master's degree in Computer Science from the University of Florida. I was fortunate to work as a graduate assistant in Data Intelligence Systems Lab (DISL) lab under the supervision of Prof. Mattia Prosperi and Prof. Jiang Bian, where I conducted research on the intersection of deep learning and causal inference. I also worked closely with Prof. Kevin Butler as a Graduate Research Assistant at the Florida Institute of Cybersecurity (FICS) Research.

[ Google Scholar  |  Semantic Scholar  |  OpenReview  |  Github  |  LinkedIn  |  Twitter ]

Research

My research interests span computer vision and medical image analysis, focusing on interpretability and explainable AI (X-AI). My work involves investigating different architectures of supervised, unsupervised and self-supervised deep neural networks to improve their generalizability, robustness, and trustworthiness. Specifically, I aim to answer the following research questions:
1. Can we decipher the failure modes of deep neural networks through multimodal vision-language representations and large language models (LLMs) for improved reliability and debugging? (Ongoing)
2. Can we develop new vision-language foundation models for previously unexplored imaging modalities, e.g, breast mammograms in biomedical AI? (In submission)
3. Can we extract interpretable models out of a pre-trained deep neural network, often treated as a black box, with the help of high-level human interpretable concepts? [MoIE (ICML 2023 + SCIS@ICML 2023)]
4. Can we use the concept-based interpretable models for better data and computational efficiency? [MoIE-CXR (MICCAI 2023 + IMLH@ICML 2023)]
5. Can we leverage weak labels from the radiology reports to localize a disease and its progression without relying on ground-truth bounding box annotation? [AGXNet (MICCAI 2022)]

During my time at UF, I was interested broadly in biomedical informatics with a focus on causal inference. I developed deep learning models, namely DPN-SA (JAMIA 2021), PSSAM-GAN (CMPB-U 2021) and DR-VIDAL (AMIA 2022), to compute propensity scores for the efficient estimation of individual treatment effects (ITE). For a detailed overview of my Master's research, refer to the slides available at this link.

News

[Jun 2024] I'm joining as an Applied Scientist Intern at Amazon Web Services (AWS) Security Analytics and AI Research (SAAR) Team in New York City. My project aims to discover systematic errors in self-supervised models.
[Oct 2023] I'm serving as a reviewer for ICLR 2024, Medical Image Analysis (MedIA) and CVPR 2024.
[Aug 2023] I'm invited to serve as a Program Committee (PC) member for AAAI 2024.
[Jul 2023] I'm serving as a reviewer for NeurIPS 2023 and the journal Computer Methods and Programs in Biomedicine.
[Jun 2023] I'm now a PhD candidate. Also, two papers are accepted at SCIS and IMLH workshops at ICML 2023.
[May 2023] Our work Distilling BlackBox to Interpretable models for Efficient Transfer Learning is accepted (Early accept, top ~ 14%) at MICCAI 2023.
[Apr 2023] Our work Dividing and Conquering a BlackBox to a Mixture of Interpretable Models: Route, Interpret, Repeat is accepted at ICML 2023.
[Dec 2022] I'm joining Boston University in Spring 2023 in the Department of Electrical and Computer Engineering following my advisor's move. My research will be supported by Doctoral Research Fellowship.
[Jun 2022] Our work on doubly robust estimation of ITE is accepted as an oral presentation at the AMIA 2022 Annual Sympossium.
[Jun 2022] Our work on weakly supervised disease localization is accepted at MICCAI 2022.
[Aug 2021] I'm joining the University of Pittsburgh in the Intelligent Systems Program under the supervision of Dr. Kayhan Batmanghelich in Fall 2021.
[May 2021] I graduated with a Master's degree in Computer Science from the University of Florida . Go Gators!!
[Apr 2021] Our work to balance the unmatched controlled samples by simulating treated samples using GAN, is accepted in the Journal of Computer methods and programs in biomedicine update.
[Dec 2020] Our work to estimate the Propensity score by dimensionality reduction using an autoencoder, is accepted in the Journal of the American Medical Informatics Association.
[Apr 2020] I'm joining DISL lab as a graduate assistant under the supervision of Prof. Mattia Prosperi and Prof. Jiang Bian.
[Aug 2019] I'm moving to the US to join the Master's program in the department of Computer Science in the University of Florida in Fall 2019.

Publications

Academic Projects

  • Explaining why Lottery Ticket Hypothesis Works or Fails Proposal Report Code
    As a part of CMU 16-824: Visual Learning and Recognition, we studied the relationship between pruning and explainability. We validated if the explanations generated from the pruned network using Lottery ticket hypothesis (LTH) are consistent or not. Specifically we pruned a neural network using LTH. Next we generated and compared the local and global explanations using Grad-CAM and Concept activations respectively.
  • Efficient classification by data augmentation using CGAN and InfoGAN Proposal Report Code
    As a part of CIS6930 - Deep Learning for Computer Graphics, we used two novel variants of GAN: 1) Conditional GAN and 2) InfoGAN to augment the dataset and compare the classifier’s performance using a novel dataset augmentation algorithm. Our experiments showed that with less training samples from the original dataset and augmenting it using the generative models, the classifier achieved similar accuracy when trained from scratch.
  • Deep Colorization Problem Description Report Code
    As a part of CIS6930 - Deep Learning for Computer Graphics, a CNN was created to train to color grayscale face images.
  • Deep Multitask Texture Classifier(MTL-TCNN) Report Code
    As a part of the independent research study in Spring 2020 (Feb - April), under Dr. Dapeng Wu, I developed a Deep Convolutional Multitask Neural Network (MTL-TCNN) to classify textures. We used an auxiliary head to detect normal images other than textures to regularize the main texture detector head of the network.
  • Implementation of TCNN3 paper Code
    As a research assistant under Dr. Dapeng Wu, I implemented TCNN3 architecture in end to end manner from scratch (no pretraining) for DTD dataset, discussed in the paper Using filter banks in Convolutional Neural Networks for texture classification.
  • Implementation of Deep Counterfactual Networks with Propensity-Dropout Code
    As a research assistant of DISL, I implemented the paper Deep Counterfactual Networks with Propensity-Dropout, which was subsequently used in my other research.
  • Peer to peer (p2p) network Problem Description Code Video
    This project was created as a part of the p2p project for Computer Networks (CNT5106C) at the University of Florida for the Master's in Computer Science program. A simplified peer to peer network where any number of peers can share any type of file among themselves. Implemented in Java.

Academic Service

Conference reviewer
  • Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024
  • IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024
  • Causal Learning and Reasoning (CLeaR) 2024
  • International Conference on Learning Representations (ICLR) 2024
  • Association for the Advancement of Artificial Intelligence (AAAI) 2024
  • Neural Information Processing Systems (NeurIPS) 2023
  • ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB) 2022
Journal reviewer
  • Medical Image Analysis (MedIA)
  • Journal of the American Medical Informatics Association (JAMIA)
  • Computer Methods and Programs in Biomedicine (CMPB)
  • Biometrical Journal
Workshop reviewer
  • Causal Representation Learning workshop (CRL), NeurIPS 2023
  • Spurious Correlations, Invariance and Stability (SCIS), ICML 2023
  • Interpretable Machine Learning in Healthcare (IMLH), ICML 2023

Teaching

Courses
Introduction to Software Engineering (EC 327) - Fall 2023
Teaching Assistant
Boston University

Talks

Oral Presentation @ AMIA 2022 Annual Symposium [Slides]

Fall 2023, ISP AI Forum @ University of Pittsburgh [Slides]

Tutorials

Tutorial on Variational Autoencoder (VAE)

Tutorial on Pearl's Do Calculus of causality

As a software engineer

In my past life, I spent 6+ years in software service/product development as a full stack software engineer across Lexmark International India Pvt Ltd and Cognizant Technology Solutions India Pvt Ltd, using Angular/Angular.js, C#/.Net, WCF web services, Node.js, Oracle and MS SQL Server. For Cognizant, I used to build WCF webservices using contract-first approach. For Lexmark, I was a part of the development team which created this.

Community service

I have been an active member of Cognizant Kolkata Outreach council (NGO related engagement for development of underprivileged children in Kolkata). Refer to the link for the certificate of recognition. Refer here for pictures clicked by me in one of such events in 2014.

Miscellaneous

I am originally from Kolkata, once the capital of India. I have lived in Gainesville (FL), Pittsburgh (PA) and Boston (MA).

Interviews

  • Podcast hosted by Kishlay Das to provide my insights for the admission and research in the US for MS/PhD aspirants.