I investigate the intelligence encoded inside a deep neural network.
Department of Electrical and Computer Engineering, Boston University
407-07, Photonics Center, 8 St Mary's St, Boston, MA 02215
shawn24 [at] bu [dot] edu
Pic courtesy: My wife, Payel Basak

I am a lifelong proud gator and a Ph.D. candidate in Electrical Engineering at Boston University, advised by Prof. Kayhan Batmanghelich at Batman Lab. I collaborate closely with Dr. Clare B. Poynton from Boston University Medical Campus. Before our lab moved to Boston, I was a Ph.D. student in the Intelligent Systems Program (ISP) at the University of Pittsburgh. While at Pitt, I used to collaborate with Dr. Forough Arabshahi from Meta, Inc. At Pitt, I was also a cross-registered student at Carnegie Mellon University, where I registered for the courses Foundations of Causation and Machine Learning (PHI 80625) and Visual Learning and Recognition (RI 16-824). My current research interest lies in robustness and generalization by leveraging vision language representations to understand, explain and audit any pre-trained deep neural network. I believe that understanding a deep model's behavior is essential to mitigating bias and engendering trust in AI. During the summer of 2024, I worked as an Applied Scientist Intern with the AWS SAAR team at Amazon in New York City, under the guidance of Dr. Mikhail Kuznetsov. My project focused on learning robust representations to mitigate systematic errors in pre-trained self-supervised models applied to AWS logs.

Prior to that, I graduated with a Master's degree in Computer Science from the University of Florida. I was fortunate to work as a graduate assistant in Data Intelligence Systems Lab (DISL) lab under the supervision of Prof. Mattia Prosperi and Prof. Jiang Bian, where I conducted research on the intersection of deep learning and causal inference. I also worked closely with Prof. Kevin Butler as a Graduate Research Assistant at the Florida Institute of Cybersecurity (FICS) Research.

[ Google Scholar  |  Semantic Scholar  |  OpenReview  |  Github  |  LinkedIn  |  Twitter ]


Applied Scientist Intern, Security Analytics and AI Research (SAAR), AWS (Summer 2024)
Ph.D. Electrical Engineering (2023 - Present)
Ph.D. Intelligent Systems (2021 - 2023) transferred to BU
Cross-Registered Student (2021-2023)
Master of Science in Computer Science (2019 - 2021)

Research

My core research interest lies in representation learning across computer vision and medical imaging, with a particular focus on interpretability and explainable AI. I investigate the representations learned across different modalities, architectures, and training strategies to enhance their generalizability, robustness, and trustworthiness. Specifically, I aim to answer the following research questions:
1. Can we decipher the failure modes of a deep model through multimodal vision-language representations and large language models (LLMs) for improved reliability and debugging? (In submission, pre-print, code)
2. Can we lean robust vision-language representations with limited data efficiently and localize disease with sentences? [Mammo-CLIP (MICCAI 2024, top 11%]
3. Can we extract a mixture of interpretable models from the representation of a blackbox model using human interpretable concepts? [MoIE (ICML 2023 + SCIS@ICML 2023)]
4. Can we use robust mixture of interpretable models for data and computationally efficient transfer learning? [MoIE-CXR (MICCAI 2023, top 14% + IMLH@ICML 2023)]
5. Can we leverage radiology reports localizing a disease and its progression without ground-truth bounding box annotation? [AGXNet (MICCAI 2022 + RAD: AI)]

At UF, I was interested broadly in biomedical informatics with a focus on causal inference. I developed deep learning models, namely DPN-SA (JAMIA 2021), PSSAM-GAN (CMPB-U 2021) and DR-VIDAL (AMIA 2022, oral), to compute propensity scores for the efficient estimation of individual treatment effects (ITE). For a detailed overview of my Master's research, refer to the slides available at this link.

Deep Learning Resources

My friend Kalpak Seal and I have developed a comprehensive repository where you can access a curated collection of academic lecture videos focused on machine learning, deep learning, computer vision, and natural language processing (NLP). If you're interested in contributing to this resource, feel free to collaborate with us by submitting a pull request. Whether it's adding new lecture videos or improving the existing structure, we welcome all contributions!

News

Publications

Academic Projects

  • Explaining why Lottery Ticket Hypothesis Works or Fails Proposal Report Code
    As a part of CMU 16-824: Visual Learning and Recognition, we studied the relationship between pruning and explainability. We validated if the explanations generated from the pruned network using Lottery ticket hypothesis (LTH) are consistent or not. Specifically we pruned a neural network using LTH. Next we generated and compared the local and global explanations using Grad-CAM and Concept activations respectively.
  • Efficient classification by data augmentation using CGAN and InfoGAN Proposal Report Code
    As a part of CIS6930 - Deep Learning for Computer Graphics, we used two novel variants of GAN: 1) Conditional GAN and 2) InfoGAN to augment the dataset and compare the classifier’s performance using a novel dataset augmentation algorithm. Our experiments showed that with less training samples from the original dataset and augmenting it using the generative models, the classifier achieved similar accuracy when trained from scratch.
  • Deep Colorization Problem Description Report Code
    As a part of CIS6930 - Deep Learning for Computer Graphics, a CNN was created to train to color grayscale face images.
  • Deep Multitask Texture Classifier(MTL-TCNN) Report Code
    As a part of the independent research study in Spring 2020 (Feb - April), under Dr. Dapeng Wu, I developed a Deep Convolutional Multitask Neural Network (MTL-TCNN) to classify textures. We used an auxiliary head to detect normal images other than textures to regularize the main texture detector head of the network.
  • Implementation of TCNN3 paper Code
    As a research assistant under Dr. Dapeng Wu, I implemented TCNN3 architecture in end to end manner from scratch (no pretraining) for DTD dataset, discussed in the paper Using filter banks in Convolutional Neural Networks for texture classification.
  • Implementation of Deep Counterfactual Networks with Propensity-Dropout Code
    As a research assistant of DISL, I implemented the paper Deep Counterfactual Networks with Propensity-Dropout, which was subsequently used in my other research.
  • Peer to peer (p2p) network Problem Description Code Video
    This project was created as a part of the p2p project for Computer Networks (CNT5106C) at the University of Florida for the Master's in Computer Science program. A simplified peer to peer network where any number of peers can share any type of file among themselves. Implemented in Java.

Academic Service

Conference reviewer
  • International Conference on Learning Representations (ICLR) 2024, 2025
  • Association for the Advancement of Artificial Intelligence (AAAI) 2024, 2025
  • Neural Information Processing Systems (NeurIPS) 2023, 2024
  • Artificial Intelligence and Statistics (AISTATS) 2025
  • Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024
  • IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR) 2024
  • Causal Learning and Reasoning (CLeaR) 2024
  • ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB) 2022
Journal reviewer
  • Journal of Biomedical Informatics (JBI)
  • Medical Image Analysis (MedIA)
  • Journal of the American Medical Informatics Association (JAMIA)
  • Computer Methods and Programs in Biomedicine (CMPB)
  • Biometrical Journal
  • Information Fusion
Workshop reviewer
  • Workshop on GenAI for Health: Potential, Trust and Policy Compliance (GenAI4Health), NeurIPS 2024
  • Causal Representation Learning workshop (CRL), NeurIPS 2023
  • Spurious Correlations, Invariance and Stability (SCIS), ICML 2023
  • Interpretable Machine Learning in Healthcare (IMLH), ICML 2023

Teaching

Courses
Introduction to Software Engineering (EC 327) - Fall 2023
Teaching AssistantBoston University
Deep Learning (EC 523) - Fall 2024
Teaching AssistantBoston University

Talks

Invited Talk @ MedAI, Stanford University

Fall 2023, ISP AI Forum @ University of Pittsburgh [Slides]

Invited Talk @ ISP AI Forum, University of Pittsburgh

Fall 2023, ISP AI Forum @ University of Pittsburgh [Slides]

Oral Talk @ AMIA 2022 Annual Symposium

Oral Presentation @ AMIA 2022 Annual Symposium [Slides]

Tutorials

Tutorial on Variational Autoencoder (VAE)

Tutorial on Pearl's Do Calculus of causality

As a software engineer

In my past life, I spent 6+ years in software service/product development as a full stack software engineer across Lexmark International India Pvt Ltd and Cognizant Technology Solutions India Pvt Ltd, using Angular/Angular.js, C#/.Net, WCF web services, Node.js, Oracle and MS SQL Server. For Cognizant, I used to build WCF webservices using contract-first approach. For Lexmark, I was a part of the development team which created this.

Community service

I have been an active member of Cognizant Kolkata Outreach council (NGO related engagement for development of underprivileged children in Kolkata). Refer to the link for the certificate of recognition. Refer here for pictures clicked by me in one of such events in 2014.

Miscellaneous

I am originally from Kolkata, once the capital of India. I have lived in Gainesville (FL), Pittsburgh (PA), Boston (MA) and New York city (NY).

Interviews

Podcast hosted by Kishlay Das for the admission and research in the US for MS/Ph.D. aspirants