On a bright Tuesday, June 10th, Northeastern University’s Silicon Valley campus was buzzing with a unique energy. The NEURAI Lab hosted a pivotal workshop for the National Deep Inference Fabric (NDIF), gathering a diverse group of thinkers — from neuroscientists to computer science professors and students; from our own Oakland and Boston campuses, SFSU, and beyond. The mission of the day: to tackle one of the most profound challenges of our time — understanding the minds of the artificial intelligences we’ve created.
The event was more than a series of presentations; it was the start of a new chapter. NDIF, a groundbreaking project born at our Boston campus and fueled by a grant from the National Science Foundation (NSF), is planting its roots in the Bay Area. Led by Professor David Bau of the Khoury College at Boston campus, the project aims to build a community of researchers and industry partners right here in Silicon Valley, equipped with the tools to look inside the “black box” of AI.
The Urgency of Looking Within
As AI systems become more powerful, they also become more mysterious. The event opened by highlighting this urgent need for interpretability, citing a powerful quote from Anthropic’s Dario Amodei: “I consider it basically unacceptable for humanity to be totally ignorant of how they work.” Professor David Bau, the driving force behind NDIF, articulated the challenge perfectly. We are in an era of unprecedented AI capability, yet a profound scientific mystery remains:
How do these models actually work?
This question is central to the work done at the Bau Lab, which studies the rich internal structure of deep networks. Their goal is to lay the groundwork for a future of human-AI collaborative software engineering, where humans and machines can teach and learn from each other. This is the core mission of NDIF: to create a national resource that provides the computational tools and infrastructure needed to conduct scientific inquiry into AI systems, asking not just “what” they can do, but “how” and “why”.
Brains, Bytes, and Braille: A Lesson from Neuroscience
Highlighting the event’s interdisciplinary spirit was a captivating talk by Dr. Santani Teng, a neuroscientist from the Smith-Kettlewell Eye Research Institute. Dr. Teng drew fascinating parallels between the human brain and the artificial neural networks that seek to emulate it. He explained that our brains are, in a sense, modular, hierarchical, and plastic—able to change and adapt.
These aren’t just abstract biological principles; they are guideposts for AI research. Dr. Teng showed how artificial networks can be used as scientific models to better understand our own minds, citing remarkable research on how the brain transforms the touch of braille into meaning. The talk was a powerful reminder that the path forward in AI is a two-way street, where understanding the brain can help us build better machines, and our machines can help us unlock the secrets of the brain.
From Big Ideas to Hands-On Tools
The excitement in the room was palpable as the discussion shifted from the theoretical to the practical. If NDIF’s goal is to open the black box, what tools do we use to pry it open?
A notable presentation was given by Caden, a talented sophomore who has been instrumental in developing NNsight and the Logit Lens for NDIF. He presented the Interpretability Workbench, a hands-on platform for dissecting AI models.
Key highlights from his workbench session included:
- Transformer Fundamentals: An essential background on transformer architecture, including the role of residual streams, attention mechanisms, and MLPs.
- The Logit Lens: A powerful technique that allows researchers to “unembed hidden states early to inspect model predictions at different depths,” revealing how a model’s “thinking” evolves.
- “Overthinking the Truth”: A real-world phenomenon where models initially know the correct answer but lose accuracy in later layers, a quirk the Logit Lens helps uncover.
The interactive component was particularly engaging, with attendees working in groups to explore factual recall using the online workbench, investigating fundamental questions like, “Where do language models store facts?”
Breakthrough Discoveries and a National Community
Professor Bau also gave the audience a glimpse into the groundbreaking research already emerging from the NDIF community, which has produced 26 research papers and grown to over 900 active members on Discord across 47 institutions.
He showcased several fascinating discoveries about how LMs work internally:
- Knowledge Localization: Factual knowledge can be precisely located within specific layers and positions in a model.
- The “Lookback” Mechanism: Models use sophisticated pointer-like mechanisms to track references and beliefs, enabling capabilities that resemble a theory of mind.
Dual Induction Routes: Models have separate pathways for literal token copying versus conceptual understanding that can work across different languages.
The Vision: Building AI Leadership Through Understanding
Perhaps the most important message of the day was that solving AI’s mysteries isn’t just about computational power. As Professor Bau noted, “The mysteries in AI will not be solved by the chips. It is about the people. Building strength in AI is about teaching, enabling, and empowering an ecosystem.“
The NDIF project represents a comprehensive approach to this challenge through:
- Research Progress: Advancing the fundamental science of AI interpretability.
- Outreach and Training: Growing the community of researchers who can work on these problems.
- Software Capabilities: Building accessible tools like the NNsight API.
- Operational Infrastructure: Providing the shared computational resources needed for large-scale research.
Looking Forward & How to Get Involved
The event concluded with exciting plans for NDIF’s future, including broader adoption across more research fields, improved software, and expanded hardware deployment, with potential for a European DIF. It was made clear that understanding AI is a collaborative mission.
For those inspired to contribute, there are several ways to get involved with the NDIF project and its community of early adopters:
- Join the Community: Connect with hundreds of researchers for real-time chat on the NDIF Discord, engage in longer-form conversations on the NDIF Discussion Forum, and tune into the weekly public seminars from the Talks at the Bau Lab to stay at the forefront of interpretability research.
- Start Exploring with NNsight: Dive into the open-source NNsight library using the comprehensive walkthrough and tutorials on LLM interpretability techniques to begin your own experiments. The library works with PyTorch and can be easily installed on your local machine.
- Contribute Directly to the Code: As the project is open-source, you can explore the repositories and contribute directly on GitHub.
The workshop hosted by the NEURAI Lab was a powerful demonstration that through open collaboration and accessible tools, we are moving closer to revealing the inner workings of AI—and everyone is invited to be part of the journey.
