This article is based on Matt’s keynote at LDV Capital’s Annual General Meeting 2024.

Matt Uyttendaele is the director of the AI for Chemistry group at Meta and an Expert with us at LDV Capital. His group applies machine learning to material discovery for several applications: clean hydrogen, sorbents for direct air capture, and next-generation displays. Before that, Matt was the director of the perception team in Reality Labs which developed real-time computer vision algorithms for Meta’s AR/VR devices.

Prior to Meta, Matt worked at Microsoft Research and Bell Labs. At Microsoft, his research in computer vision, computational photography, and satellite imaging resulted in product features in Windows, Bing, and Office. At Bell Labs, Matt focused on video processing, developing compression algorithms (MPEG, H.263) and VLSI implementations.

His research has been presented at SIGGRAPH, CVPR, and the IEEE International Solid-State Circuits Conference. Matt holds 63 US patents. He received an MS in electrical engineering from Rensselaer Polytechnic Institute.

At our 2018 LDV Vision Summit, Matt spoke about enabling persistent Augmented Reality experiences across the spectrum of mobile devices. In 2023, he was one of 11 brilliant researchers we interviewed about sectors that will have the most societal impact from research advancements leveraging visual technologies and artificial intelligence in the next 10 years. Most recently, in 2024, we were honored to have him speak at our Annual General Meeting about a new application of AI that he has been working on for the last few years and how it relates to his career in visual technologies.

We are thrilled to share the lightly edited transcript below.

“Back in 2012, when I was a Computer Vision Researcher at Microsoft Research, 2 papers came out on deep learning: Deep Neural Networks for Acoustic Modeling in Speech Recognition & ImageNet Classification with Deep Convolutional Neural Networks. That was my first inclination that something was about to change. However, I was skeptical about it because I'd been working on beautiful hand-engineered solutions for computer vision for years. Suddenly these 2 papers came out and said, “You don't need to hand-engineer anything anymore. It's all data-driven! These AI magic black boxes will solve all these problems”. From 2012 to 2015, I was still skeptical, but curious, not using these techniques yet.

By 2015, I had moved to Meta and started deploying AI and deep neural net-based solutions and it was amazing on pretty much every product problem we worked on.

Deep neural nets ended up being the solution… I was still skeptical.

Can we run this at scale so that any Facebook user can upload a photo that we can then turn into a 3D photo? Yes, we got that to work.
Can we run this on a consumer device that tracks a person on a low-end computer? Yes, we got that to work too.
Can we do this on a portable device in a Quest headset to track your hands? Yep, we got that to work.
Can we use this to track, and build 3D environments in a headset traditionally one of the most difficult computer vision problems? Deep neural nets solved that better than anything we'd ever developed in the past!

Where is the next AI breakthrough?

I'm in the Fundamental AI Research group at Meta. I made a leap from visual tech and computer vision to creating the FAIR chemistry team.

We're not a science company and chemistry is not our domain. What can we contribute to the space of chemistry? It turns out there are some things that we can contribute. One, we have massive data centers and just the downtime of our data centers is enough that we have more compute available to us than any chemistry department in the world. So in those downtimes, we're creating massive data sets that aren't available in the chemistry and material science space. We've built some of the largest data sets in material science, and it turns out many of the deep learning techniques that we developed for computer vision are applicable to material discovery. We have deep learning experts and we were able to repurpose many of these techniques for material discovery.

Back in the '60s, some smart people started using computers to simulate atoms, they developed the technique called Density functional theory (DFT), which is a very good approximation to speeding up atomic simulations... They were awarded the Nobel Prize for that.

We have developed a different approximation, the graph neural network that simulates atoms. Atoms are on the nodes of these graphs, and the interactions between atoms are on the edges of the Graph.

Some highlights of the last two years: the team published a couple of papers. In one, we've shown a 2,000 X speed-up on the density functional theory versus traditional techniques. Also, we have another paper where we announced the discovery of 200 new materials using AI. To give some context, it usually takes a PhD thesis to discover a new material, and to be able to do that at that scale is exciting!

Here’s another way to look at it:

We estimate ~36% of the Department of Energy supercomputer cluster’s calculations are these DFT calculations where they're discovering new materials, or doing atomic simulations?

Yes, and a 2,000 X speed-up of that means their supercomputer suddenly becomes 2,000 times more powerful. That's significant!

In the next 30 years, climate tech is going to be important. We're gonna have to reinvent so many different industries!

Let's look at applications where material discoveries could have an impact:

Improved batteries
Petrochemicals: transport fuels, polymers (plastics, resins, fibers), detergents, coatings, construction chemicals, and medicine
Synthetic fuels (air travel)
Carbon removal
Catalyzers (green hydrogen)
Green steel
Clean concrete

At the heart of every one of these is a chemistry problem and AI could potentially accelerate discoveries.

AI to Address Climate Change

Climate change may make large areas of the planet uninhabitable in the coming 50 years, so anything we can do to address that will have a huge societal impact. There are many opportunities to leverage computer vision and AI to combat climate change. Measuring carbon in the earth’s biomass, detecting methane leaks, and picking optimal sites for solar installations are just a few examples of important climate-related problems that can be solved through AI that is trained to analyze the planet.

My recent focus has been to transfer some of the deep neural net (DNN) technology from core computer science (vision, speech, language) to material science. It turns out that many of the DNN architectures and other data-driven techniques transfer over well and could be used to discover new materials that function to better generate clean hydrogen or capture CO2 from the atmosphere.

At Meta, we are working on two climate tech projects:

Electro catalyzers for clean hydrogen: turning electricity into clean hydrogen. In other words, AI searches for replacements for expensive elements. Catalyzers are made of very expensive iridium or platinum. To replace it would be a huge win! We're using AI to search for cheaper materials.
Project on direct air capture: pulling CO2 out of the atmosphere. Open Direct Air Carbon Capture (OpenDACC-MOFs). Uses AI to search for materials that bind and release CO2 energy efficiently.

Consumer Devices Require Chemistry Breakthroughs

Consumer devices also all require new materials:

Improved batteries
Wrist device interface
Thermal conductivity
Lightweight wearable devices
Displays
Optics

Most of the team’s work is on new display materials for AR glasses. There are prototype AR glasses, and they use a display material called silicon carbide. It has nice optical properties, but it's heavy and expensive. We need novel materials to make it cheaper.

We've deployed our AI models to search for new materials that have the optical properties that we care about, that are lighter in weight and we are getting some promising results. We have found a new class of materials that we hope provides a breakthrough.

Lots of people in the chemical industries are very skeptical just like I was early in the days of computer vision. Computational chemistry has been an academic pursuit. It’s ready to disrupt the industry.

Generative AI Will Expand Materials Known to Man

AI will move material discovery from a wet lab to a computer. Everything I've talked about so far is about analyzing known materials for properties that we care about using AI, but just as generative AI has had a huge impact on visual tech, another big opportunity lies in chemistry. Generating new, unknown materials via generative techniques is going to be a huge opportunity. New materials are going to be discovered on the scale of weeks in the coming decades, as opposed to the years it takes today”.

Ray-Ban Meta Demos

Matt is also an active user of Ray-Ban Meta, the currently available generation of Meta’s smart glasses. His favorite things to do with the glasses are:

Listening to podcasts while jogging. The speakers in the frames allow hearing your surroundings while listening to media.
High-quality photos and videos taken easily in the moment.
Contextual AI (getting better with every update). Glasses give you info about what you are looking at, for example: trees, dog breeds, museum pieces.

You can check out more example photos and videos on Matt’s Facebook feed.

LDV Blog