Merge AI and underwater photography to reveal the hidden ocean world | MIT News

In the northeastern United States, the Bay of Maine represents one of the most biologically diverse marine ecosystems on Earth, home to whales, sharks, jellyfish, herring, plankton and hundreds of other species. But even if this ecosystem supports abundant biodiversity, it is experiencing rapid environmental changes. Maine’s bays are 99% faster than the world’s oceans, and the consequences are still developing.

A new research program developed by MIT Sea Grant, called Lobstger, is abbreviation for marine biological biological systems by generating representatives – bringing together artificial intelligence and underwater photography to document marine life vulnerable to these changes and sharing it with the public in new visual ways. Co-led by MIT Sea Grant Keith Ellenbogen and MIT Mechanical Engineering Phd student Andreapoulos, the project explores how generative AI can scale up scientific storytelling with live-based photographic data.

Just as the cameras of the 19th century changed our ability to record and reveal the natural world, capturing life in unprecedented detail and bringing distant or hidden environments into view – the generated AI marks a new boundary for visual storytelling. Like early photography, AI opens up a space for creativity and concept, challenging how we define authenticity and how we communicate scientific and artistic perspectives.

Generative models are trained only in Ellenbogen’s selected library of original underwater photos – each image is made with artistic intent, technically accurate, accurate species recognition and clear geographical environment. By constructing high-quality data sets based on real-world observations, the project ensures that the resulting images maintain visual integrity and ecological relevance. Additionally, Lobstger’s model is built using custom code developed by Mentzelopoulos to protect the process and from the output of any potential bias from external data or models. Lobstger’s generative AI is built on real photography and expands the researchers’ visual vocabulary to deepen public connections with the natural world.

At the heart of Lobstger is operating at the intersection of art, science and technology. The project draws on the visual language of photography, the observational rigor of marine science, and the computing power of generating AI. By uniting these disciplines, the team is not only developing new ways to visualize marine life, but also reimagining how to tell an environmental story. This comprehensive approach makes Lobstger both a research tool and a creative experiment, reflecting MIT’s long tradition of interdisciplinary innovation.

As we all know, underwater photography in coastal waters of New England is difficult. Limited visibility, rotating sediments, air bubbles, and unpredictable movements of marine life all pose ongoing challenges. Ellenbogen has encountered these challenges over the past few years and has established a comprehensive record of biodiversity in the region through the project “Space of Sea: Visualizing New England’s Marine Wilderness”. This large underwater image dataset lays the foundation for training Lobstger’s generative AI model. These images cover various angles, lighting conditions and animal behavior, resulting in a visual archive that is both artistic and eye-catching and biologically accurate.

Lobstger’s custom diffusion model is trained to replicate not only the biodiversity Ellenbogen documentation, but also the artistic style he used to capture it. By learning from thousands of true underwater images, the model internalizes the details of fine particles such as natural lighting gradients, species-specific shading, and even atmospheric textures created by suspended particles and refracted sunlight. The result is that the image not only looks visually accurate, but also feels immersive and moving.

All of these models can unconditionally generate new, synthetic but scientifically accurate images (i.e., no user input/guidance is required) and conditionally enhance real photos (i.e., image-to-image generation). By integrating AI into your photography workflow, Ellenbogen will be able to use these tools to restore details in turbid water, adjust lighting to emphasize key themes, and even simulate scenes that are almost impossible to capture on site. The team also believes that this approach may benefit other underwater photographers and image editing, facing similar challenges. This hybrid approach aims to speed up the curation process and enable storytellers to construct a more complete, coherent visual narrative of life beneath the surface.

In a key series, Ellenbogen captures high-resolution images of lion’s mane jellyfish, blue sharks, American lobsters and marine people (Cool) Free diving in coastal waters. “Getting high-quality datasets is not easy,” Ellenbogen said. “It requires multiple dives, missed opportunities and unpredictable conditions. But these challenges are part of making underwater documentation difficult and meaningful.”

Mentzelopoulos developed the original code to train a family of potential diffusion models for Lobstger based on Ellenbogen images. Developing such models requires a high level of technical expertise, while training models from scratch is a complex process that requires hundreds of hours of calculation and meticulous hyperparameter adjustment.

The project reflects a parallel process: live documentation through photography and model development through on-site training. Ellenbogen works in the field, capturing rare and brief encounters with marine animals. Mentzelopoulos works in the lab to transform these moments into machine learning environments that can expand and reinterpret the visual language of the ocean.

“The goal is not to replace photography,” Mentzelopoulos said. “This is built and complemented on the basis of the foundation – making invisible visible and helping people see environmental complexity in a way that resonates emotionally and intellectually. Our model is designed not only to capture biological realism, but also to drive real-world engagement and action.”

Lobstger points to a hybrid future that integrates direct observation with technical explanations. The team’s long-term goal is to develop a comprehensive model that can visualize a wide range of species in the Maine Gulf and ultimately apply a similar approach to marine ecosystems around the world.

Researchers believe that photographic and generative AI form continuity rather than conflict. Photography captures texture, light, and animal behavior during actual encounters – and AI extends the vision beyond what is understandable, inferred or imagined based on scientific data and artistic vision. Together, they provide a powerful framework for conveying science through image making.

In areas where ecosystems are changing rapidly, visual behavior is more than just documents. It becomes a tool of consciousness, participation, and ultimately protection. Lobstger is still in its infancy and as the project develops, the team looks forward to sharing more discoveries, images and insights.

Answer from the leading image: Use Lobstger’s unconditional model to generate the left image and the right image is real.

For more information, please contact Keith Ellenbogen and Andreas Mentzelopoulos.