1005 Alcyon Dr Bellmawr NJ 08031
+1 484-446-2566
support@bmcpros.com
Sara Beery came to MIT as an assistant professor in the Department of Electrical Engineering and Computer Science (EEC) at MIT, eager to focus on ecological challenges. She revolves around the opportunity to develop her research career, applying her expertise to computer vision, machine learning, and data science to address real-world problems in conservation and sustainability. Beery was attracted by the institute’s commitment to “earth computing” and set out to bring her approach to environmental and biodiversity monitoring globally.
In the Pacific Northwest, salmon have a disproportionate impact on the health of their ecosystems, and its complex reproductive needs have attracted Beery’s attention. Every year, millions of salmon begin to migrate to lay eggs. Their journey begins with a freshwater flow bed where eggs hatch. Young Salmon Fry (newly hatched salmon) enters the ocean, where they spend several years adults. As an adult, salmon return to the stream of birth to lay eggs, thus ensuring the continuation of their species by storing the eggs in the gravel of the stream bed. Both male and female salmon died shortly after providing the next generation of salmon to river habitat.
Throughout the migration, salmon support a wide range of organisms in the ecosystems they pass through. Salmon, for example, brings nutrients such as carbon and nitrogen from the upstream of the ocean, enhancing their availability to these ecosystems. Furthermore, salmon are key to many predator relationships: they are a food source for various predators such as bears, wolves, and birds, while helping to control other populations, such as insects, through predation. After they die from spawning, the decomposed salmon carcasses also supplement valuable nutrients to the surrounding ecosystem. Salmon migration not only maintains their own species, but also plays a crucial role in the overall health of the rivers and oceans in which they live.
Meanwhile, salmon populations play an important role in the region both economically and culturally. Commercial and recreational salmon fisheries have made a significant contribution to the local economy. For many indigenous peoples in the Pacific Northwest, salmon have significant cultural value because they are at the heart of diet, tradition and rituals.
Monitor salmon migration
Increased human activity, including overfishing and hydropower development, as well as habitat loss and climate change, has had a significant impact on salmon populations in the region. As a result, effective monitoring and management of salmon fisheries is very important to ensure a balance between competitive ecology, culture and human interests. To track the success of threatened populations, assess the success of recovery strategies, guide fishing seasonal regulations, and support the management of commercial and recreational fisheries, accurate is crucial for the seasonal migration of salmon into their spawning in the Natal River. Accurate population data helps decision makers adopt the best strategies to protect the health of ecosystems while meeting human needs. Monitoring salmon migration is a labor-intensive and inefficient undertaking.
Beery is currently leading a research project that aims to simplify salmon monitoring using cutting-edge computer vision methods. The project is in line with Beery’s broader research interests, which focuses on interdisciplinary spaces between artificial intelligence, the natural world and sustainability. Its relevance to fishery management is well suited for MIT’s Abdul Latif Jameel Water and Food Systems Lab (J-WAFS) funding. Beery’s 2023 J-WAFS Seed Grant is the first research fund received since joining MIT faculty.
Historically, surveillance efforts have relied on humans to manually calculate salmon from the river bank using vision. Over the past few decades, an underwater sonar system has been implemented to help calculate salmon. These sonar systems are essentially underwater cameras, but they differ in that they use acoustics rather than light sensors to capture the presence of fish. Using this method requires people to calculate the tent by the river based on the output of the sonar camera hanging on the laptop to count the salmon. Although the system is an improvement on the original method of vision monitoring salmon, it still depends heavily on human efforts and is a difficult and time-consuming process.
Automated salmon monitoring is necessary for better management of salmon fisheries. “We need these technical tools,” Beery said. “We can’t keep up with the need to monitor, understand and study these truly complex ecosystems that we can use without some form of automation.”
To automatically calculate salmon populations migrated in the Pacific Northwest Pacific, project teams including EEC doctoral student Justin Kay have been collecting data in video formats from Sonar cameras on different rivers. The team annotated a subset of data to train computer vision systems to autonomously detect and calculate the number of fish when they migrate. Kay describes how the model calculates the process of each migrating fish: “Computer vision algorithms are designed to locate the fish in a frame, draw a box around it, and then track it over time. If the fish is detected on one side of the screen and then left on the other side of the screen, we count it as a move upstream.” On the river where the team created training data for the system, it produced strong results with only 3% to 5% of count errors. This is much lower than the team and collaborative stakeholders’ goals, whose counting errors are no more than 10%.
Testing and Deployment: Balancing human efforts with automation
The researchers’ technology is being deployed to monitor salmon migration on the newly restored Klamas River. Recently, four dams on the river were removed, making it the largest dam removal project in U.S. history. The dam fell after a 20-year campaign with scientists, environmental groups and commercial fishermen. After the dam is removed, 240 miles of rivers are now free to flow, and salmon can access nearly 800 square miles of habitat. Beery noted that in the Klamas River, salmon populations regenerate almost immediately: “I think this is within eight days of the dam coming down, and they start seeing salmon actually migrating upstream outside the dam.” In a collaboration with California Trout, the team is currently processing new data to fit and create a custom model that can then be deployed to help calculate newly migrated salmon.
The system revolves around the challenges of training models to accurately calculate fish in unfamiliar environments with changes such as riverbed features, water clarity, and lighting conditions. These factors can significantly alter the performance of fish on the sonar camera output and confuse the computer model. When deployed in new rivers where data has not been collected before, such as Klamath, the performance reduction and error magnitude of the system will increase significantly to 15-20%.
The researchers built an automated adaptation algorithm in the system to overcome this challenge and create scalable systems that can be deployed to any site without human intervention. This customization technique can automatically calibrate new conditions and environments to accurately calculate migrated fish. In tests, the automatic adaptation algorithm is able to reduce the counting error to the range of 10% to 15%. Improvements in computing errors through custom features mean that the technology is closer to a situation where it can be deployed in new locations without additional human efforts.
Use “Fish Box” to achieve real-time management
Another challenge facing the research team is developing an effective data infrastructure. In order to run a computer vision system, videos produced by sonar cameras must be mailed to the lab via the cloud or manually from the river site. These methods have obvious drawbacks: a cloud-based approach is limited due to the lack of internet connections at remote river site locations and the latency problem introduced by data passing data.
Instead of relying on these methods, the team implemented a power-efficient computer that created a “fish box” that can be used on the spot to perform processing. The Fishbox consists of a small, lightweight computer with optimized software that fisheries managers can plug into their existing laptop and sonar cameras. The system is then able to run the salmon counting model directly at the sonar site without an internet connection. This allows managers to make decisions per hour by hour, thus supporting more sensitive, real-time management of salmon populations.
Community Development
The team also worked to bring a community together to monitor salmon fishery management in the Pacific Northwest. “It’s really exciting to have us passionate about getting stakeholders who are keen to get them working and to work more rigorously with them,” Beery said. “I think especially when you’re working in food and water systems, you need to collaborate directly to help promote impact because you want to make sure that what you’re developing is actually meeting the needs of the people and organizations you’re helping support.”
Last June, Beery’s lab held a workshop in Seattle, convening NGOs, tribes, and state and federal departments of fish and wildlife to discuss the use of automated sonar systems to monitor and manage the use of salmon populations. Kai noted that the workshop was “a great opportunity for everyone to share the different ways they use sonar and think about how the automation approach we are building fits with that workflow.” The discussion now continues through the shared Slack channel created by the team, with more than 50 participants. Convening this group is a major achievement, as many of these organizations would otherwise have no chance to unite and work together.
expect
As the team continues to adapt the computer vision system, refine its technology, and interact with a variety of stakeholders, from indigenous communities to fishery managers, the project is expected to make significant improvements in the efficiency and accuracy of salmon monitoring and management in the region. As Beery grows, J-WAFS SEED Grant is helping to maintain challenges such as fishery management.
“The fact that there is a J-WAFS seed grant here in MIT allows us to continue working on the project as we move here.” Beery commented: “It also expands the scope of the project and allows us to stay active in collaborative on what I think is a very important and impactful project.”
As J-WAFS marks the 10th anniversary of this year, the program aims to continue to support and encourage MIT teachers to engage in innovative projects aimed at promoting knowledge and creating practical solutions for the real-life impact of challenges in global water and food systems.