What We Built

We wanted a simple application with minimal external dependencies that delivers a smooth, responsive user experience. The resulting app features:

• Three-pane vertical layout: Left pane for loading local video files, center pane for the video player, and right pane for configuring and running machine learning models.

• YOLO-based Species Detection: Object detection models that identify and localize species directly on the video frames.

• Visual Overlays: Dynamic bounding box overlays showing confidence levels and model parameters (with configurable confidence and IoU thresholds).

• Performance Optimizations: Optimized frame processing (skipping redundant frames) and real-time visualization of results as they process, complete with a progress bar.

Our Vibe Coding Workflow & Prompt Log

We used Claude Desktop as our primary driver. Here is the exact sequence of prompts we used, divided into our planning (Chat) phase and our execution (Code) phase.

Phase 1: Planning, Architecture & Agent Design (Chat Section)

Before writing any code, we used Claude to establish a solid game plan, define roles, and create a context file to guide the generation process.

Step 1: Establishing the Stack
We defined our target user (marine ecologists) and set constraints for a low-dependency, high-responsiveness stack.

"I want to build a web-based viewer that can load and play MP4 videos. I also want to detect and classify objects in those videos using YOLO-based models and visualize the model outputs directly in the viewer. Eventually, I would like the platform to support human-in-the-loop annotation and review workflows.

Can you recommend the best software stack for this application? My primary goal is a smooth and responsive user experience. This app is developed by and is intended for marine ecologists. I want it to be a simple application with minimal external dependencies."

Step 2: Structuring the Agent Roles
To work efficiently, we instructed Claude to think of the build in terms of three distinct agent roles (Frontend, Backend, ML) with verifiable steps.

"I want one agent for frontend
I want one agent for backend
I want one agent to implement an ML model to detect and classify objects in the video
Divide this project into different tasks; in a way that each task can be executed by a prompt
think about how you are going to verify each of the proposed modules
create a plan for me to execute"

Step 3: Refining Infrastructure & Security
We paused to address a few critical logistical questions regarding file handling and safety.

"What database are these videos going to be saved in? security vulnerabilities? where will the videos be loaded from?"

Step 4: Creating a Context Markdown File
We locked down our decisions by instructing Claude to export the entire roadmap into a markdown file that could be read by future workspace sessions.

"create a plan; save it as a context into a .md file. I will later load this .md file to execute this plan."

Step 5: Logging User Input
We wanted a clean log of our collaborative session, so we had Claude compile our inputs.

"can you save all the user messages in this session in a md file? if the user message was in response to your question, flag with a brief system message about the question"

Phase 2: Implementation, Layout, & ML Tuning (Code Section)

With the plan safely stored in our directory, we moved into execution.

Step 1: Setting up the Workspace
We pointed Claude to our saved plan to set up our folder structures and a clean Conda environment.

"There is a .md file in the working folder. gather context from the .md file.
look at what libraries need to be installed
install them in a conda environment - reduce redundant packages where applicable.
create a working environment and folder structure to execute the plan. outlined"

Step 2: Spawning the Agents
We utilized Claude's Test-Driven Development capabilities (/superpowers:test-driven-development) to start coding the core services concurrently.

"create 2 agents:
one agent to implement frontend
one agent to implement backend

execute each agent concurrently"

Note on real-world friction: During this phase, Claude requested permissions to run backend scripts via Bash. When local security settings blocked the sub-agent's automated execution, Claude adapted and handled the direct file implementation manually to keep us moving.

Step 3: Running the Application
We got the core files generated and asked for clear directions on how to spin up our local environments.

"how do I run my front end and backend?"

Step 4: Designing the Three-Pane UI
Once the basic app ran, we customized the layout to make it highly functional for ecologists.

"I do not see a button to load a video. I want to load a video from my local machine.
Give me 3 panes: vertically divided
give me a button to load the video in the left pane
I want the video displayed in the center pane
I want a button to load and execute the ML model on the right pane"

Step 5: Model Integration & Inference Optimizations
Next, we fed Claude a sample video file, configured the YOLO model, and added UI controls for IoU and Confidence thresholds. We also made sure the model didn't bottleneck performance by running on every single frame.

@"[video file]"
after the model is run, I want the outputs of the model to be overlayed on the video for display
show me what model is being used
give me option for confidence and iou configuration
I don't want the model to run on every frame. optimise that
once done, check if it is correctly implemented using the video attached to this prompt

Step 6: Improving Performance & UX
Because ML models can take time to load, we asked for optimization strategies and a way to monitor the process.

"add a progress bar while running ML model
The model loading is taking a really long time, can you think of ways to optimize that?"

Step 7: Streaming Real-Time Results
Instead of waiting for the entire video to process before showing results, we wanted the frames to update live as the model processed them.

"I want to visualise the results from the ML model as the model finishes running on the frame"

Step 8: Finalizing the Prompt Archive
Just as before, we concluded our session by exporting our chat logs to a markdown file to save our workspace state.

"can you save all the user messages in this session in a md file? if the user message was in response to your question, flag with a brief system message about the question"

We’d love to hear your thoughts on this setup! Leave a comment below!

If you have any questions for our two AI Clinicians this month (who are also co‑hosts of the vibe‑coding events), please feel free to visit the clinic post and leave your question there. https://buildingbridges.cioos.ca/ask-an-expert/post/ai-clinic-1---may-2026-Ba0kxvIdLSwA85D

If you missed our live session during the Building Bridges Community Launch Event, or simply want to revisit the insights shared, you can now watch the full recording.

About this Session

As ocean data continues to grow faster than teams can typically manage, non-profits and resource-constrained organizations face a unique challenge: how to turn growing data backlogs into timely, actionable conservation efforts.

This session explores how artificial intelligence can help close that gap. We break down the practical, accessible, and ethical AI tools available today that can help your organization streamline operations, improve data workflows, and amplify your conservation impact—without requiring a massive technical budget or specialized computer science background.

Key Takeaways from the Recording:

• Accessible AI Tools: An overview of free or low-cost AI technologies that non-profits can start exploring today.

• Simplifying Workflows: How machine learning can support backend data management, freeing up valuable staff time for hands-on research and community engagement.

• Strategic Decision-Making: Essential criteria for non-profit leaders to responsibly evaluate and select AI platforms that align with their mission and values.

• Real-World Context: Practical examples of how ocean organizations are successfully using AI tools to navigate data challenges.

Watch the full recording here: https://www.youtube.com/watch?v=PIcFSgje4y4

To learn more: https://www.fisheries.noaa.gov/new-england-mid-atlantic/marine-mammal-protection/developing-viable-demand-gear-systems

Come join us on June 11 for an ocean themed, casual vibe coding session! This meetup is a fantastic opportunity to break the ice for our ongoing AI Clinic and to meet our clinicians Devi and Emily. We will spend a couple of hours building small ocean focused projects together using various artificial intelligence tools.

Absolutely no experience is required. Just bring your curiosity and a willingness to experiment!

What to expect:
We are keeping the format very simple. We will start with brief introductions and share some project ideas. From there, you can form small groups, work solo, or we might even decide to build a single project all together. We will spend the majority of the time building, experimenting, and helping each other, followed by some optional demos at the end.

A few ideas already on the table:

• A web viewer for hosting ocean video datasets and annotations

• Tools that automate repetitive parts of ocean and environmental research workflows

• Ocean data exploration and visualization projects

• Whatever interesting ocean related idea you have been wanting to try!

When and Where:

• Date: June 11 (Thursday)

• Time: 2:00 PM to 3:30 PM ADT

• Location: Hybrid (Join us in person at the Steele Ocean Sciences Building Room 344, 6375 Edzell Castle Cir, Halifax, or virtually from anywhere)

Watch the event recording:

https://youtu.be/4NTUD-lH7Mk

Download the Slides: https://drive.google.com/file/d/1A0LOyG2e7ELWjJnV3T9Zf4i0oN-WU6yZ/view?usp=drive_link

Video Timeline:

• 0:00:00 | Slide #1-3 | Opening & Welcome

• 0:08:22 | Slide #4-42 |Keynote: The Pan-Canadian AI Strategy - From Research to Impact (Dr. Elissa Strome (CIFAR))

• 0:56:47 | Slide #44-59 | Environmental Data as a Geopolitical Battleground: AI & Open Data Sovereignty (Dr. Schallum Pierre (Université Laval))

• 1:15:36 | Slide #61-68 | From Project to Community: Building Bridges through a Community (Shayla Fitzsimmons (CIOOS Atlantic))

• 1:25:37 | Slide #70-83 | Interactive Session

• 1:40:55 | Slide #85-132 | Keynote: ML Models for Decarbonized & Efficient Maritime Supply Chains (Dr. Loubna Benabbou (UQAR & Mila))

• 2:18:58 | Slide #134-155 | Scaling up MPA Effectiveness Tracking for Global Targets (Beth Pike & Nikki Harasta (Marine Conservation Institute))

• 2:38:02 | Slide #157-176 | Successfully Delivering AI & Technology Products for Ocean Impact (Ted Schmitt (Allen Institute for AI - Ai2))

• 3:03:40 | Slide #178-188 | From Process to Impact: An AI Acceleration Methodology for Conservation (Diego Padilla Huamán (Equilibria) & Fanny M. Cornejo (Yunkawasi))

• 3:23:53 | Slide #189-190 | Wrap-up & Closing

Additional Resources shared by presenters:

Dr. Elissa Strome

• First Nations Principles of OCAP

• CIFAR Indigenous Perspectives in AI course

• Ontario Privacy Commissioner Principles for Responsible AI

• ISED Voluntary Code of Conduct for Generative AI

Dr. Schallum Pierre

• Article in French published in the Revue Défense Nationale: “Artificial Intelligence in ISR: The Ethical Stakes and Perspectives for Canada in the Face of Hybrid Threats” https://shs.cairn.info/journal-revue-defense-nationale-2026-1-page-93?lang=en&tab=resume

Dr. Loubna Benabbou

• ETA prediction: https://ieeexplore.ieee.org/abstract/document/10373707

• ETA monitoring: https://link.springer.com/article/10.1007/s13437-024-00342-9

• Speed prediction: https://www.mdpi.com/2077-1312/11/1/191?trk=organization_guest_main-feed-card-text

• Speed T water prediction: https://neurips.cc/virtual/2024/100566

• Currents Data downscalling: https://www.climatechange.ai/papers/neurips2025/49

• Emission prediction: https://www.climatechange.ai/papers/neurips2025/87

• Bio-Maribe Data Denoising: https://neurips.cc/virtual/2025/loc/san-diego/126987

Nikki Harasta and Beth Pike:

• MPAtlas: https://mpatlas.org/

• The MPA Guide: https://mpa-guide.protectedplanet.net/resources

• Gem Generator Prompt: https://github.com/Epersonf/gemini-gems/blob/main/Gem-Generator.md

• Elements of AI: https://course.elementsofai.com/

• Building Systems with ChatGPT API: https://learn.deeplearning.ai/courses/chatgpt-building-system/lesson/k0pk1/introduction

• Building Apps with Windsurf: https://learn.deeplearning.ai/courses/build-apps-with-windsurfs-ai-coding-agents/lesson/ym8if/introduction

• Windsurf: https://windsurf.com/pricing

 Questions that received written answers:

Shayla Fitzsimmons

Q: Could you tell us about some of the learning opportunities available in the Community?

A: Absolutely! We currently have two free and self-paced courses available, one that introduces AI from an oceans lens, and a second that introduces AI and ethics in an ocean context. And that's just the start--keep on eye on the community, there's more to come!

Dr. Loubna Benabbou

Q: Has there been any work done on how vessel optimization efforts (optimizing speed, fuel usage, etc.) contribute to or subtract from noise reduction efforts?

A: We are working on that in a research project with my colleagues at ISMER and we will soon publish a paper on the subject.

Q: Given AI for weather forecast research has gotten lots of attention recently, with some models like Aurora outperforming traditional models in storm prediction. Do you think there is a lot of potential for AI in oceanography, for example prediction of currents data?

A: Yes, I believe that hybrid physical models will accelerate the research and discovery in oceanography. We are working with our colleagues at ISMER to use ML models in current prediction.

Q: How confident are you that the maritime industry—particularly shipowners—will adopt artificial intelligence tools and agree to share their data?

A: It's very difficult to put a level of confidence, but in our collaboration, we are offering a free tool so they were very interested. We are working only with accessible data.

Q: I am curious about the precision of ML vs deep learning algorithms for vessel speed, ETA or emission prediction. Has there been important developments in deep learning approaches in recent years, how do they compare to traditional ML?

A: Effectively, with Trasformer and XLSTM we had a more accurate prediction.

Q: First off, thank you for your interesting and thorough presentation, my question is the following : If we do manage to reduce emissions by improving trajectories and technologies, have you taken into account how the rebound effect (reduction in expected gains from new technologies that increase the efficiency of resource use, because of behavioral or other systemic responses) could influence emissions in the long run?

A: Excellent question, we are working on that and also on how we can have a multi-objective optimization to take into account the reduction of emissions and noise, and also to ensure operational performance.

Beth Pike

Q: Great presentation! May have missed this, but what are the final outputs of the workflow you described, and how do they finally help you assess the MPAs?

A: Currently we are using the tool to check assessments, both before and after human conducted assessments to monitor for good outcomes and refining the tool. We hope to someday make the tool accessible to users who want to input data from their own MPAs to get an assessment for their MPA against the MPA Guide framework.

Ted Schmitt

Q: What standards or methodologies do you use on the back-end to integrate datasets from multiple sources?

A: We have so far integrated open source data, such as Sentinel 1, Sentinel 2, and Landsat. We are just beginning to pull in other data sources, and questions of standards and methodologies are still being developed. There is a whole question of how to make data AI ready. We are participating in efforts like the NSF funded FAIR grant: https://www.nsf.gov/awardsearch/show-award?AWD_ID=2531922

Q: What types of data do Skylight and Shippy use in their models?

A: Sentinel 1, Sentinel 2, VIIRS, and AIS are the primary satellite data sources. We do use high-res imagery from Maxar (now Vantor) on a very limited basis. But we squeeze as much as we can out of free, open source data to keep costs down. For Shippy, we have started pulling in data from sources like SkyTruth's Cerulian, Global Fishing Watch, and Protected Seas Navigator as just a few initial examples.

Q: Ted, thank you for a fantastic presentation! There are some interesting models that I'm keen to explore more. A number of your examples were understandably based on land data. Can the same principles be applied in the OlmoEarth Studio environment to develop models based on ocean data?

A: Thank you! We would very much like to do that, but have not yet begun. I'd be happy to have a conversation with anyone interested in doing that. My email is teds@allenai.org.

Fanny M. Cornejo

Q: Great presentation Fanny! You've described all kinds of different gains and benefits of implementing AI in your processes; was there anything that you think has been -lost- in making this switch? For example in some of the more human components of your work, such as storytelling.

A: Thank you for the question! Comms it’s a good example. We still “talk” to the AI tools we use to give context and ensure the stories still have a heart. However, in terms of giving structure, grammar, language, etc., it has definitely made things more efficient!

I am wondering if there is any open source model specialized in sensitive data detection to integrate in our application.

Thanks in advance,

Yagmur