Bubble Pop
What started as an inventory control system for a client became a home security project, which somehow turned into an AR game my kids play. Here's the whole story.
⚡ Clap to unleash a blast
🔥 Dodge lasers with your whole body
💃 Dance party mode with your kids
// The Story
It started with a client project — an inventory control system that needed computer vision to track items. That got me deep into YOLO object detection and real-time video processing. Once I saw what was possible, I thought: why not build a security system for my own house?
So I built Sentry — a fully local computer vision pipeline that captures video, runs YOLO object detection, tracks people with pose estimation, and logs events to a web dashboard. All on a Mac mini M4, nothing leaves the machine. Real-time person detection, skeleton tracking, the whole deal.
Then one night, staring at the skeleton tracking output, I had a completely different idea.
Then one night at 11:15 PM, I had a thought:
"Let's make a pose detection game that would be fun for a few little kids to play and interact with what's on the screen."
Three minutes later, a coding agent had built v1 from the Sentry baseline. It was terrible. It used the wrong camera, froze after a few seconds, and the hands were mirrored — move left, your on-screen hand goes right. Classic.
But it proved the concept. Seeing yourself on screen popping bubbles with your actual hands was immediately fun. Even broken, you could feel it.
So I kept going.
By 2 AM, we'd built 17 versions. Full body tracking with a neon skeleton overlay. Laser beams you dodge by ducking and jumping. A clap-to-blast mechanic where you charge up energy between your palms and fire it across the screen. Spring physics that make everything feel smooth and alive. Gesture-controlled menus — no mouse needed. And a progressive soundtrack built entirely from Web Audio oscillators.
But the real pivot came when I tested it with my family.
The original plan was arcade-style: punishing escape penalties, bomb bubbles, difficulty that pushes you to fail. Classic arcade design. Then I watched my daughter play and realized I had it all wrong.
"I don't want to punish the user for bubbles escaping. This is supposed to be fun for kids too. I want them to just be able to play for a long time… shared rewards so if something is coming down and two players are running to collect it, both people should get it. This is a fun community game where we are all having fun together."
That killed the arcade version and birthed what Bubble Pop actually is: a family celebration engine that happens to use your body. Think of it like a digital piñata — everyone swings, everyone cheers, candy falls for everyone. Nobody loses.
"The game should make you feel good and be happy to play."
Before going to sleep at 2 AM, I told my AI assistant one last thing:
"When I wake up in the morning, I can't wait to play it with my daughter to show her all of the amazing stuff we built!"
That's the soul of this project. It's not about the tech. It's about seeing two glowing skeletons on screen — a parent and a kid — popping bubbles together.
// The Build Log
Version by version — the interesting pivots from one overnight session.
🫧 "The First Pop"
Built from the Sentry baseline in 3 minutes. Camera feed as background, colorful bubbles floating across the screen, wave your hands to pop them. Four bubble types, combo multipliers, Web Audio synth pops.
What went wrong: Wrong camera. Froze after a few seconds. Hands mirrored. No error handling — any hiccup killed the whole pipeline.
The lesson: The first version always breaks. But it proved the concept — seeing yourself on screen popping bubbles was immediately fun.
💀⚡ "Neon Skeleton"
Five rounds of iteration from live playtesting. This is where the game got its identity.
- Full 17-keypoint body skeleton drawn on screen with color-coded neon bones — cyan head, pink arms, purple torso, green legs
- Laser dodge obstacles — horizontal and vertical beams sweep across the screen. Duck, jump, sidestep.
- Clap-to-blast — bring hands together, charge for 0.8 seconds, fire a glowing projectile that plows through everything
- 3 lives with screen shake on hit
- Camera picker, mirror fix, hardened error handling, auto-reconnecting SSE
Architecture locked in: FastAPI + YOLO pose on M4 GPU pushes data via SSE → vanilla JS canvas at 60fps interpolates between updates.
🎮🔥 "It Feels REAL"
1,567 lines of handcrafted game engine. This is where it stopped feeling like a tech demo and started feeling like an actual game.
The smoothness breakthrough: Spring physics. Every skeleton keypoint lives inside a spring-damper system — pulled toward the real tracked position, but with momentum and damping. Camera sends data at 20fps, player sees 60fps butter. The actual AI model didn't change — we just made the presentation layer respect the limits of the data.
- Scale awareness — measures shoulder distance, scales everything proportionally. A 4-year-old at 3 feet and an adult at 8 feet both get a playable experience.
- Feet are live — ankles as poppers, stomp gesture creates expanding shockwave
- Four powerups — Shield, Fire Hands, Slow-Mo, Magnet
- Gesture menus — hover hand over button for 1.5s to activate, fully hands-free
- Progressive audio engine — adaptive kick, hi-hat, bass, tempo changes
- Game feel juice — hit stop, screen flash, score ticker, streak fire
🔧 "Making It Feel Right"
Four rapid-fire fixes from live playtesting in 15 minutes:
- The jiggle fix — replaced spring physics with EMA (exponential moving average). Dead simple:
pos += (target - pos) * alpha. No overshoot, no jiggle. Just butter. - Benchmarked YOLO at different sizes — settled on imgsz=384 for 96 FPS on M4
- Stuck keypoints — points that never got real data were drawing at (0,0). Added validity tracking.
- Weapon lock system — clasp hands once, rapid fire with aim, body-scale-relative thresholds
- MPS GPU — enabled Metal acceleration for pose estimation. ~15% faster.
🎯 "This Is a Family Game"
The moment that changed everything. Tested with the family, had a revelation.
Punish missed bubbles
One player competes
"GAME OVER"
Bombs cost hearts
Bubbles just float away
Everyone plays together
"GREAT GAME! 🎉"
Only positive feedback
"The game should make you feel good and be happy to play. It shouldn't be overloading."
Co-op gesture designed: two players hold all four hands together → 1 second charge → screen-clearing rainbow shockwave. Nearly impossible to trigger accidentally, incredibly satisfying when intentional.
👨👩👧 "Family Fun Update"
Multiplayer, waves, shared rewards. Multiple skeletons tracked simultaneously, wave-based progression, wave names that feel like chapters: "🫧 Let's Pop!", "✨ Golden Bubbles!", "🌧️ Bubble Rain!", "💃 Dance Party!"
Then I went to sleep and my AI assistant kept building through the night — weapon evolution, sound redesign around wind chimes and music boxes, progressive tutorials, brand identity alignment.
⚡🌋 "Polish & Floor Is Lava"
Morning playtesting and a string of rapid improvements:
- One weapon, one team — stripped the confusing 3-weapon system, built a single satisfying beam with physical reload (pull hands apart to reload like pump-action)
- The shadowBlur massacre — found 55 Gaussian blur calls per frame killing performance. Eliminated all of them. Simple shapes with alpha look 90% as good at 10% of the GPU cost.
- Floor Is Lava — backend silently scans for furniture with YOLO every 5 seconds. Player sees nothing. Then randomly: "🌋 THE FLOOR IS LAVA!" — furniture zones glow green as safe spots, 12 seconds to get your feet on something.
"I don't want the furniture or anything to be known until it's about to be floor is lava time. That's when the glowing or whatever can start."
🏎️ "Lean & Mean"
Final performance pass. Furniture detector off by default (enable with --lava). MJPEG background dropped to 480×270 at quality 35. Inference optimized to imgsz=256 — 2.3x faster. Combined: ~40% less CPU/GPU load.
Two clean run modes: --game for fast and smooth, --game --lava when you want the surprise.
// The Tech
Everything runs locally on a Mac mini M4. No cloud, no subscriptions, no accounts. Just a computer and a webcam.
YOLOv11n-pose
17-keypoint body tracking on Apple Silicon GPU (MPS). ~96 FPS inference at 256px. Real-time skeleton of everyone in frame.
SSE Streaming
Server pushes pose data at 30fps via Server-Sent Events. Browser interpolates with EMA smoothing for 60fps rendered movement.
Web Audio Synth
All sounds generated from oscillators. C major pentatonic scale, triangle waves, no audio files. Mathematically incapable of sounding bad.
Zero Dependencies
The entire game is one HTML file — ~3,400 lines of vanilla JS. No React, no frameworks, no build step. Canvas 2D + requestAnimationFrame.
FastAPI Backend
Python server handles camera capture, YOLO inference, and SSE streaming. Async everywhere, lightweight, stable.
Apple Silicon
Mac mini M4, 24GB RAM. Metal Performance Shaders for GPU inference. Everything stays on-device — nothing leaves the machine.
// What's Next
This is a living project. Ideas on the table:
// About
Built by Bryan Brodsky — exploring what's possible when you point AI at real problems and refuse to stop iterating. Bubble Pop started as an inventory system for a client, became a home security project, and turned into a family-tested AR experience by morning. That's the fun of building things.
The whole codebase is 29 commits of "what if we tried this?" — and this page will keep growing as the project does.