Over the past week I've been working on a game written in JavaScript (in the browser) which can be played with hand gestures seen by a camera (wanted to use an IR camera but couldn't get one in the time I had). It is the well-know Shell Game!

Blood, sweat and tears went into this because I wrote my code extremely bad so it became a huge clutter.

Will refactor some day, but I'm happy with the result!

A VR/AR OS, with gesture recognition and virtual tools, to substitute workstations.

Imagine painting a picture in a AR canvas, for example.

Or using photoshop in VR for a distraction free enviroment. For coding, I would want to use a real keyboard, of course, but I would use virtual screens for everything.

The possibilities are endless, but people would probably use it mostly for porn, obviously.

I have a system design question I was hoping u guys could help me with.

So I want to make a display that shows an animated wallpaper. I want to have a camera in the display that will recognize finger gestures and will change the wallpaper accordingly (shuffle to a new one, etc.)
EX: when the current wallpaper is displaying, when the user shows the camera the peace sign, it will switch the current wallpaper to a random wallpaper.

I used tensorflow and the mediapipe hands library to get the finger gesture recognition to work. I used tensorflow, python, and openCV.

Now that the finger gesture recognition is finished, I moved to the animated wallpapers and I made a couple with p5js.

How could I best combine the two? Should I switch from tensorflow in python to tensorflow.js? Or should I find a way to re-do my animated wallpapers in python?

My original plan was having the foundation of the whole system be python. This is so I can have the finger gesture recognition run on the main thread, and have another thread deal with the wallpapers.

Let me know if u guys have any ideas or if I’m not clear. Thanks!!!