I wanted to use PoseNet to control various bird sounds. Playing around with Google's bird sound project inspired me, and I would like to create a similar interface (but would need to purchase the library of sounds) and then use PoseNet to trigger specific sounds in replace of the mousePress.
My original idea was that if you shimmy your shoulders that perhaps a sort of cacophonous flapping or symphony of birds would be triggered. With PoseNet, I felt the tracking was unsteady at times, and the amount of shimmy-ing would need to be over-exaggerated. I decided that the way you move your entire body should control the rate of various bird sounds. In this way, the user’s approach to the camera becomes a dance in itself, like how the Bird of Paradise attracts its mate.
The four points that trigger sounds are:
Left Hand: Riflebird
Right Hand: Parotia
Left Shoulder: Nightingale
Right Shoulder: Raven
I mapped each bird’s song to the PoseNet’s keypoint’s Y position. Between the hand positions and shoulder positions, I found myself curling and expanding my body in such a way to alter the way the sounds layered and interacted with each other.
Before the PoseNet keypoints are picked up by the camera, the preloaded GIFs show up on the top of the canvas. My question for this week: Is there a way to hide these GIFs altogether before they align on the canvas?