Computer Vision for beginners - Reinforcement learning with games
HappyBearLandia - VR/AR/Robots/Simulator
primary TODO
Make Funny a resource - open ended humor
Play Stick of Truth
make games playable
complete adding object for each object in map - 10 left
add light fixture
use utils folder for single file functions and reorganize ./game/index.js to import utils folder
dynamic lighting + shadows
dynamic conversation - use transformer api
2 people talking to each other and then - you add to the conversation and it changes
play disco elysium
get piano bear to move paws and music notes to be playing out of piano
make a 3d model of a rainbow 🌈
switch to 1st person for aiming spells
add combo spell - freeze + missle metroid
add light switch → toggle 3d model
Backlog
make scene data use paths to load GLTF+OBJ at runtime
add audio reactive pool - https://threejs.org/examples/?q=water#webgpu_compute_water
get editor to change object location / properties and save to file system
for a given map image, individually label each object and then ask for a 3d model of the object, then place into scene, then visually inspect scene to ensure correct placement
add house
add lighting to house
add person
add door
add behaviors folder
change gravity
alacrity
add snowball item
add NPC with dialogue tree
add CPU optional for not online
add a level-list and make-room → returns a shareable url
get books on hadith
make eval - try to add a skill to llama
change passwords - security audit
call admissions and ask if they received application - check email
reorganize folder - rename for clarity/routing(inference,play-game,scenes folder in data) -
readme publish v1 - (how to install and run repo) - is readme comprhensive to onboard a new developer
walljump
music collision
object behaviors
magic spell fruit
installation
`bun install`
Usage instructions
how to run application locally
`bun run dev`
how to run augmented reality locally
op run --env-file=.env.op -- bun scripts/fs-sync-mediapipe.ts
Repo structure
blog - observable dashboard for publishing
data - stores local snapshot of cloud storage
game - simulator client that runs multiplayer scenarios to create training data - use this data in llama to generate new scenes
inferenece - augmented reality backend -rename to inference
scripts - for checking if setup was completed without error
test - integration tests for ensuring all routes render
tools - user interfaces for working with data
Routes
/play-game:scene_id - simulator plays scene.json => makes events
/replay-analyzer:scene_id - (event_log)=> adds comments to events
/robot:robot_id pick-and-place robot: play board game => makes events
/edit-game:scene_id Augmented Reality Media Pipe gestures - edit video game => makes scene.json
/data/:sensor_id - browser for sensor data or game data - 3x3 grid scrolls and lets you click on scene and then see people playing it
Goal: improve more neuralnet effectiveness from a sqlite (left side game : right side dashboard w/ evaluation improving for adding skill to llama - progress bar
train computer vision neural net from scratch
Goal: have 64 fun puzzles
study cause and effect and human choice for training data for AI - language model dynamic story
one server for web, one server for gpu - try not to delegate
remove data folder server.js → scripts/collect_sensor_data → /admin - has a record button for each sensor
chapters
vr
s3 bucket cloud storage decouple machines
self hosted IOT device setup w/ tailscale
sqlite
pytorch, tensorflow
mediapipe
three.js webgpu
bun
uv
jupyter
arm w/ lerobot
dance
yoga