Anyone ever tried using user-to-user collaborative filtering to classify the mnist digits dataset?

This is about as far as I got:
https://hastebin.com/obinoyutuw.py

It's literal copy-and-paste frankencode because this is only the second time I've ever done something like this, so pardon the hatchjob.