Pro tip: always make sure your methods return the correct variable.

I’m currently working with deep neural networks using tensorflow. I needed to generate some test data and wrote a program to create it. I had two text files which each consisted of approximately 5000 lines of text.

I wrote a method that should sort out some words, and make my final data shorter. When I executed the program first time on our server, it spent about 25 minutes, then crashed due to MemoryError (which in Python means that the server didn’t have enough ram). That seemed quite weird since I only had about 10k lines of text, and I even sorted out a bunch of it, and the server has 128gb ram, and nothing’s using it.

Apparently I returned the wrong variable. That meant that my program tried to save 750 quadrillion lines of text rather than just a few thousand.

Always make sure to return the correct variables!