Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "csv yeah you know me"
-
I am an indie game developer and I lead a team of 5 trusted individuals. After our latest release, we bought a larger office and decided to expand our team so that we could implement more features in our games and release it in a desirable time period. So I asked everyone to look for individuals that they would like to hire for their respective departments. When the whole list was prepared, I sent out a bunch of job offers for a "training trial period". The idea was that everyone would teach the newbies in their department about how we do stuff and then after a month select those who seem to be the best. Our original team was
-Two coders
-One sound guy(because musician is too mainstream)
-Two artists
I did coding, concept art(and character drawings) and story design, So, I decided to be a "coding mentor"(?).
We planned to recruit
-Two coders
-One sound guy
-One artist (two if we encountered a great artstyle)
When the day finally arrived I decided to hide the fact that I am the founder and decided that there would be a phantom boss so that they wouldn't get stressed or try flattery.
So out of 7, 5 people people came for the "coding trial session". There were 3 guys and 2 girls. My teammate and I started by giving them a brief introduction to the working of our engine and then gave them a few exercises to help them understand it better. Fast forward a few days, and we were teaching them about how we implement multiple languages in our games using Excel. The original text in English is written in the first column and we then send it to translators so that they can easily compare and translate the content side by side such that a column is reserved for each language. We then break it down and convert the whole thing into an engine friendly CSV kind of format. When we concluded, we asked them if they had any questions. So there was this smartass, who could not get over the fact that we were using Excel. The conversation went like this:(almost word to word)
Smartass: "Why would you even use that primitive software? How stupid is that? Why don't you get some skills before teaching us about your shit logic?"
Me:*triggered* "Oh yeah? Well that's how we do stuff here. If you don't like it, you can simply leave."
Smartass: "You don't know who I am, do you? I am friends with the boss of this company. If I wanted I could have all of you fired at whim."
Me:"Oh, is that right?"
Smartass:"Damn right it is. Now that you know who I am, you better treat me with some respect."
Me: "What if I told you that I am not just a coder?"
Smartass:"Considering your lack of skills, I assume that you are also a janitor? What was he thinking? Hiring people like you, he must have been desperate."
Me:"What if I told you that I am the boss?"
Smartass:"Hah! You wish you were."*looks towards my teammate while pointing a thumb at me* "Calling himself the boss, who does he think he is?"
Teammate:*looks away*.
Smartass:*glances back and forth between me and my teammate while looking confused* *realizes* *starts sweating profusely* *looks at me with horror*
Me:"Ha ha ha hah, get out"
Smartass:*stands dumbfounded*
Me:"I said, get out"
Smartass:*gathers his stuff and leaves the room*
Me: "Alright, any questions?"*Smiling angrily*
Newcomers: *shake heads furiously*
Me:"Good"
For the rest of the day nobody tried to bother me. I decided to stop posing as an employee and teaching the newcomers so that I could secretly observe all sessions that took place from now on for events like these. That guy never came back. The good news however, is that the art and music training was going pretty well.
What really intrigues me though is that why do I keep getting caught with these annoying people? It's like I am working in customer support or something.16 -
So I was applying for a research position in linguistic department, and had the interview today.
Prof: So you know excel right
Me: (show a project to him to prove I at least know csv file)
Prof: Ok so you know excel.
Me: Yeah kinda.
Prof: Ok that's good. Cuz right now we are using amazon Turk, and the data they returned, which are excel files, are not really the way we want it.
Me: Ok sounds like a parser can fix it......
Prof: Yeah.... the students in the lab are doing it manually now
(Dead silence)
Prof: Ok move onto next matter7 -
I just got four CSV reports sent to me by our audit team, one of them zipped because it was too large to attach to email.
I open the three smaller ones and it turns out they copied all the (comma separated) data into the first column of an Excel document.
It gets better.
I unzip the "big" one. It's just a shortcut to the report, on a network share I don't have access to.
They zipped a shortcut.
Sigh. This'll be a fun exchange.3 -
So I setup a nice csv file for the customer to fill in the shop items for their webshop, you know? with a nice layout like
name - language - description etc.
(just temporary, because the legacy website is going under a ((sadly frontend only)) rework, so it now also has to display different 'kind' of products... and because the new cms isn't done yet they
have to provide the data with other means)
my thoughts were to create a little import script to write the file into the database.... keep in mind of the relations... etc...
guess what? TWO MONTHS later, I get a file with a custom layout, empty cells, sometimes with actual data, sometimes (in red / green text color) notes for me
I mean WHY.... WHY DO YOU MAKE MY LIVE HARDER???
So now I have to put data in 6 columns and 411 rows in the database BY HAND...
oh and did I mention they also have relations? yeah... I also have to do that by hand now...3 -
Data wrangling is messy
I'm doing the vegetation maps for the game today, maybe rivers if it all goes smoothly.
I could probably do it by hand, but theres something like 60-70 ecoregions to chart,
each with their own species, both fauna and flora. And each has an elevation range its
found at in real life, so I want to use the heightmap to dictate that. Who has time for that? It's a lot of manual work.
And the night prior I'm thinking "oh this will be easy."
yeah, no.
(Also why does Devrant have to mangle my line breaks? -_-)
Laid out the requirements, how I could go about it, and the more I look the more involved
it gets.
So what I think I'll do is automate it. I already automated some of the map extraction, so
I don't see why I shouldn't just go the distance.
Also it means, later on, when I have access to better, higher resolution geographic data, updating it will be a smoother process. And even though I'm only interested in flora at the moment, theres no reason I can't reuse the same system to extract fauna information.
Of course in-game design there are some things you'll want to fudge. When the players are exploring outside the rockies in a mountainous area, maybe I still want to spawn the occasional mountain lion as a mid-tier enemy, even though our survivor might be outside the cats natural habitat. This could even be the prelude to a task you have to do, go take care of a dangerous
creature outside its normal hunting range. And who knows why it is there? Wild fire? Hunted by something *more* dangerous? Poaching? Maybe a nuke plant exploded and drove all the wildlife from an adjoining region?
who knows.
Having the extraction mostly automated goes a long way to updating those lists down the road.
But for now, flora.
For deciding plants and other features of the terrain what I can do is:
* rewrite pixeltile to take file names as input,
* along with a series of colors as a key (which are put into a SET to check each pixel against)
* input each region, one at a time, as the key, and the heightmap as the source image
* output only the region in the heightmap that corresponds to the ecoregion in the key.
* write a function to extract the palette from the outputted heightmap. (is this really needed?)
* arrange colors on the bottom or side of the image by hand, along with (in text) the elevation in feet for reference.
For automating this entire process I can go one step further:
* Do this entire process with the key colors I already snagged by hand, outputting region IDs as the file names.
* setup selenium
* selenium opens a link related to each elevation-map of a specific biome, and saves the text links
(so I dont have to hand-open them)
* I'll save the species and text by hand (assuming elevation data isn't listed)
* once I have a list of species and other details, to save them to csv, or json, or another format
* I save the list of species as csv or json or another format.
* then selenium opens this list, opens wikipedia for each, one at a time, and searches the text for elevation
* selenium saves out the species name (or an "unknown") for the species, and elevation, to a text file, along with the biome ID, and maybe the elevation code (from the heightmap) as a number or a color (probably a number, simplifies changing the heightmap later on)
Having done all this, I can start to assign species types, specific world tiles. The outputs for each region act as reference.
The only problem with the existing biome map (you can see it below, its ugly) is that it has a lot of "inbetween" colors. Theres a few things I can do here. I can treat those as a "mixing" between regions, dictating the chance of one biome's plants or the other's spawning. This seems a little complicated and dependent on a scraped together standard rather than actual data. So I'm thinking instead what I'll do is I'll implement biome transitions in code, which makes more sense, and decouples it from relying on the underlaying data. also prevents species and terrain from generating in say, towns on the borders of region, where certain plants or terrain features would be unnatural. Part of what makes an ecoregion unique is that geography has lead to relative isolation and evolutionary development of each region (usually thanks to mountains, rivers, and large impassible expanses like deserts).
Maybe I'll stuff it all into a giant bson file or maybe sqlite. Don't know yet.
As an entry level programmer I may not know what I'm doing, and I may be supposed to be looking for a job, but that won't stop me from procrastinating.
Data wrangling is fun.1