Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
Get a devDuck
Rubber duck debugging has never been so cute! Get your favorite coding language devDuckBuy Now
Search - "hadoop"
Fucking HR cunts. I found a nice computer vision vacancy and decided to apply, look what these motherfuckers require :
- Amazon Web Services
- Spring Boot
- Web Sockets
- Grafana - Prometheus
- Java EE
- Graph QL
FUCK OFF FUCK OFF FUCKING FuCK OFF you monkeys. Shove your vacancy up your ass.
Another vacancy, another instance, I'm going to apply at employers site, (they sell some sort of CRM crapshit) forms not working, bc not fucking connected to anything. Just a fucking HTML and JS popup notification for submission laying around with its legs wide spread. Like its copied from w3schools or something.
Looked more on the site it is fucked from all sides like it's made by a school kid.
So I fired an email to the HR, something like "I went to apply but your forms are not working, I found the issue, it's that. Also I found some glory holes in your site, hope your I.T. finds them too. Anyways here is my CV, have a nice day"
Day later I'm shortlisted.35
Every goddamn time.
Boss: Hey, how is the project going?
Me: promising. We have some basic functions working, but at this point it is more like a proof of concept.
Boss: Ah ok, I see.
Couple of days later...
Boss: I talked to a client who was very interested in the stuff you are just building. Made a really good deal! We need to be live by the end of the week.
Went to an interview for the position ‘PHP Web Developer’. Interviewer scans through my CV for 2mins and then starts the interview.
Interviewer: Do you know Java?
Me: I know Java but I don’t have any professional experience
Interviewer: Do you know Hadoop?
Me: No. I’ve never worked on it
Interviewer: Our company works on Hadoop hence you should be able to work on that after joining.
Me: I thought this is a PHP web dev position.
Interviewer: Of course. But you will have work on various other things too!
Me: I don’t think I want to become jack of all trades. Thanks for the opportunity!
I got up and left the interview...7
Sorry, we need someone who knows AWS8
Manual Data Entry: Most boring job
This reminds me of one conversation with one of my faculty..
Faculty: Why not try some Machine Learning Project?
Me: Cool. Any ideas you have already thought
Faculty: Comes up with a really noble idea
Me: Awesome idea. But we need data
Faculty: Don't worry. I will get it. Just help me setup Hadoop (see the irony.. no data yet, and he wants big data setup)
Me: But we don't have data. Let's focus of data collection, Sir
Faculty: I will get it. Don't worry. Trust me.
( I did setup for him twice coz he formatted the system on which I did the setup first time)
After 6 months,
Me: (same question) Sir, Data??
Faculty: I got it.
Me: Great. Give me, I can start looking into it from today.
Faculty: Actually, it's in a register written manually in a different language (which even I can't understand) I will hire data entry guys to convert it into English digital contents.
Road to Manual data entry to Big Data
Dedicating this pencil to the individuals keeping the register up to date and Sir in hopes of converting it into big data..
Long way to go..4
Some companies be like-
.. In job posting - We are the next big thing. We are going to change the industry. We are like Google / Facebook etc...
..in Introduction - We are the next big thing. We are going to change the industry. We are like Google / Facebook etc...
.. in Interviews - We are the next big thing. We are already changing the industry. Think of us like Google / Facebook etc...
.. during Interviews - Our interview process is rigorous because we are the next big thing. We are going to change the industry. We are like Google / Facebook etc...
.. questions in interviews - Since we are Google / Facebook, please answer questions on Java, C/C++, JS, react, angular, data structure, html, css, C#, algorithms, rdbms, nosql, python, golang, pascal, shell, perl...
.. english, french, japanese, arabic, farsi, Sinhalese..
.. analytics, BigData, Hadoop, Spark,
.. HTTP(s), tcp, smpp, networking,.
.. starwars, dark-knight, scarface, someShitMovie..
You must be willing to work anytime. You must have 'no-excuses' attitude
Now in Salary - Oh... well... yeah... see.... that actually depends on your previous package. Stocks will be given after 24 re-births. Joining bonus will be given once you lease your kidneys.
But hey, look... We got free food.
Well, SHOVE THAT FOOD UPTO YOUR ASS.
FUCK YOUR 'COOL aka STUPID PIZZA BEER - CULTURE'.
FUCK YOUR 'FLAT- HIERARCHY'.
FUCK YOUR REVOLUTIONARY-PRODUCT.
My colleague unknowingly uses the word "paralyze" for "parallelize". When is the good time to correctly him? So far he has successfully paralyzed my code.4
There was this 'Hadoop' trend going on. So I thought that i'll start Hadoop. After one month people have started to trend Spark claiming that it is faster. -.-
And now I am outdated.13
Someone at work snuck something past the censors.
Our Hadoop servers all have "bigd" in their name 😂5
This day I have received the most glorious news in e-pistolary form. For some years, I was suffering in support of a client who was, well, insufferable. My presence there paralleled the divine comedy in both essence and fact.
I opened the missive, expecting another plea to bail them out of whatever clusterfuck they found themselves in. Instead, what I found was something truly magical.
I hope this finds you well. I'm not sure if you remember a few years back, we were trying to decide between IBM Cloud and AWS. Well, after years of battling FF*, we're finally moving ahead with AWS. He failed one too many times to deliver anything visibly. After you left, there was no one left he could use to steal credit, ideas, and work.
FF is still pushing to have them use IBM cloud as a "warm backup" in the event "AWS fails." We will see where that goes.
I figured you'd like to know; you were the void in the wilderness for a long time. I don't want to think about how much time we could have saved if we had just listened.
This event represents a personal victory, albeit belated, over a few peoples' absurd amount of privilege. Towards the end, I was vicious about my contestation to the insanity of adopting a desperate hedge attempt-as-cloud offering from a failing company. Some examples:
// cloud 'strategy meeting'
Moi: What cloud platform are we looking at using?
FF: We're looking at IBM cloud and AWS as a second.
Moi: Why is that? I understand you're obligated to rep your offering first, but that decision doesn't seem to have the customer's best interest at heart.
FF: IBM cloud is a market leader; AWS isn't as good.
Moi: I see. I mean, that's the tech equivalent of the company's fleet management considering monkeys on tricycles as a strong competitor to service trucks, but I get what you mean.
// steering meeting
Director: Who can we look to as an example? Who is currently using the IBM cloud?
Moi: No one; they account for a single-digit portion of the actual cloud market. Their long game to sell you a "Hybrid Cloud," which means put some front end payload in a CDN, and buy n-frame units of IBM z servers for the DC with IBM gateway appliances acting as connective tissue. So it's not the cloud at all, really.
Director: How does it compare in cost?
Moi: It's generally 40% more expensive than other clouds, and it only goes higher as you option their software.
Director: What about Watson? I hear Watson is good?
Moi: It's a brand name. Most of the "Watson" product is just a facade on top of FOSS products like Spark, Hadoop, Elasticsearch, etc.
Director: Those were words. They sounded good. FF say it's good tho so we'll believe him because we're from the same city.
Moi: *deletes Director from LinkedIn*
Moral of the story: Never trust a vendor that only recommends their products.
*FF = FatFuck - an embarrassingly rotund individual whose girth is roughly equivalent to his height. He shit his way into an IBM architect position in his mid-20s purely due to winning the visa lottery. He had fake hair glued to his head for his wedding to hide his male pattern baldness; his arrange-married wife undoubtedly cries herself to sleep after sex.
**PeeEm - the then project manager, now portfolio manager of some satellite projects. An overall decent human being, capable.12
Hadoop was born at Yahoo! Startups like Cloudera or Hortonworks that built their entire business around Hadoop are worth more than Yahoo!3
When a client refuses to use cloud services for Hadoop in lieu of 4 local machines... For TB scale data.4
Fax machines connected to VoIP connections...
Had a nightmare recently, where my fathers machine that he really needs refused to work after he moved appartements.
Uncounted calls with different tech departments, a furious fathers and two weeks later they found out, that they forgot to activate the protocol.2
I started learning to use Hadoop recently, and am running a VM with all I need installed on it (the HDF Sandbox to be exact). The VM wants 8GB to run and my laptop only has just that, except it also needs to run Windows at the same time...
At first I thought I was screwed, that I'd need a more capable computer to learn. I gave it a shot anyway, and told VirtualBox to give the VM 4GB, hoping the VM itself would use RAM swap to function. And it did!
What I didn't expect was Windows not slowing down even a bit. Turns out Windows can triple the computer's RAM with virtual memory that it keeps on disk.
So the bottom line is: my VM is using 4GB as if it was 8GB, and at the moment my Windows is using 8GB as if they were 14GB. All of this without breaking a sweat. The more you know!3
At what point do you stop optimizing queries and realize it's a database architecture, scaling problem?
We've been having production issues this week because a lot more users with more demands, and I'm going we need more servers... We can't just have one db, we need to parallelize like Hadoop...
Everyone else is going, how do we optimize queries, indexes, reduce the load...11
One of my colleagues who thinks he knows all about "big data": "I try to put everything in hadoop, that is my philosophy".
We don't even have a hadoop cluster.
Back in my college, my professor was trying to demonstrate how to install hadoop using my laptop.
Now, he asked me to type my password
Even I thought it was asking my password. So I started typing it but my password was my crush's name (aakank....).
I typed till aaka and then realized what happened.
Fortunately I was able to handle the situation by typing
"Aakanakabaka" and saved my self.1
I decided to start learning about hadoop... I found a great, free tutorial, but it expects me to know python. ...well it looks like I'm learning two things at once.2
Some people are so crazy about Hadoop, that for every problem, they think of solving it using Hadoop. Even when you need to process a 10MB file!!!!2
"Bro, what's inside your bag?"
"Just a MacBook Air, why?"
"Looks so heavy?"
"There is Hadoop inside my Mac Air."
Set-up a 5 node EC2 cluster in AWS; Install my dependencies on all; Add private keys between all for handshake; Submit my Hadoop job for processing; AWS closes my instance within 10 minutes of starting a job that took me almost half an hour to set-up because the master node's spot costs have reached more than 15 USD :(2
Setting up a single node Hadoop cluster. Then installing intellij idea, to find that it doesn't detect any installed jdk. Then uninstalled all jdks, and then reinstalled one, then Hadoop won't work. Now everything seems fine.
Took a day off and came back to find my team and I were moved from frontend to backend (which I always wanted to be).
Manager says we get to learn Java, Mongodb and even Hadoop.
I am so so excited.
But the only little hold back is, there is limited support for Java in sublime text and I should instead use eclipse. I had dozens of shortcuts customised with all those beautiful themes in sublime, but eclipse takes the fun out of coding(no offense to eclipse lovers).5
I have heard about Apache Hadoop for a while but never bothered to learn anything about it.
What can it be used for? Can I use it for hosting thousands of websites powered by php/hhvm?
I am starting to have a need for a really HA and High Performance solution that is futureproof too. My current solution is doing great also when it comes to performance and HA, but it is always nice to try out new things...
So the question is, can Hadoop be used in a hostingsolution?4
When you and your team invested a lot of their weekend time into the project to meet the clients unrealistic deadline and get not even a simple "thank you" from management...3
I really regret switching to manjaro. So many things keep breaking, like my laptop won't sleep anymore, it stays up, whenever I plug in another display I get an error thrown at me. Among other weird behaviors (all screen related) that I can't seem to fix and make the experience feel like I'm running a very clunky win-poop machine.
On the other hand, setting up a very custom sddm theme and installing certain software like hadoop, rust, gimp, xfce tweaks and other things was such a breeze D: just "yay hadoop" and 90% of the work was done.
Grhhh... Wondering if I should accept defeat, and maybe switch to Linux MX or spend hours fixing what probably is a display driver issue that's pissing me off 😠2
We had a tutorial on how to use spark/Hadoop.. part of the tutorial was the installation instructions for Ubuntu vms.
The Prof insisted we used an older version of Hadoop (v2.1.5), so naturally this required pulling older repos and older versions of java.
Naturally, some of the people in the class got some namespacing issues and garbage left from uninstalled packages.
Now, the tutorial was geared towards business/math people, not com sci. So most of the people didn't understand why apt didn't let them run certain commands (even though it very clearly just asked for them to run autoremove or autoclean, like in the "error message"). When the Prof and their "experienced TA" saw these messages, their recommendation was "make a new fresh vm".
The fuck? I heard that, run over and was like no. Just run the suggested command, it's literally a simple issue. And the guy didn't believe me. I had to sit him down, show him how I literally typed what the console was asking for, and everything just worked... The guy's response was "well that's Linux for you, its really complicated and can never trust anything, this time it worked, but next time it might not". Dude... Do you even know what you are saying? Like you are a supposed expert, least have some understanding of the package manager you are using. Maybe things will then be less "schrodinger cat".
God damn I can't wait to be out of this stupid fucking school. Never going back to academia.1
A friend went to a financial corp for a data analytic position. The interviewer proudly told him that they just got a new toy, and it's called Hadoop. My friend almost fainted.2
Sometimes I think My company thinks like Hadoop ,
Instead of getting a experienced guy like 10 years exp . Let's take 5 with 2 years exp which should be more efficient !! 😒1
I eagerly wait for the day some people will realize and believe that Hadoop is a file system and NOT a database!!1
How many languages does one have to learn...? Learnt C, C++ and Java because of college courses. Learnt HTML, CSS, and vanilla JS because I wanted to learn frontend. Now learning R for big data analytics. Today, I came to know that I need to learn more Java or start learning Python for Hadoop...!!
When your boss is hell bent on shoving the the words Hadoop , architecture and Revamp in every 3rd sentence. 🙄4
Should i do hadoop big data course ? I am thinking of this summer to do on simplilearn. I am third year student undergraduate in IT. I am java guy and good in RDBMS.. Should i learn then?2
Didn't know how to program but enrolled in a master's degree that needed that. So I attended the programming courses of the bachelor's degree that gave me just enough knowledge and just in time to save my ass writing a Hadoop MapReduce program! And I actually did well. All Java though.
These days I am banging my head against the wall learning Rust.1
Today my project manager called Hadoop a data warehouse and a Big Data lake in a meeting. I couldn't decide whether to laugh my ass off or spend the next 30 mins explaining to her what Hadoop actually is.2
When your training system has 8 gigs of ram but you have to run HDFS, 3 PuTTY terminals, Eclipse, Firefox, a few explorer windows and sublime, it definitely feels like a tech Hurt locker. Every mouse click can fuck you up.
I was a frontend developer, and I am new to hadoop or anything related to big data.
I am currently working as a Hadoop developer and I get to work on one of existing codebase also I am trying to recollect Java which I learnt during college.
Can u please provide me any inputs on how to get started with Hadoop, a personal view point on scope and future of Hadoop. A rough time span of how long it took for you to get out of the noob zone.
If you could provide me with a good tutorial or blog that would be awesome.
Thanks in advance1
A project on hive....
Well the server has some error....
So i need to install hive on windows....
For that i need hadoop on windows...
For that i need either vmware or cygwin...
What was my project again??
Why, oh why do we keep using such bizarre tech names e.g Hadoop, Pig, Oozie, Sqoop, Bleeg, Flume...5
I ain't getting any summer internship so thinking to do a good course on big data and hadoop. Can't find the free proper source for beginners😕! Any suggestions? Whats your plan btw..m thinking to dive into web dev as well.2
Windows is a shameful dev enviro but when you stuck in Africa, you gotto work with what you have. I dabble in node, R and hadoop and setting up environment and building modules on windows is a walk on hot ash in a desert.
To go around setting up of different dev enviros, i use a windows pre installed hack specially meant for that purpose. A new user account for each dev job. Kips my machine clean and sane while avoiding the blue screen.
After all, who still shares laptops today enough to use different user accounts😂😂1
There was this faculty who had a masters degree in big data, during my graduation.
She asked the whole class to install 'hadoop' on their machines as an assignment; in a situation where most of the class didn't even properly knew what Linux was.
I installed it and showed it to her.
She: Shreyans, can you help me install it on my system, I'm getting some errors.
Me: Sure ma'am.
She: On what Linux did you install it?
Me: Linux Mint ma'am.
She: But Mint's setup won't work for my Ubuntu. Do you have a setup for my Ubuntu?
And I stood there frozen thinking what to reply..
#facepalm #facechair #facetable
Debugging Spark errors is frustrating. Been running a model which takes an hour only to come across errors. And it's not even related to my code. Something to do with the cluster. FML1