buzzword translations:

"cloud" -> someones computer

"big data" -> lots of somewhat irrelevant data

"ai" -> if if if if if if if if if if if if if else

"algorithm" -> something that works but you don't know why

"secure" -> https://

"cyber security" -> kali linux + black hoodie

"innovation" -> adding something completely irrelevant such as making a poop emoji talk

"blockchain" -> we make lots of backups

"privacy" -> we store your data, we just don't tell you about it

Big data is like teenage sex

Solving crashes via *big data*

Ummm, thanks but no...

Buzzword dictionary to deal with annoying clients:

AI—regression
Big data—data
Blockchain—database
Algorithm—automated decision-making
Cloud—Internet
Crypto—cryptocurrency
Dark web—Onion service
Data science—statistics done by nonstatisticians
Disruption—competition
Viral—popular
IoT—malware-ready device

BIG DATA reality..

Here's a list of unpopular stuff which I agree with:

1) I love Java more than any other programming language.

2) I love sleeping more than working.

3) I'm not a night owl. I thrive the most during daylight.

4) I don't like or need coffee. Tea is fine.

5) Webdev is a huge clusterfuck which I secretly wish that could just die already.

6) Cybersecurity is a meme and actually not that interesting. Same passes for Cloud, Machine Learning and Big Data.

7) Although I'm a huge fan of it Linux is too unstable and non-idiot proof to ever become mainstream on the desktop.

8) Windows is actually a pretty solid OS.

9) The real reason I don't use macos is because I'm a poorfag that can't afford an overpriced laptop.

10) I don't like math and I hate that people push math shit into random interview questions for dev jobs which have nothing to do with math.

Post yours.

I miss old internet.
- without politics
- without robots
- without money
- without big portals
- without commercials
- without advertising
- without data centers
- without ipv6
but with great usenet and community

Shit fuck I’m old

Story time:

I was once working on a project that dealt with incredibly sensitive financial data.

We needed a client’s database to do a migration.

They wouldn’t send it over the internet because it was too big and they didn’t think it would be secure.

They opt to send it in the post on an encrypted usb drive.

(Fair enough thinks I)

USB drive arrives.
Is indeed encrypted.

MFW there’s a post it note in the envelope with the password on.

MFW this is a billion dollar multinational petrochem company.

MFW this same company’s ‘sysadmin’ and ‘dba’ once complained because a SQL script I sent them didn’t work - they’d pasted it twice and couldn’t work this out from the fucking “table already created” error message management studio was throwing at them.

Big client, sells products in 30+ countries.

Tries to generate newsletter subscribers, so asks for a system to send a coupon upon subscription.

2 days later, client calls in panic. “We have too many subscribers, our marketeers say it’s a bot issue, can you do something?”.

Checked the data, checked analytics, turns out there’s a lot of referral traffic from freebie-sites, no sign from a bot issue.

Called the client back, “sorry, but there’s nothing you can do about that, you wanted your newsletter to become popular. Not you’re the victim of your own success”.

Client: “can we add captcha?”

Me: “why would you want to do that? You don’t even have a bot issue”

Client: “to make it harder for people to subscribe”

I tried to talk som sense in their heads, but after 3 times I gave up and implemented the damn captcha. It’s still there, doing nothing but annoying thousands of people, including me...

Where are the BIG data guys ?

I'd like to extend my heartfelt fuck-you to the following persons:

- The recruiter who told me that at my age I wouldn't find a job anymore: FUCK YOU, I'll send you my 55 birthday's cake candles, you can put all of them in your ass, with light on.

- The Project Manager that after 5 rounds of interviews and technical tests told me I didn't have enough experience for his project: be fucked in an Agile way by all member of your team, standing up, every morning for 15 minutes, and every 2 weeks by all stakeholders.

- The unemployment officer who advised me to take low level jobs, cut my expenses and salary expectations: you can cut your cock and suck it, so you'll stop telling bullshit to people

- The moron that gave me a monster technical assignment on Big Data, which I delivered, and didn't gave me any feedback: shove all your BIG DATA in your ass and open it to external integrations

- the architect who told me I should open my horizons, because I didn't like React: put a reactive mix in your ass and close it, so your shit will explode in your mouth

- the countless recruiter who used my cv to increase their db, offering fake jobs: print all your db on paper and stuff your ass with that, you'll see how big you will be

To all of them, really really fuck you.

Data Manipulation...

Source: Analytics Vidhya

*Interview*

Interviewer: We have an opening. Are you interested to work?

Me: What is that I'll be doing?

I: What technologies and languages do you know?

Me: I know Scala, Java, Spark, Angular, Typescript, blah blah. What is your tech stack?

I: Any experience working on frontend?

Me: Yes. But what do you use for it?

I: Can you work with databases?

Me: I can, on SQL based. What are yours?

I: Can you do big data processing?

Me: I know Spark, if that's what you are asking for. What is it that you actually do?

I: Any experience in cloud development?

Me: Yes. AWS? Azure? GCP?

I: Do you know CI CD?

Me: Excuse me.. I've been asking a lot of questions but you're not paying attention to what I'm asking. Can you please answer the questions I asked.

I: Yes. Go ahead.

Me: What will be my position?

I: A full stack developer.

Me: What technologies do you use in your project?

I: We use all the latest tech.

Me: Like?

I: All latest tech.

Me: You mentioned big data processing?

I: Yes. Processing data from DB and generating reports.

Me: what do you use for that?

I: Java.

Me: Are you planning to rebuild it using Spark or something and deploy in the cloud?

I: No we're not rebuilding it. Just some additions to the existing.

Me: Then what's with cloud? Why did you ask for that?

I: Just to know if you're familiar.

Me: So I'll be working with Java. Okay. What do you use for UI?

I: Flash

Me: 🙄

I sat for a couple of minutes contemplating life.

I: Are you willing to join?

Me: No. Not at all. Thankyou for the offer.

This can annoy the hell out of me. When people ask me if they can have your Facebook or whatsapp or something and I'm like 'sorry I don't have that' and they ask why and you explain because privacy reasons and they go like 'oh you're a little paranoid are ya?'.

There's a motherfucking big difference between wanting control over your data as much as possible and being paranoid.

Fucking hell.

I guess that is what you get for bringing up security issues on someones website.

Not like I could read, edit or delete customer or company data...

I mean what the shit... all I did was try to help and gives me THIS? I even offered to help... maybe he got angry cause I kind of threw it in his face that the whole fucking system is shit and that you can create admin accounts with ease. No it's not a framework or anything, just one big php file with GET parameters as distinction which function he should use. One fucking file where everything goes into.

Me: you should not open that log file in excel its almost 700mb
Client: its okay, my computer has 4gb ram
Me: *looking at clients computer crashing*
Client: the file is broken!
Me: no, you just need to use a more memory efficient tool, like R, SAS, python, C#, or like anything else!

If Big O notations where emojis.  This chart shows you common big-Os with emoji showing how they'll make you feel as your data scales. Source blog.honeybadger.io

Was programming on the privacy site REST api.

Needed a break and started searching for a good movie or documentary.

Found a documentary about big data/mass surveillance.

I now have loads of motivation for programming on this again as this showed me the importance of secure services/software.

A few years ago:

In the process of transferring MySQL data to a new disk, I accidentally rm'ed the actual MySQL directory, instead of the symlink that I had previously set up for it.

My guts felt like dropping through to the floor.

In a panic, I asked my colleague: "What did those databases contain?"

C: "Raw data of load tests that were made last week."

Me: "Oh.. does that mean that they aren't needed anymore?"

C: "They already got the results, but might need to refer to the raw data later... why?"

Me: "Uh, I accidentally deleted all the MySQL files... I'm in Big Trouble, aren't I?"

C: "Hmm... with any luck, they might forget that the data even exists. I got your back on this one, just in case."

Luck was indeed on my side, as nobody ever asked about the data again.

Manual Data Entry: Most boring job
This reminds me of one conversation with one of my faculty..
Faculty: Why not try some Machine Learning Project?
Me: Cool. Any ideas you have already thought
Faculty: Comes up with a really noble idea
Me: Awesome idea. But we need data
Faculty: Don't worry. I will get it. Just help me setup Hadoop (see the irony.. no data yet, and he wants big data setup)
Me: But we don't have data. Let's focus of data collection, Sir
Faculty: I will get it. Don't worry. Trust me.
( I did setup for him twice coz he formatted the system on which I did the setup first time)
After 6 months,
Me: (same question) Sir, Data??
Faculty: I got it.
Me: Great. Give me, I can start looking into it from today.
Faculty: Actually, it's in a register written manually in a different language (which even I can't understand) I will hire data entry guys to convert it into English digital contents.
Me: *facepalm*
Road to Manual data entry to Big Data
Dedicating this pencil to the individuals keeping the register up to date and Sir in hopes of converting it into big data..
Long way to go..

Just wasted 2 hours. What an idiot I am.

Thanks to my friend.

Some companies be like-

.. In job posting - We are the next big thing. We are going to change the industry. We are like Google / Facebook etc...

..in Introduction - We are the next big thing. We are going to change the industry. We are like Google / Facebook etc...

.. in Interviews - We are the next big thing. We are already changing the industry. Think of us like Google / Facebook etc...

.. during Interviews - Our interview process is rigorous because we are the next big thing. We are going to change the industry. We are like Google / Facebook etc...

.. questions in interviews - Since we are Google / Facebook, please answer questions on Java, C/C++, JS, react, angular, data structure, html, css, C#, algorithms, rdbms, nosql, python, golang, pascal, shell, perl...
.. english, french, japanese, arabic, farsi, Sinhalese..
.. analytics, BigData, Hadoop, Spark,
.. HTTP(s), tcp, smpp, networking,.

..

..

..

.. starwars, dark-knight, scarface, someShitMovie..

You must be willing to work anytime. You must have 'no-excuses' attitude

.........................................

Now in Salary - Oh... well... yeah... see.... that actually depends on your previous package. Stocks will be given after 24 re-births. Joining bonus will be given once you lease your kidneys.

But hey, look... We got free food.

Well, SHOVE THAT FOOD UPTO YOUR ASS.

FUCK YOU...
FUCK YOUR 'COOL aka STUPID PIZZA BEER - CULTURE'.
FUCK YOUR 'FLAT- HIERARCHY'.
FUCK YOUR REVOLUTIONARY-PRODUCT.

FUCK YOU!

Just wanted to say a 'thank you' to all people who bear with my privacy stuffs! I know quite some people who installed messaging apps, signed up to privacy services and so on, solely because --> I <-- want to communicate in private and I realise (I've always realized that though) that that can be tough sometimes.

Also a thank you to those people for not requiring me to get data fed into the big companies :).

Thanks!

My client is trying to force me to sign an ethics agreement that would allow them to sue me if found in breach of it. At the same time they are scraping eBay's data without their consent and refuse to sign the licence agreement. Apparently they don't understand irony.

Paypal Rant #2

Paypal might just be the only company with 98% of their employees being support staff because not a soul on this planet knows how to work with that fucking piece of shit of a company's service.

No really, if there was a shittiness-rating from 1-10 (10 = worst) you would have to store paypal's rating as a string or invent a new data type because no CPU could fucking work with such a big ass fucking number.

If I had to choose between Paypal and going back to manually trading physical goods/animals for stuff I would gladly choose the latter, because Paypal, go suck a bag of dicks you useless fucking shitpile of a "company".

Mum: Is this the big data?

Father: Do you know anything about Bitcoin? Can you explain me what it is?

National Health Service (nhs) in the UK got hacked today... Workers at the hospitals could not access patient and appointment related data... How big a cheapskate you gotta be to hack a free public health service that is almost dying for fund shortages anyway...

When I was 10 my younger brother saved over my fully completed pokedex in Pokemon blue.

First big data loss taught me a good life lesson. Now I backup everything on a local server.

Overheard some guy talking about robotics on the phone, turns out it was all about MS excel macros.

people need to stop abusing terms like big data, AI etc. to make them sound 'smart' 🙄

I've dreamed of learning business intelligence and handling big data.

So I went to an university info event today for "MAS Data Science".

Everything's sounded great. Finding insights out of complex datasets, check! Great possibilities and salery.

Yay! 😀

Only after an hour they've explained that the main focus of this course is on leading a library, museum or an archive. 😟 huh why? WTF?

Turns out, they've relabled their librarian education course to Data science for getting peoples attention.

Hey you cocksuckers! I want my 2 hours of wasted life time back!
Fuck this "english title whitewashing"

I always thought that this could only happen with big orgs with precious data. One of my coworkers sent me this last night

"Big data" and "machine learning" are such big buzz words. Employers be like "we want this! Can you use this?" but they give you shitty, ancient PC's and messy MESSY data. Oh? You want to know why it's taken me five weeks to clean data and run ML algorithms? Have you seen how bad your data is? Are you aware of the lack of standardisation? DO YOU KNOW HOW MANY PEOPLE HAVE MISSPELLED "information"?!!! I DIDN'T EVEN KNOW THERE WERE MORE THAN 15 WAYS OF MISSPELLING IT!!! I HAD TO MAKE MY OWN GODDAMN DICTIONARY!!! YOU EVER FELT THE PAIN OF TRAINING A CLASSIFIER FOR 4 DAYS STRAIGHT THEN YOUR GODDAMN DEVICE CRASHES LOSING ALL YOUR TRAINED MODELS?!!
*cries*

Developer vs Tester

(Spoiler alert: developer wins)

My last developent was quite big and is now in our system testing department. So last week i got every 20 minutes a call from the tester, that something did not work as expected. For about 90% of the time i looked at the testing setup or the logs and told him, that the data is wrong or he used the tool wrong. After a couple of days i got mad because of his frequent interruptions. So I decided to make a list. Every time he came to me with an "error" i checked it and made a line for "User Error" or "Programming Error". He did not liked that much, because the User Error collum startet to grow fast:

User Errors: ||||| |||
Programming Errors: |||

Now he checks his testing data and the logs 3 times before he calls me and he hardly finds any "errors" anymore.

What kind of cum gargling gerbil shelfer stores and transmits user passwords in plain text, as well as displays them in the clear, Everywhere!

This, alongside other numerous punishable by death, basic data and user handling flaws clearly indicate this fucking simpleton who is "more certified than you" clearly doesn't give a flying fuck about any kind of best practice that if the extra time was taken to implement, might not totally annihilate the company in lawsuits when several big companies gang up to shower rape us with lawsuits over data breaches.

Even better than that is the login fields don't even differentiate between uppercase or lowercase, I mean WHAT THE ACTUAL FUCK DO YOU SELF RIGHTEOUS IGNORANT CUNTS THINK IS GOING TO HAPPEN IN THIS SCENARIO?

Whoever implemented the data import in Numbers on Mac needs to be lined up against a wall and shot with needles until they wish they were dead.

Why on all of gods unholy green and shitty earth would i want data i import (EVEN IN CSV FOR FUCK SAKE) to be delimited by an arbitrary text width? WHAT THE ACTUAL FUCK

WHY WHY why would I EVER want to delimit my carefully structured data by fucking text width instead of new line or comma? AAAAARRRHHH

And what fucking big brain genius made this the DEFAULT SETTING for imported text AND CSV FILES. IT STANDS FOR COMMA SEPARATED FILE YOU FUCK BOI MAYBE JUST MAYBE I WANT IT SEPARATED BY FUCKING COMMMMMMMAAAAASSSSSS

I absolutely hate the way we are taught programming in Indian colleges.

FML #1: I'm pursuing a UG CS course, and this semester, I only had one subject of Computers, that too only 1 credit. The rest with all electronics.

FML #2: In that 1 credit course, we had to make a C++ project which had "data handling". No one cares if you build something cool or not, just that a project should have "extensive use" of data handling.

FML #3: Source code had to be >= 1000 lines. This is the only place where ADDING MORE LINES OF CODES THAN REDUCING IT is appreciated. Had to stuff my code with all kinds of comments and violating the basic principle of DRY.

So, yeah, we're fucked big time. 😥

FUCK PHP!!!
We were trying to go live with a big online shop, it is connected through a crappy API to SAP.
PHP keeps outputting fucking errors because our intern doesn't fucking know how to properly write PHP.

YEAH, JUST FUCKING RETURN A WHOLE DIFFERENT TYPE OF DATA IF THE FUNCTION HAS AN ERROR.

Oh and using fucking strtr( ... ) to insert stuff in a string is REALLY FUCKING 1337...

And when you think the whole fuckery has reached the summit, just look at how HE FUCKING CREATED THE UGLY EMAIL TEMPLATE:
$content .= "UGLY HTML ABOMINATION";
$content .= "MORE UGLY HTML";
$content .= "HTML WITH SPELLING ERRORS";
$content .= "<table>";
$content .= "TEARS OF TIM BERNERS LEE";
$content .= "<table>HE FOGOT THE FUCKING '/'";

and dozens more of these lines...

and the whole piece for ALL 3 FUCKING LANGUAGES...

Thanks for writing the fucking backend stuff, it is better to rewrite the whole piece.

GOT AN A+ FOR MY LAST PROJECT OF HIGH SCHOOL!!! SO FUCKING HAPPY!!!

(by the way, we built a search engine for this project. A pretty big and fast one too)

From a design meeting yesterday:

MyBoss: "The estimate hours seem low for a project of this size. Is everything accounted for?"
WebDev1: "Yes, we feel everything for the web site is accounted for."
-- ding ding...my spidey sense goes off
Me: "What about merchandising?"
MerchDevMgr: "Our estimate pushed the hours over what the stakeholders wanted to spend. Web department nixed it to get the proposal approved."
MyBoss: "WTF!? How the hell can this project go anywhere without merchandising entering the data!?"
WebDev2: "Its fine. We'll just get the data from merchandising and enter it by hand. It will only be temporary"
Me: "Temporary for who? Are you expecting developers to validate and maintain data?"
WebDev1: "It won't be a big deal."
MyBoss: "Yes it is! When the data is wrong, who are they going to blame!?"
WebDev1: "Oh, we didn't really think of that."
MerchDevMgr: "I did, but the CEO really wants this project completed, but the Web VPs would only accept half the hours estimated."
Me: "Then you don't do it. Period. Its better to do it right the first time than half-ass. How do think the CEO will react to finding out developers are responsible for the data entry?"
MerchDevMgr: "He would be pissed."
MyBoss: "I'm not signing off on this design. You can proceed without my approval., but I'll make a note on the document as to why. If you talk to Eric and Tom about the long term implications, they'll listen. At the end of the day, the MerchVPs are responsible to the CEO."
WebDev1: "OK, great. Now, the database, it should be SQLServer ..."

I checked out after that...daydreamed I was a viking.

You've heard it!! To become a Web 3.0 daveluper, you have to do Blockchain programming, and you get additional investor points when you do artificial intelligence, Big Data and IoT 😝

True story.
Some clients (especially in India) don't want to pay, but they want everything to be implemented in the project.

Big data.... Check
Machine learning.... Check
Deep learning..... Check
Espresso maker.... Check.

They want all the buzz words that are buzzing to be put in your project and they want you to put it in the 'cloud', for which you have to pay.....

I just got four CSV reports sent to me by our audit team, one of them zipped because it was too large to attach to email.

I open the three smaller ones and it turns out they copied all the (comma separated) data into the first column of an Excel document.

It gets better.

I unzip the "big" one. It's just a shortcut to the report, on a network share I don't have access to.

They zipped a shortcut.

Sigh. This'll be a fun exchange.

Creating an anonymous analytics system for the security blog and privacy site together with @plusgut!

It's fun to see a very simple API come alive with querying some data :D.

Big thanks to @plusgut for doing the frontend/graphs side on this one!

A company gave a placement talk in college today.

First, they talked about their company's facts and figures, which no one was interested in.

Second, they talked about Amazon and Jeff's vision, AirBnB and their revolutionary idea, more than their own company and products.

Third, they showed some testimonial videos of their employees and customers.

"What the fuck is going on?" I thought. We were there to get information about a placement test.

Buzzwords started coming in. Machine Learning, Artificial Intelligence, Big Data and what not.

Last 15 minutes, a guy came. He talked about test date, test format and test topics, finally.

An hour and half wasted for 15 minutes of information.

Fuck placement talks.

Everybody talking about Machine Learning like everybody talked about Cloud Computing and Big Data in 2013.

When there are no widely approved Swedish translations of big data terminology, such as "big data" itself. When discussing this kind of terms you have to resort to using the English words for them, which results in a horrible language mix, Swenglish.

An AI to work with big data, made with blockchain

My co-workers hate it when I ask this question on a technical interview, but my common one is "what is the difference between a varchar(max) and varchar(8000) when they are both storing 8000 characters"

Answer, you cannot index a varchar(max). A varchar(max) and varchar(8000) both store the data in the table but a max will go to blob storage if it is greater than 8000.

No one ever knows the answer but I like to ask it to see how people think. Then I tell them that no one ever gets that right and it isn't a big deal that they don't know it, as I give them the answer.

Spend 14 hours a week studying more with my free time.

Things to be studied:

-discrete math
-data structures
-algorithms
-coding challenges
-problem defining
-abstraction
-other relevant maths

Other things I want to improve:

-confidence at work
-reaching out to teams with questions
-social skills
-time management
-enjoying the little things
-patience
-consistency (with everything above)

Last big thing would be being more conscious with what type of data/platforms I am digesting everyday. Just like a good diet I want to get in the habit of consuming “good” useful content that’s thought provoking or knowable rather than fast food social media carbs

Wish everyone a productive New Year!

We are building this big-data engine for a client's product for which we were using a cluster on GCP and they were billed ~1100$ for the last month's usage.
The CTO - the CHIEFFUCKING TECHNOLOGY OFFICER told us to hook up 5-6 laptops in our server room and create our own cluster because they cannot afford so much bill.

Client: we need a big data implementation in AWS to be fully HA and DR.... Money is no object

*3 weeks later when the bill comes in *

Client: its too expensive we don't need this HA stuff we don't even know what it stands for anyhow so can you take it out? But the system still needs 24/7 availability....

Coworker: since the last data update this query kinda returns 108k records, so we gotta optimize it.
Me: The api must return a massive json by now.
C: Yeah we gotta overhaul that api.
Me: How big do you think that json response is? I'd say 300Kb
C: I guess 1.2Mb
C: *downloads json response*
Filesize: 298Kb
Me: Hell yeah!
PM: Now start giving estimates this accurate!
Me: 😅😂

I promise! Just because you can add ANN (Artificial Neural Network) doesn't mean you should!

Context:
PM is not an IT professional but somehow leads IT operations ... (yes... I know)
---

PM: "Hey xxzero0, do you remember about the XYZ project?"

xxzero0: "Yes, tell me"

PM: "I told the big boss we can use it to make starships and explore the universe, I also said we can cut the developing time because we are already at 70% with it".

xxzero0: "....... Do you understand we planned to use this project to deploy a small ship in the sea?"

PM: "Yes, but you clearly inexperienced developer, don't know it needs only some refactoring to explore the universe"

xxzero0: "It is more complicated. There is no logic at all. It is just displaying data without doing anything and..."

*Get interrupted*

PM: "Yes, we need some refactoring, I'm such a genius."

Currently, a classmate and I are working on our technical thesis.
It is all about industry 4.0, IIoT, big data and stuff.
This week, we presented our interim results to our supervisor. He is very pleased with our work and made the following suggestion:
He thinks it would be awesome to publish our work on our own GitHub repository and make it open source because he is convinced that this thesis is able to kind of "set a new standard" in some specific fields of using big data analysis in production processes.

I guess I'm kind of proud :)

User: Hey, we got a big issue with one of your tools. One of your pages isn't loading.
Me: Ok, so when did this happen?
User: We don't know? Its been like that for a long time though, so we thought it was normal 😃
Me: ....ok. So do you know what data is supposed to appear?
User: Uhhh we're not sure as well. Since, you know, its been like that for a while.

Just great 😑

Remove all the outdated and unwanted topics which were taught during Indus Valley civilization like: 8080 microprocessor, Java 6, Software Testing principles etc. And add more interesting and realistic topics like: Algorithm design, graphs and other data structures, Java 8 (at least for now), big data, Basics about AI, etc.

I used to do audits for private companies with a team. Most of them where black box audits and we were allowed to physically manipulate certain machines in and around the building, as long as we could get to them unnoticed.

Usually when doing such jobs, you get a contract signed by the CEO or the head of security stating that if you're caught, and your actions were within the scope of the audit, no legal action will be taken against you.

There was this one time a company hired us to test their badge system, and our main objective was to scrape the data on the smartcards with a skimmer on the scanner at the front of the building.

It's easy to get to as it's outside and almost everyone has to scan their card there in order to enter the building. They used ISO 7816 cards so we didn't even really need specified tools or hardware.

Now, we get assigned this task. Seems easy enough. We receive the "Stay-out-of-jail"-contract signed by the CEO for Company xyz. We head to the address stated on the contract, place the skimmer etc etc all good.

One of our team gets caught fetching the data from the skimmer a week later (it had to be physically removed). Turns out: wrong Building, wrong company. This was a kind of "building park" (don't really know how to say it in English) where all the buildings looked very similar. The only difference between them was the streetnumber, painted on them in big. They gave us the wrong address.

I still have nightmares about this from time to time. In the end, because the collected data was never used and we could somewhat justify our actions because we had that contract and we had the calls and mails with the CEO of xyz. It never came to a lawsuit. We were, and still are pretty sure though that the CEO of xyz himself was very interesed in the data of that other company and sent us out to the wrong building on purpose.
I don't really know what his plan after that would have been though. We don't just give the data to anyone. We show them how they can protect it better and then we erase everything. They don't actually get to see the data.

I quit doing audits some time ago. It's very stressful and I felt like I either had no spare time at all (when having an active assignment) or had nothing but spare time (when not on an assignment). The pay also wasn't that great.

But some people just really are polished turds.

Warning:: Buzzword alarm!
I was doing smalltalk earlier and this guy asks me what I do for a living. me: I work for a company that builds software that is, among other things, able of giving your company prognosises how busy it'll be, so they can plan ahead. Him: ohhh, so like Big Data?
Me: *dies a little inside*... yeah, that's right! (didn't feel like having the talk with him at that point). No dude, not everything is big data 😥

Hey guys!
Been away for some time. Didn't have anything to tell tbh - until today.

As you may remember I started a new apprenticeship at big (and super cool) company. We develop software for various platforms.
I'm currently working on a plugin displaying some weatherdata. This plugin is supposed to be in 2 of our products. One is pretty nice but a pain in the ass to compile. The second one is a pure WTF. It's a web app with approximately half the functionality but 5x loading time (without demo data!) of the other product.
Well, today I learned why we have this loading time. They use I don't know how much different js frameworks and libraries for every single simple task. "It's easy and Facebook does it too. Besides that why are there frameworks if you could do it native?" HMMM LET ME GUESS BECAUSE SOME PEOPLE ARE FUCKING LAZY!?
Why do you need MULTIPLE frameworks for a simple web display!?

Me - Yeah great so you say it's big data we are gonna be analyzing and having to store, are you currently utilizing a service and aggregating any of it into smaller manageable segments?

Client - well yeah it's lots and lots of data, we can share it with you if you sign a nda.

Me - ok... sure, how are you gonna share it with me.

Client - oh I can email you the spreadsheet.

Me - .... Spreadsheet ... Um... Ok... 'Stands up and walks away to tell this as the most interesting meeting of the month, to some one that will get it'

--

Buzz word for the win!

This might actually be my first real rant.

Whatever fucking cockgoblin decided that making dynamics GP so fucking confusing needs to suck a big bag of dicks. I'm so fucking tired of having to google every damned table name and column name because nothing makes any motherfucking sense.

Am I supposed to instinctively know what PM20201 does? What data it holds? I don't mind reading documentation. But it's hard to even know where to start when the shitbird API and database are more complicated than calculating orbital fucking decay.

I am done. Fuck you gp. Fuck you and your nonsense. I guess our sales people don't get to know when an invoice was paid.

One thing I've noticed about devRant is the ratio of web dev/mobile dev posts to database/architecture/big data dev posts. There's A LOT of you web peeps out there, and not enough data dudes, which I guess justifies my constant demand, salary and lack of competition. Just an observation.

Everytime people talk about big data/ operations research

Oh gosh.. i can finally understand the CV and application nightmare stories... We're getting new people in, and there are quite a few interesting ones.

0) pages of randomly placed info. PAGES. I'm lost in there!!

1) no basic info whatsoever. Like, no nationality(we're recruiting internationally), no birthdate, barely his name and email. I know that the first ones are not really needed for the job, but they're still customary.

2) entry level back and/or frontend job. This guy's a phd graduate, working research with big data in a bio-something department. We're a web startup.

3) there are some listing so much unrelevant stuff, I'm not even sure if they meant to apply to us.

4) (my favourite) email subject: application, email body: empty, attached: short_application.doc ("hi, this is an application to the posted job. Best regards, Name") WAIT WHAT?

Today in IT class our teacher said: 'you can only use char and int in a switch statement'

I was confused because I was 100% sure that you could also use string and so when I got out of school I immediately looked it up.

It is true, well it was true until 2011 when Java 7 was released which added the possibility to also use the string data type in switch statements.

In this I see a huge problem with the education system. Teachers (almost) never 'update' their knowledge and then teach outdated stuff to their pupils. While this may not be a problem in some subjects, it definitely is a huge problem in IT.

The development world is always evolving but if the teachers don't follow along the pupils get taught outdated stuff which, in my opinion, is a really big problem when they finish school and go out in the world to find jobs.

Best exp:
( ͡ ͡° ͜ ʖ ͡ ͡°)
\╭☞ \╭☞ learning python and working with big data
Worst one:
(╯°□°）╯︵ learning php and visiting classes of programming at my college

fuck code.org.

here are a few things that my teacher said last class.

"public keys are used because they are computationally hard to crack"

"when you connect to a website, your credit card number is encrypted with the public key"

"digital certificates contain all the keys"

"imagine you have a clock with x numbers on it. now, wrap a rope with the length of y around the clock until you run out of rope. where the rope runs out is x mod y"

bonus:

"crack the code" is a legitimate vocabulary words

we had to learn modulus in an extremely weird way before she told the class that is was just the remainder, but more importantly, we werent even told why we were learning mod. the only explanation is that "its used in cryptography"

i honestly doubt she knows what aes is.

to sum it up:

she thinks everything we send to a server is encrypted via the public key.

she thinks *every* public key is inherently hard to crack.

she doesnt know https uses symmetric encryption.

i think that she doesnt know that the authenticity of certificates must be checked.

Client from a big company requested that all sensible data should be encrypted, passwords included.
We agreed that was OK, and that we were already saving the hashes for the passwords.
The reply was "Hashes should be encrypted too"

Hey !
A big question:
Assume we got an android app which graphs a sound file .
The point is: the user is able to zoom in/out so the whole data must be read in the begining , but as the file is a little longer , the load time increases.
What can i do to prevent this?

Someone at work snuck something past the censors.

Our Hadoop servers all have "bigd" in their name 😂

I hate it when marketing people decide they're technical - quote from a conference talk I regrettably sat through:

"The fourth industrial revolution is here, and you need to make sure you invest in every aspect of it - otherwise you'll be left in the dust by companies that are adopting big data, blockchain, quantum computing, nanotech, 3D printing and the internet of things."

Dahhhhhhhhhh

I work in big data and security analytics... So naturally I am a very detail oriented and analytical person.... Something that I constantly get told off about for 'over analysing' every personal situation.... Especially by my wife!

Today she was chatting to our best friend on WhatsApp who's been travelling Europe for 6 months seeing how life is etc... At the end of the conversation I point out he never once told her where he was and avoided the questions.... And that I was sure he was gonna turn up tonight....

She tells me I'm over analysing and full of it as she's in Spain.... Not even an hour later he walked through our front door and we are in the pub having an awesome catch up!

Who said I'm over analytical! More like right!?

Goals for the next 100 weeks eh?
- Teach my 8yo and 11yo to be awesome java coders
- Take my 2yo to her first day at school
- Grow my department in work to over 40 people worldwide
- Start and finish my Masters degree in Big Data*
- Speak at 5 major international conferences (1000+ ppl)

Every 2019 tech startup: We do deep machine learning with big data in the cloud.
Investors: Please take our money!

This buzzgasm is getting to artists too.

The GitHub graphql API is pretty neat, mostly because it's a great example of a product where graphql has advantages over REST. As a code reviewer for repos with hundreds of simultaneous PRs, I use it to filter through branches for stuff that needs my attention the most.

NewRelic's NRQL API is also quite nice, as it provides an unusual but very direct interface into the underlying application metrics.

I'm also a big fan of launchlibrary, purely because I love spaceflight, and their API is an extremely rich and actively maintained resource. This makes it a great data source for playing around with plotting & statistics libraries — when I'm learning new languages or tools, I prefer to make something "real" rather than following a tutorial, and I often use launchlibrary as a fun and useful data backend.

"How we use Tensorflow, Blockchain, Cloud, nVidia GPU, Ethereum, Big Data, AI and Monkeys to do blah blah... "

Recently found an artist called "Big Data". His song titles describe what Big Data actually is really well:

Perhaps more of a wishlist than what I think will actually happen, but:

- Everyone realises that blockchain is nothing more than a tiny niche, and therefore everyone but a tiny niche shuts up about it.
- Starting a new JS framework every 2 seconds becomes a crime. Existing JS frameworks have a big war, until only one is left standing.
- Developing for "FaaS" (serverless, if I must use that name) type computing becomes a big thing.
- Relational database engines get to the point where special handling of "big data" isn't required anymore. Joins across billions of rows doesn't present an issue.
- Everyone wakes up one day and realises that Wordpress is a steaming pile of insecure cow dung. It's never used again, and burns in a fire.

The world is talking about AI, self-driving cars, big data, IOT and there are roboter driving around on Mars.
And here I stand, trying to figure out why a small change in a silly batch-script works on Windows7 and raises an error on Windows XP.
In 2020.

How to get investors wet:

“My latest project utilizes the microservices architecture and is a mobile first, artificially intelligent blockchain making use of quantum computing, serverless architecture and uses coding and algorithms with big data. also devOps, continuous integration, IoT, Cybersecurity and Virtual Reality”

Doesn’t even need to make sense

I received 2 job offers:

1: c++ / c# / unity developer for a VR studio, tons of vr visors and shit to use

2: python / Java/somethingelse developer for machine learning, iot, big data

Offer n1 is from a small business 35 employees - casual outfit
Offer n2 is from medium/big business with 100-200 employees - suit and tie for all.

Same economic offer, 2 different and divergent paths on different but trending topics.

What do you choose and why

I see the industry popularizing Machine Learning programs using AI to implement ethical Blockchain as a Javascript framework using Scrum techniques for Big Data Web2.0 in Responsive Virtual Reality for your IoT Growth Hacking operations.

LabVIEW.

Because WHY THE ACTUAL FUCK should you want to use a visual programming language in a professional environment and pay for it.

(Other than: the manufacturer of your measurement device/power supply/electronic load/etc. has already provided a LabVIEW module so you just have, you know, 'click' your program together and be done.

No, we won't give you the documentation on how to do it properly without that piece of crap or even give you code snippets.

(If you don't feel the urge to shoot yourself in the foot, you have obviously too much time on your hands and could simply be reading the interface definitions for that particular interface. At least it's standardized, d'uh.)

Oh, and you want a lightweight application? Here comes the runtime environment! A big clunky ... thing you'll need now to start up even a simple measure-and-log-data-thing.

Well, OK, it works for the occasional Measure-and-Log-Thing. If you don't need the data too fast.

If you want to do something a bit more complex, knock yourself out, but don't ask me to debug it for you afterwards because that colourful entanglement of wires and connections and blocks is a DAMN HUGE MESS and trying to understand how it works feels like defusing a bomb in a shitty action movie.)

Never again.

First year: intro to programming, basic data structures and algos, parallel programming, databases and a project to finish it. Homework should be kept track of via some version control. Should also be some calculus and linear algebra.

Second year:
Introduce more complex subjects such as programming paradigms, compilers and language theory, low level programming + logic design + basic processor design, logic for system verification, statistics and graph theory. Should also be a project with a company.

Year three:
Advanced algos, datastructures and algorithm analysis. Intro to Computer and data security. Optional courses in graphics programming, machine learning, compilers and automata, embedded systems etc. ends with a big project that goes in depth into a CS subject, not a regular software project in java basically.

We started working with some pretty big (in data volume) client. Around 4.000 projects with about 10 to 15 deliverables by project. Our software helps them plan/manage that.
US : Hey, so on this page we only display first 10, so it is fast and you can adjust using filters.
Client: No, I want to see all 4.000 projects on the same page
US : Well, for one year it will generate : 4000x10(deliverables)x12 editable fields. Your browser will crash. (No time to add virtual scroll)
Client: No, I want to see all 4.000 projects on the same page
US : Ok, here is pagination to help you.
Client: No, I want to see all 4.000 projects on the same page
US: …
Tomorrow is going to be fun.

Want to make someone's life a misery? Here's how.

Don't base your tech stack on any prior knowledge or what's relevant to the problem.

Instead design it around all the latest trends and badges you want to put on your resume because they're frequent key words on job postings.

Once your data goes in, you'll never get it out again. At best you'll be teased with little crumbs of data but never the whole.

I know, here's a genius idea, instead of putting data into a normal data base then using a cache, lets put it all into the cache and by the way it's a volatile cache.

Here's an idea. For something as simple as a single log lets make it use a queue that goes into a queue that goes into another queue that goes into another queue all of which are black boxes. No rhyme of reason, queues are all the rage.

Have you tried: Lets use a new fangled tangle, trust me it's safe, INSERT BIG NAME HERE uses it.

Finally it all gets flushed down into this subterranean cunt of a sewerage system and good luck getting it all out again. It's like hell except it's all shitty instead of all fiery.

All I want is to export one table, a simple log table with a few GB to CSV or heck whatever generic format it supports, that's it.

So I run the export table to file command and off it goes only less than a minute later for timeout commands to start piling up until it aborts. WTF. So then I set the most obvious timeout setting in the client, no change, then another timeout setting on the client, no change, then i try to put it in the client configuration file, no change, then I set the timeout on the export query, no change, then finally I bump the timeouts in the server config, no change, then I find someone has downloaded it from both tucows and apt, but they're using the tucows version so its real config is in /dev/database.xml (don't even ask). I increase that from seconds to a minute, it's still timing out after a minute.

In the end I have to make my own and this involves working out how to parse non-standard binary formatted data structures. It's the umpteenth time I have had to do this.

These aren't some no name solutions and it really terrifies me. All this is doing is taking some access logs, store them in one place then index by timestamp. These things are all meant to be blazing fast but grep is often faster. How the hell is such a trivial thing turned into a series of one nightmare after another? Things that should take a few minutes take days of screwing around. I don't have access logs any more because I can't access them anymore.

The terror of this isn't that it's so awful, it's that all the little kiddies doing all this jazz for the first time and using all these shit wipe buzzword driven approaches have no fucking clue it's not meant to be this difficult. I'm replacing entire tens of thousands to million line enterprise systems with a few hundred lines of code that's faster, more reliable and better in virtually every measurable way time and time again.

This is constant. It's not one offender, it's not one project, it's not one company, it's not one developer, it's the industry standard. It's all over open source software and all over dev shops. Everything is exponentially becoming more bloated and difficult than it needs to be. I'm seeing people pull up a hundred cloud instances for things that'll be happy at home with a few minutes to a week's optimisation efforts. Queries that are N*N and only take a few minutes to turn to LOG(N) but instead people renting out a fucking off huge ass SQL cluster instead that not only costs gobs of money but takes a ton of time maintaining and configuring which isn't going to be done right either.

I think most people are bullshitting when they say they have impostor syndrome but when the trend in technology is to make every fucking little trivial thing a thousand times more complex than it has to be I can see how they'd feel that way. There's so bloody much you need to do that you don't need to do these days that you either can't get anything done right or the smallest thing takes an age.

I have no idea why some people put up with some of these appliances. If you bought a dish washer that made washing dishes even harder than it was before you'd return it to the store.

Every time I see the terms enterprise, fast, big data, scalable, cloud or anything of the like I bang my head on the table. One of these days I'm going to lose my fucking tits.

So as applying for an internship to a new company, they wanted me to make an account and do some things to get use to the website... That's great, until I learned their website is fucking garbage!

Takes 5 seconds to load any page (they import and link so much shit, it's poorly optimized), their website is vulnerable to Javascript injection (in many different places), im sure it will be vulnerable to sql injection too.

Their design looks bad, icons are terrible, no common design flow, super busy. And they are taking about using machine learning and big data? Bitch you need to fucking make your site usable first!! If contacted them and will give them 30 days to fix their shit before I write about it

Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it… — Dan Ariely

During an internship, I spent some time automating reports with VBA. Basically, imagine a few big excel sheets with 1000 formulas and a few thousand lines of VBA.
One of the reports was handed directly to the bosses boss of our boss. After 4 weeks, he came to me and asked why the table entry in row 23 or so was always 84. Well.. I dont know. This data is automatically calculated / retrieved from a database. Went and checked, already sweating, and found that
THE OTHER INTERN COULDNT FIX A FORMULA SO HE OVERRODE IT WITH PLAIN TEXT. WITH A FUCKING PLAIN VALUE OF 84. A FOOKING EXCEL SHEET WITH A THOUSAND DIFFERENT FORMULAS AND LOTS OF VBA. Needless to say, everything is password protected now.

"Our company encourages cryptocurrency big data agile machine learning, empowerment diversity, celebrate wellness and synergy, unpack creative cloud real-time front-end bleeding edge cross-platform modular success-driven development of digital signage, powered by an unparalleled REST API backend, driven by a neural network tail recursion AI on our cloud based big data linux servers which output real time data to our Wordpress template interactive dynamic website TypeScript applet, with deep learning tensor flow capabilities.

Don't get what the fuck I just said? Udemy offers countless courses on python based buzzwords. Be the first out of 13 people to sell your soul and private information, and you'll get the first three minutes of the course free!"

Pffff...... Wanna make an app tomorrow...
Got no clue what to make....

Maybe something with big AI learning data machine. Yeah I think that hits all the right buzzwords :P

Any ideas you're willing to share?

Fuck you Intel.
Fucking admit that you're Hardware has a problem!

"Intel and other technology companies have been made aware of new security research describing software analysis methods that, when used for malicious purposes, have the potential to improperly gather sensitive data from computing devices that are operating as designed. Intel believes these exploits do not have the potential to corrupt, modify or delete data"

With Meltdown one process can fucking read everything that is in memory. Every password and every other sensible bit. Of course you can't change sensible data directly. You have to use the sensible data you gathered... Big fucking difference you dumb shits.

Meltown occurs because of hardware implemented speculative execution.
The solution is to fucking separate kernel- and user-adress space.
And you're saying that your hardware works how it should.
Shame on you.

I'm not saying that I don't tolerate mistakes like this. Shit happens.
But not having the balls to admit that it is because of the hardware makes me fucking angry.

🤔🤔🤔🤔🤔😓

Google.
They’re doing amazing things but they are just too big now... Too much of a monopoly and the data is scary too.

Fucking shit uni is such a waste of time. We are learning Apache Spark in Big Data module. Fucking losers have Spark 1.6.0 installed while the latest version is 2.2.1 right now.

What a bunch of cunts. We are paying tons of money to study deprecated shits and a degree. A fucking degree that is not even on a piece of paper anymore.

Fuck this shit man.

"What app is that?"

"It's like yik yak for developers..."
"Like FML for programmers..."
"Like one big groupme for computer scientists..."
"Like Josh from Intro to Data Structures..."

I'm running out of ways to describe devRant.

The project is complete, i am free :') Time to look for a new job, maybe try myself at big data

Worked as a student in a big company. Just doing data entry and checking product data. It was a nice part time job at the same time with computer science study. After a year I asked for something else and switched to the Android Team. I said I could do a little bit of Java and wrote for another half year Unit Tests. That was the point where I really learned coding and got experienced. Would never learned so much in my study because I was lazy. Now I can call me a Android Developer. Still love the company for giving me this opportunity.

My manager's boss just commited on a delivery date a month from now. We dont know what is to be delivered, nor does the client. We are supposed to work on a platform that we know nothing about. And of course the catchphrase is : yeah just use big data and spark. I'm dying...

Yeah so seems like huge companies are literally just throwing tech buzzwords like "blockchain" and "cloud" and "big data" for marketing purposes.This annoys me.

Client: THIS IS CRITICAL, SOME DATA HAS BEEN DELETED, WHAT ZE FUUK HAPPENED, UNDO THIS FAST
Us: so after carefully reviewing the code, related resources and the network traffic we conclude that was never sent in the first place.
*closes issue*

I'm glad we got such a meaningful bug report on the same day a production system started failing, one big deployment that that was like a boss with 3 phases, an unnecessary long meeting and an app developer that that wanted me to break HTTP standards.

i want to get my own social network up and running.

so far ive got -
login 100% securely
register (1000% securely)
view someone’s profile (10^7% securely)

to add -
scrypt (maybe bcrypt, however scrypt looks like the better option)
friend a user
track their every move (ill use facebooks and googles apis for that)

to describe my product -
ai
blockchain
iot
big data
machine learning
secure
empower
analysis

call me when im a gazillionaire

but seriously, im making a social network and i hope its done by wk105 tbh

Hi.. one month ago i started to learn JavaScript (my first programming language)
In the 2nd proyect we create a Data dashboard i do my very best effort to create Js funcional code and other 2 girls works in css and html.
Im really proud of my work (1st time!)
A few guys told me JavaScript is awful and difficult but in a few weeks we will start in jquery.
In 2 weeks im gonna participate in Angelhack Santiago Hackathon 2018
I need an advice for me its a really big step

Yes! What’s after Big Data ?

So here I am testing some python code and writing to a file. No big deal. But damn is it taking a long time to get data back from this API. Ah it's fine I'll let it work in the background.

40 minutes later.

Oh! The requests timed out. No big deal. I'll just cut out the parts that are already done.

1st request in.
I wonder what the file is looking like.
Only showing 1 request.

waitaminute.jpeg

I should have more than that.
*Suddenly realizes that I was writing to the file and not appending.

Fuuuuuuuuuuuuuuuuuuck

I'll just start off with how I really feel. Fuck big corporations with their career robots and retarded practices!
Now for a story. So I work remotely for most of the time nowadays, since my company has as clients big corporations. Used to be embedded with said clients, but it became kind of painful to work with them all so I asked to be reassigned to a remote position.
Now for the retarded part: The fucking Klingons I'm working with have two tiers to their VPN, but won't let me have the full version because it would be too fucking expensive. I checked and it's fucking 50 bucks per year difference.
So for that the Klingons are making me code through a remote connection that has a "best effort" priority.
Fuck.
Anyway after 3 weeks of writing code at a 400-600ms latency I finally snap.
I try to use a proxy and it. I write one myself, gets balcklisted in 2 days.
After about another week of writing code through a fuck straw I start working on node socket with 2 clients and a server that encrypts the send data, and syncs 2 folders between my workstation and the remote one.
It's been a month now and it is still working. It's not perfect, but I can at least write code without lag.

Question for you peeps: What shenanigans have you pulled to bypass shit like this?

Rant considering the latest Cyber attack and the news around it.

(A recap: a lot of Windows computers were infected with ransomware (due to security hole on Windows), which demanded 300$ in bitcoins to unlock data. After 3 days the price would double, and after 7 days the data was to be deleted)
1) In our country, one of the biggest companies was attacked (car factory). The production stopped and they got for around 1 000 000€ damage in less than 24h (1300 people without work). The news said that they were attacked because they are such a big company and were charged more, as the hackers "knew who they were dealing with" - another reason being the fact that the text was in croatian (which is our neighbor country), but noone realized that it is just a simple google translate of english text - which is obviously not true. The hackers neither know nor care who is hacked, and will charge everyone the same. They only care about the payment.
2) In UK whole (or large part) of medical infrastructure went down. The main thing everyone was saying was: "Nobody's data is stolen". Which, again, is obvious. But noone said anything about data being deleted after a week, which includes pretty much whole electronic medical record of everyone and is pretty serious.
And by the way, the base of the ransomware is code which was stolen from NSA.
All that millions and millions of dollars of damage could be avoided by simply paying the small fee.
The only thing that is good is that (hopefully) the people will learn the importance of backups. And opening weird emails.
P.S. I fucking hate all that 'hacky thingys' they have all over the news.

Why everyone is happy about Google clip? It's the single most scary instance of a big brother appliance that exists today. What are they going to do with the data? They say it's save memories of your kid or your dog. There's already something like that. It's called a brain and paying attention to your damn life. I don't want to be saved in your shitty memories just bc you are so insecure about remembering your fuck*ng memories.

I'm sorry for the outburst but that sh*t is solving a problem nobody had and it's getting applauded like those heaven's gate motherf*ckrs that say that life is improved by these shitty beliefs.

When I wrote my first algorithm that learns...

So in order to on board our customers onto our software we have to link the product on their data base to the products on ours. This seems easy enough but when you actually start looking at their data you find it's a fuck up of duplication's, bad naming conventions and only 10% or so have distinct identifiers like a suppler code,model no or barcode. After a week or 2 they find they can't do it and ask for our help and we take over. On average it took 2 of our staff 1-2 weeks to complete the task manually searching one record of theirs against our db at a time. This was a big problem since we only had enough resources to on board 2-4 customers a month meaning slow growth.

I realized when looking at different customers databases that although the data was badly captured - it was consistently badly captured similar to how crap file names will usually contain the letters 'asd' because its typed with the left hand.

I then wrote an algorithm that fuzzy matched against our data and the past matches of other customers data creating a ranking algorithm similar to google page search. After auto matching the majority of results the top 10 ranked search results for each product on their db is shown to a human 1 at a time and they either click the the correct result or select "no match" and repeat until it is done at which point the algo will include the captured data in ranking future results.

It now takes a single staff member 1-2 hours to fully on board a customer with 10-15k products and will continue to get faster and adapt to changes in language and naming conventions. Making it learn wasn't really my intention at the time and more a side effect of what I was trying to achieve. Completely blew my mind.

Big data text book

The real big data

Encryption, Data, Servers, Protection, Certificate

oOOO WEE, I use big ear old words so I must be a hacker.

Big data....😉

I was cleaning up dangling images in docker, and I accidentally removed the production database container as well.

Its not a big issue, I can just up the container back and everything should be fine. But after I up the container and connected to the database, I found out there's no data inside. I thought I fucked up, and sent msg in slack channel that I nuked the db.

Later my friend asked me which compose file I am using and that's when I realized I used the wrong config to up the db. Used the correct config to up the database again and everything goes back to normal.

It's friday evening and if I really dropped the db it would be fucking bad weekend....

I've just realised, I don't care if Facebook are sharing my data... I mean, what I get more ads for men with big penises and ripped abs? So what...

The scrum master for the project I'm working on decided to help out with changing some code (I'll add he's got a master's in software engineering and very proud about it..aka..big ego). It took him two days...yes two days to write the attached code.
I reviewed his code and sent back a response (code took about 15 seconds to write) including the link to the logging documentation explaining what fields were and were not necessary. Not sure how will look in devrant ...

var data = new InformationalDataPoint
{
Properties =
{
["RMANumber"] = rma,
["InvoiceID"] = invoiceId
}
};
Logger.Log(data);

He's stopped talking to me. Our next scrum meeting with the product owner should be ...um...awkward.

I normally just have nightmares about the projects I'm working on, especially when I struggle with a bug for days. Those are usually about just me stressing out about it. However, I have a lot of dreams about computers/technology, not necessarily coding-related:

- datacenters were just potato fields. If you go work the field, you'd go data mining

- in Biology, when being taught how having children works, you only tell that "parenting is only chmod-ing the rights of your children until they become the owners themselves"

- IP addresses with emojis instead of numbers were a standard now and they actually managed to replace IPv4, because everyone was so into emojis. They named it IPvE

- I witnessed a new Big Bang when the 32-bit Unix time overflown in 2038, and we were all quantum bits

INTERVIEW. It tells everything about the company. I recently applied for a "big" company for the position of ML Engineer. The Job description was like "someone with good knowledge of visual recognition, deep learning, advanced ML stuff, etc." I thought great, I might be a good fit. A guy called me the next day. Introduced himself as a manager of the Data Science team with 8+ years of experience. Started the talk saying "it is just an informal intro". But things escalated very quickly. Started shooting Data Science questions. He was asking questions in a very bookish way. Tells me to recite formulas (like big formulas). When I explained to him a concept, he was not understanding anything. Wanted a very bookish answer. I quickly realized I know more about ML stuff than him (not a big deal) and he is arrogant as fuck (not accepting my answers). Plus, he has no knowledge about Deep Learning. At the very end, he tells me "man, you need to clear up your fundamentals". WTH??? My fundamentals. Okay, I am not Einstein or Hinton, but I know I was answering things correctly. I have read books and research papers and blogs and all. When I don't know about things, I tell straight away. I don't cook answers. So the "interview" ended. I searched that man on LinkedIn. Got to know he teaches college students Data Science and ML. For a fee of 50,000 INR. It's a big amount!! Considering the things he teaches. You can find the same stuff (with far higher quality) free of cost (on Coursera, Udacity, YouTube, free books, what not). He is a cheater. He is making fool of college students. That is why I sometimes hate "experience". 8+ years of exp and he is such an a**hole!! BTW, I thanked God for saving me from that company. Can't imagine such an arrogant boss.

TLDR: Be vigilant during interviews. It tells a lot about the company.

If you're currently in college and wish to get placed in a major tech giant like Amazon or Facebook:

Don't learn React.js, instead learn Linked lists.

Don't learn Flutter, instead learn Binary search trees.

Don't learn how to perform secure Authorization with JWTs, instead learn how to recursively reverse a singly linked list.

Don't learn how to build scalable and fault tolerant web servers, instead learn how to optimally inverse a binary search tree.

These big tech companies don't really care what real world development technologies you've mastered. Your competence in competitive programming and data structures is all that matters.

The system is screwed. Or atleast I am.

So I've created this account specifically for this rant. I usually just browse anonymously.

I've recently been hired in a big company that is one of the biggest Microsoft users in the world and my essentially revolves on making it easier for our collaborators to work with SharePoint (and other ms software)

Never in my life have I hit that much of a roadblock. So for the past week I've been trying to integrate what Ms calls webparts. And to modify the default webparts Ms provides you need to their properties (or Metadata). Except here's the big problem these are NOT documented anywhere (unless I failed to find it, if you do know where it is documented please HMU), so I've found myself trying to reverse engineer the js scripts that are served with SharePoint to figure out what the webpart properties are called and what type of data they are! I've been going through endless github repos using the CSOM nuget package (it's the library everyone uses to interact with SharePoint) and I finally found out about this other library called PnP which is a wrapper around CSOM that makes it easier to use. That wrapper has a way for me to load existing page and look at the properties of existing webparts. So here I thought it was the end of my suffering and I could finally get an idea of what it should be. Turns out this method doesn't work because one of the dependencies it has has had breaking changes and they still updated it even though it breaks their code! So for the past two days I've been trying random combinations of key values with different data types and json serialization methods.

Oh and yeah I've also looked at all the http calls via the chrome network tab, the metadata is not served as an individual file but is computed by Ms servers when they're serving you their html files.

So uh yeah run from CSOM if you can..

I fixed my big data processing code, I think. If all works as planned, I'll wake up to some processed data that I can do some statistical analysis on... I hope...

At one point, my laptop's hard drive went down. Turns out, windows had written some garbage data to the mft, and fucked up the file structure. Luckily i was able to restore a big chunk of the data using recuva. I cleaned the disk after saving the most important files, cleaned the disk, reinstalled windows. All good so far. I put the laptop's drive and my recovery disk into my desktop to put back the files. During the install in forced me to make an account, which I wanted to delete. So I ran "rmdir /users /s" and went to grab a cup of coffee. Turns out, cmd was pointed at my recovery disk instead of my laptop disk. My whole backup wiped.

Building an interface for a client between industrial power quality meters and a database that serves a webapp of data.

Client had heard of a way of sending data between meter and raspberry. From some manager in a big firm.

Currently we where using modus to connect the meter to a raspberry. This method was tested and proofen to work. Both devices could talk to each other in modbus.

Client kept demaning to use mbus, and was nog listening to any reason because the firm suggested it. In the end we end up going modbus to mbus to send it to the raspberry. There the mbus was converted back modbus. Because the meter could not communicate in mbus.

Really weird experience to program something so useless. But protesting about it was going nowhere and taking more time than the changes would take to implant.

I never knew that I was a good mentor at SQL , specially at PL/SQL.

I gave a task to a new member of my team, to fill 5 tables with data from other 15 tables.

I informed him well about data table info and structure. He spended about 3 days to create 25 different queries in order to fill 5 tables.

After I saw the 25 queries, I told him, that he could do it with 1 main query and 5 insert statements.

So I spended 1 hour of training, in order to build,run and explain how to create the best sql statements for this task.

(First 5 minutes)
It was looking so simple at the beginning from starting with 1 simple join, after some steps he lost my actions.

(Rest 55 minutes)
I was explained the sql statements I 've created and how Oracle works.

Now , every time he meets me, he feels so thankful for learning him all those Oracle sql tips in 1 hour.

Now he is working only with big data and he loves the sql.

Testers in my team have been told like 1000 times to follow the style guides that we all follow. That's not that big a deal. The big deal is that they were put on this project without having any mathematics background when the project is all about geometric stuff. So after me as a developer having to put so many hours to explain to them why the tests are not covering the requirements or why the tests are red because they are initializing the data completely wrong, I ask them pretty please to do the checks for the coding style and I have already been 4 hours reviewing code because not only I have to go through the maths and really obscure testing code to ensure that the tests are correct, but every line I have to write at least 4 or 5 style corrections. And some are not even about the code being clean, but about using wrong namespaces or not sticking to the internal data types. For fuck shake, this is embedded software and has to obey to certain security standards...

Well, my country has a Degree called Bsc.CSIT which literally means Bachelors of Science Computer Science and Information Technology. I completed that degree and was employed right after I completed my degree. I have worked in two offices and no one cares what degree I have.

So I think Degree is not that necessary here in Nepal as long as you can get the job done.

Now I am about to pursue a Big Data related degree hope that is not as worthless as my current degree.

The word, "Code" being used as a verb,
"Cloud",
'Big Data",
Recruiters,
Scratch,
Any other crappy "Teach your kid to code!" Product,
And finally, mondays.

This is the comprehensive list of buzzwords and things in general that make me want to die right now

So I joined a course for big data analysis. And they setup a lab specifically for us. Pulling us away from the usual computer labs
AND GUESS WHAT THEY DON'T EVEN BLOODY HAVE MYSQL INSTALLED. THEY'VE CHARGED ME A FORTUNE AND THEY DIDN'T EVEN CARE TO INSTALL THE BASIC SOFTWARES AND ITS ALREADY BEEN 2 WEEKS WITH THE START OF THE COURSE AND NOTHING.
F**king hate this man!!!!

A few days after the vacation, daddy (me) finally finished copying all the photos from the digital camera to the big external hard drive. Left it lying in the living room, in the corner where all the tech stuff frequently lies. In come my boys, and I'm not sure why, they start plugging power supplies around. Unfortunately the one from the laptop fit into the hole on the external disk, but had a different (higher) voltage. Result: 1 fried hard drive, containing all the baby photos and videos of said boys up that time.
I still believe that it's only the circuit board that is fried, and the data is still all there. I still have the hope to find a used disk, replace the board and all will be back.

Agile development of a decentralised AI, using a neural network based on Blockchain technology for big data.

Is that enough buzzwords to make an employer happy? :p

A big project in my company. Had some annoying race condition that caused data to get deleted when two processes finished in the wrong order they hit the dB and override each other’s work.

Long story short. Fixed the bug and in the process the codebase shrunk by 60%. I didn’t have to delete the rest of the code, but the bug was due to a function in the legacy section of the code, and found out that it was the only function used in that section.

So I deleted it. Rewrote the function so it upserts. And bam. Smaller, cleaner code :)

What if people, life, humanity, the universe is just a cluster of CPUs running a giant Recurrent Neural Network algorithm? 🤔

-Sun and food == power source
-People == semiconductors
-Earth/a Galaxy == a single CPU
-Universe == a local grouping of nearby nodes, so far the ones we've discovered are dead or not what same data transport protocol/port as us
-Universal Expansion == the search algorithm
-Blackholes: sector failures
-Big Bang == God turns on his PC, starts the program
-Big Crunch == rm -rf

Can't get over how many big companies get away with poor/no documentation for their own APIs. The past week i have been working with a large insurance company that only via email threads explained what endpoint to send files to and what username I could use to get this to work.

I also worked with a major courier service last month that only had a two page document for all their methods and one of the pages was explaining the transportation of data via imagery haha.

I don't work for Walmart, but they almost put my job in jeopardy today. I have a console app in production that pushes Walmart orders from their marketplace into our system for fulfillment. For half a year, I have handled thousands of orders, but overnight, all customers were getting massive price cuts on products in the Walmart feed! I looked at the data and initially thought it was my error due to using a quotient instead of a product in the code. But upon closer inspection, some fool at Walmart had changed code on their end without telling my team! Broke all the things. Lucky we were able to pull a full stop before we lost disgusting amounts of money, but you would think that a big player like Wally would at least announce a breaking code change to their users. 😲😡

Deleted 5 databases out of which 3 of them contains important data. A team is working on a big feature from months on those databases. All of the data is gone.

!rant

I laughed.
Well played TeraData.

!rant

Does anyone know what the **day-to-day** differences are between working in IT (banks, hedge funds) vs tech (Google, Facebook, Netflix).

In my mind, I see Hell and Heaven. And there's a giant wall in between called "technical interviews + algorithms and data structures".

I'm on the Hell side... And not sure if I should climb the wall 😔

Is the wall even that big?

I don't know why they made so many algorithms, data structures and big O questions during interview, when all they wanted me to do was to maintain some legacy, tight coupled, spaghetti code with no architecture, documentation, tests nor any kind of engineering behind :/

Labelling almost 20 TB of Data in few Minutes

Is python a good language for building a RestAPI? Personally I don't have any experience with python yet, but what I've gathered, is that python is great for scripting, and big data.

I have a bit of knowledge about Node.js, and I really like the structure, and it's so easy to make an API using express.js.

I've already read a bunch of articles about it, but I'd like to know what the community feels about the two languages?

TL;DR - (almost) childhood trauma due to Wesrern Digital crap products lead to lot of data loss and a plege to not trust or purchase their products for the rest of my life.

....

So, I got my first ever Wester Digital 2TB Mybook, back when 2TB was a really big thing. While in the midst of moving (not copying) a LOT of data to it, the damn disk just.. died. There was no fall, no power outage, no damage, it just stopped working. I was out of words and out of options. Tried yanking out the disk and connecting it directly to a system, but no luck because it looks like it's the HDD mobo that died.

Also stupid young me did not realise back then that, even if a "moved" the data, the original data is still most likely in their original location, and so, never bothered a recovery.

Lots of good stuff lost that day.

And as with a lot of you, my disaster recovery system kicked up 10 fold. Now I got redundant local and cloud backup copies of all critical and otherwise unattainable data.

As you may have guessed, I never bought another Wester Digital product ever again. My internal HDDs are Segate, and external is a suprisingly long lived Toshiba Canvio.

I'm not sure if it is a dev experience, but definitely boosted my morale.

In 2014, my company (in India) sent me to attend a conference in Boston. The conference was about big data.

When I came back, I wrote a blog post about Apache Spark in my company's blog. Because of the blog, my name got mentioned in a prominent newspaper's article about Apache Spark.

PS: That is my only blog post till date

There's this company that works in the RPA (Robotic Process Automation) domain, they describe their work as using A.I and Machine Learning to collect Big Data whilst following the latest trends in IoT, Blockchain, Web 6.9, and any other fancy term they can use while in fact, well... They're using an outdated software that uses vb6 modules to scrap some images and no employee have ever written a single line of code.

Hello,
I just quit my job at a big market research company. It was disturbing how much processes there depended on excel and obscure visual basic scripts.

They load data from a database, do typical database tasks with excel and upload it back into the database.

PhDs run complex statical computations through an excel interface that passes the request to R.

Instead of an hour Python they execute stupid tasks with excel by hand. Day after day, month after month.

WHY? My colleagues were not dumb but instead of learning SQL and some python they build insane excel tables.

Maybe it's time pressure. But this excel insanity costs much more time in the end.

Found this gem on quickmeme.

So my customer wanted me to collect Big Data using SQLITE... (Yeap SQLITE, like wtf...)

F*** my career.

Back in the days when I knew only Windows, I used to be a Microsoft fan. I wanted to use only Microsoft products. I had a Hotmail email account that Microsoft acquired. I used a version of Windows and Microsoft Office (even though I didn't know at the time that it was pirated). I wanted to be a Microsoft Student Partner (MSP) and promote Microsoft everywhere.

Fast forward to now (or maybe to the time after I got introduced to GNU/Linux), I started hating Microsoft solely for the reason that they had a price-tag on everything. Later on, when I got to open-source software, I hated Microsoft for making all of their software closed-source. When I decided to move out of the Microsoft environment, my next favorite was of course, Big Brother (Google, if you haven't gotten it) - Gmail, YouTube, Google Drive. My personal information was the price to pay for the services even though I wasn't OK with that fact.

Then again, I realized that you could actually have your own stuff if you had the know-how. Compile / host your own software on your own systems. Oh, then I went on a compile spree. That's when I realized I didn't need any of these corporations to own my data. Today, I try my best to keep my data in my control and not some corporations who gives me free stuff for the price of my data and personal information, no thanks.

Client: "..and you will send us the big data via mail... right?"

At the end we send a 100 kb file.

I used to be a big security guy, not allowing stuff like most of the social media, not bringing my phone anywhere, carrying a RPi tablet for privacy reasons. Very Stallman stuff.

Recently I noticed that I don't care so much.. I see these things as opportunities, for instance Microsoft products could be benefitial for job opportunities, I have some workout sessions on my phone.

I could restrict myself... but is it worth it just to decline some capitalist/politician's row in a dataset for analysis?

But then again I feel as a society I think we should either do this or request this data to be distributed to us as well.

Should you be playing a game of cards, when the enemy can see your hand? What do u think?

In my project to process CSV file it needs to be transformed into XML file. Basically 100 MB CSV is turned into 3GB XML. They can be deleted afterward but it is not implemented and no one cares. Our storage gains around 1-2 TB data each month. Right now we have ~200 TB of XML in our s3
I think I can add Big Data to my CV *sarcasm*

For all the hate that Java gets, this *not rant* is to appreciate the Spring Boot/Cloud & Netty for without them I would not be half as productive as I am at my job.

Just to highlight a few of these life savers:
- Spring security: many features but I will just mention robust authorization out of the box
- Netflix Feign & Hystrix: easy circuit breaking & fallback pattern.
- Spring Data: consistent data access patterns & out of the box functionality regardless of the data source: eg relational & document dbs, redis etc with managed offerings integrations as well. The abstraction here is something to marvel at.
- Spring Boot Actuator: Out of the box health checks that check all integrations: Db, Redis, Mail,Disk, RabbitMQ etc which are crucial for Kubernetes readiness/liveness health checks.
- Spring Cloud Stream: Another abstraction for the messaging layer that decouples application logic from the binder ie could be kafka, rabbitmq etc
- SpringFox Swagger - Fantastic swagger documentation integration that allows always up to date API docs via annotations that can be converted to a swagger.yml if need be.
- Last but not least - Netty: Implementing secure non-blocking network applications is not trivial. This framework has made it easier for us to implement a protocol server on top of UDP using Java & all the support that comes with Spring.

For these & many more am grateful for Java & the big big community of devs that love & support it.

I will major in AI. No, I will major in Big data, wait, I want to major in cloud too. I think I should first complete the courses I enrolled on cloud academy or the tens of courses in enrolled on Udemy on all the domains possible first! So many technologies, so many dreams!

I hate doing front-end development...
I was hired along with another dev to build a webapp to manage the personnel of this big (2000+) company.
I made the backend and some of the frontend (mainly handling the data movement between the two), but my partner was let go after we delivered a first version because "there was not enough work for both of us".
The backlog is months of work for me and now I have to do everything and it's wearing me down...
I want to quit but it's paying well and I don't want to search for something new.
What do?

truth about big data 😂

What would be the better approach for loading very big in size or in quantity files in java?
1) Loading data parallel through multiple threads
2) Loading data in series in a single thread
3) any other methodology?

Just asking because loading time is varying both cases.

“Big data” and “cloud” words are missing

TIFU by showing login data during presentation

I was presenting my school project when my teacher asked if I could show him the source code. I said ofc, just let me login to the FTP server. I completely forgot that it was also shown on the big screen, and a random funny student logged in and tried to replace the index file with a joke file. Of course, he didn't want to make damage, so he made a backup. But this backup caused the problem, because he connected to the FTP through Windows Explorer (wtf?), and when he made a copy of the original file, it was renamed to "Copy of xy", but in a localized version, which contains special characters. Because of these characters, some FTP clients couldn't even connect, others just couldn't interact with the file. No download, no rename, no delete, nothing. After trying out like 8-9 FTP clients, I just remembered that I could rename it in PHP. Well, it got deleted instead of being renamed, but at least it wasn't there anymore. I have spent like half more hour with searching for a backup version on my computer until I found it.

TL;DR: showed FTP credentials during presentation on big screen, random student accessed and renamed a file, special characters in name fucked up the server, luckily I found a backup.

One of my colleagues who thinks he knows all about "big data": "I try to put everything in hadoop, that is my philosophy".

We don't even have a hadoop cluster.

The past few months i got a bunch of emails and calls from my previous "boss" (hes the head of the research), that he would be grateful if i helped them out. I got a few friends still working on that piece of shit project so i said yeah, i can help.

Now this whole thing is a research involving most of the big universities, lots of math phds, and is kinda secret. They couldnt find anyone to sketch up a few stupid algorithms for them so i did just that.

Yesterday i got the specifications for the task. Its the core functioning algo, the one i made from fucking discrete integer data, it took me 3 fucking months to correct their mistakes, and now they want me to create 2 similar patterns for 2 completely different...things. Yeeeah no.

Just found the most embarrassing security hole. Basically a skelleton key to millions of user data. Names, email addresses, zip codes, orders. If the email indicates a birthdate, even more shit if you chain another vector. Basically an order id / hash pair that should allow users to enter data AND SHOULD ONLY AUTHORIZE THEM TO THE SITE FOR ENTRING DATA. Well, what happend was that a non mathing hash/id pair will not provide an aith token bit it will create a session linked to that order.

Long story short, call url 1 enter the foreign ID, get an error, access order overview site, profit. Obviously a big fucking problem and I still had to run directly to our CEO to get it prioritized because product management thought a style update would be more important.

Oh, and of course the IDs are counted upwards. Making them random would be too unfair towards the poor black hats out there.

Can gamedevelopers stop using lua as their freaking scripting language..
Every time I try and figure out how tables work and think I finally get it it throws a big fuck you curve ball.

Oh and then they use json file to store the data of a table except that those json interfaces are complete retards.

If you are going to support json files then why the fuck won't you put in a small fucking inconsecential JS interperter so you can actually find some docs regarding more complex fucking docs then those simple minded t[guildName] = "guild"

Another thing, why the fuck does lua not use {} like every other langauge. I use those curly brackets to figure out where shit start and ends half the freaking time.

Fuck this I'm out for today...
And a big fuck you with both middle fingers to any dev that thinks lua is a great scripting language for plugins.

Big data loss... idk
I once lost a semicolon

Random opinion question:

I'm working on a thing where the user provides a big CSV and we process it and put it in the database, or update existing records.

This data impacts other things, but the data isn't front and center as a group of n the application for them to notice / see again (well they can query for it).

I'm thinking of taking the CSV and then presenting them with a table showing how we processed that data giving them a chance to review it before they commit it to the database...

I like this idea for two reasons:

1. If something goes sideways there's a chance someone will see it and I'm not sure I can do enough validation on a big ass CSV from god knows where to be sure we're going to process it right... (I'm going to do some validation but just can't cover it all)

2. It takes some of the mystery out of what happened / is happening for the user for.

Anyone try this in the past? Seems reasonable, but lots of things do before they go sideways.

Every single time I present a tool for data visualization:

"Oh that's great! Have you considered integrating it with service XYZ? It would be great to see the data from XYZ alongside this."

The answer I would like to give:

"No, you retard! Nobody gives a fuck about your crappy service! Nobody uses it, not even your own team! This is the 10th service that I've been asked to integrate and I don't have time to dig into the details of yet-another-shit. If you have time to waste, please go ahead but don't bother me."