Do all the things like ++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatarSign Up
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple APILearn More
Search - "data analysis"
WHY THE FUCKIDY FUCK DO PEOPLE THINK THAT JUST BECAUSE YOU ARE A DEVELOPER THEN YOU CAN DO EVERY FUCKING THING IN A SNAP?!
THIS ENTIRE SUMMER I WORKED FOR THIS MIDGET OF A FUCK AND THE IDEA WAS TO MAINLY DEVELOP AN IOS APP FOR THEIR PRODUCT. THAT ONE APP TRANSLATED TO THREE SEPARATE APPS AND KEEP IN MIND THAT I AM THE ONLY DEVELOPER THEY HAVE SO I HAVE TO DEVELOP IT, TEST IT AND DOCUMENT IT!! AND WHAT'S EVEN WORSE IS HE KEEPS GIVING ME DESIGNS AT THE LAST MINUTE SO I ALWAYS END UP HAVING TO CREATE NEW PROJECTS ALL THE TIME. WHEN A WEEK PASSES BY WITHOUT ANY UPDATE HE AND HIS FUCKED UP BRAIN CELLS GATHER A MEETING WITH HIS 'DONALD TRUMP HANDS-SIZED' BALLS TO ASK ME, 'WHY ARE YOU TAKING SO LONG? THESE ARE JUST THREE BUTTONS ON THIS VIEW?' MOTHERFUCKING COCKSUCKER!! GO GET YOUR MOM TO DO THIS WORK THEN IF IT'S JUST THREE BUTTONS. TO HIM EVERYTHING IN IOS WHICH INCORPORATES A TAP IS A FUCKING BUTTON! BUTTON THIS! BUTTON THAT! AND IT'S NOT LIKE HE HAS SIMPLE DESIGNS..NO.NO.NO.NO. THIS ASSHOLE-SHAPED-HEAD MUGGET DESIGNS SHIT WHICH REQUIRES ME TO HAVE TO DRAW A UIVIEW AS THE SHAPE OF A HUMAN BODY AND HEART. THEN ASIDE FROM THAT, JUST BECAUSE MY RESUME SAYS I MINORED IN MATH AND APPLIED MATH, HE SENDS ME A PAPER THAT EXPLAINS MATHEMATICAL CONCEPTS OF DATA ANALYSIS THEN WANTS TO MEET WITH ME TO DISCUSS THE SUGGESTED METHODS THEN IMPLEMENT THEM. AT THIS POINT I HAD ALREADY IMPLEMENTED AN ALGO FOR COUNTING THE NUMBER OF STEPS THAT WAS WAY WAY WAY MORE ACCURATE THAN THE SHIT THEY HAD IN THEIR CONTROLLER..
SO AS I AM ABOUT TO FINISH IMPLEMENTING JUST THE INITIAL 5 VIEWS OF THIS 'FINAL' APP, HE SERIOUSLY WALKS IN AND TELLS ME, SO I'M STARTING TO WORRY THAT WE'LL NOT MEET THE AUGUST DEADLINE SO I'M THINKING MAYBE YOU SHOULD START SWITCHING BETWEEN DEVELOPMENT. WORK ON IOS FOR 4 HOURS THEN SWITCH TO ANDROID FOR THE REMAINDER OF THE TIME. OH AND UHH IF YOU FEEL LIKE TAKING THE IMAC WITH YOU TO WORK FROM HOME, THAT'S FINE AS WELL AND I'LL BE AVAILABLE IN THE WEEKENDS AS WELL. IT WAS AT THIS MOMENT THAT I REALLY REALLY HOPED I WERE GAY! LIKE 'NO LUBE' STYLE KINDA GAY!! OH BTW AT SOME POINT HE HAD ME PROGRAM ONE OF THEIR CONTROLLERS, AND WAS ASKING IF I COULD START RESEARCHING MEANS OF WRITING AN SDK FOR THEM.
KEEP IN MIND THAT I'M AN INTERN WITH ONE YEAR IOS DEV EXPERIENCE.. THEN HE WANTS ME TO ENTIRELY START LEARNING ANDROID AND GIVE HIM TWO VERSIONS OF THIS THIRD APP IN TWO WEEKS.
HE CAN GO SUCK HIS OWN DICK WHILE GETTING FINGER FUCKED BY A FORK WITH A BLUE WHALE'S 6FT LONG DICK UP HIS MOUTH.
*** that felt good ****36
Each month my department compiles a 4M row 150 column data table for compliance with a federal agency. Before submitting, we check it against about 400 rules.
The existing system was simply 400 queries that ran in sequence, table-scanning 4M rows each time, taking upwards of 6 hours, which is a huge bottleneck, especially if you have to make changes and rerun. Plus the output was rather one-dimensional.
I built a proper normalized database and created a sort of rules engine, running all 400 rules in one table scan. Not only does it complete in 30 minutes, but the reports generate automatically, and the results can be filtered on several dimensions to aid with root-cause analysis.
Management was pleased.4
No it's not AI. YOU ARE RUNNING FUCKING SQL QUERIES AND CALLING IT AI!
No it's not AI. YOU ARE RUNNING SIMPLE DATA ANALYSIS MACROS AND FUNCTIONS IN EXCEL!
Stop labelling everything as AI, you attention and investment seeking morons! @&£$¢×xo##!29
Husband wanted to not go out after dinner to "go cuddle in bed." This apparently is code for him browsing mathmatica stack exchange and reading books on data analysis with R... 😂9
Smart India Hackathon: Horrible experience
Background:- Our task was to do load forecasting for a given area. Hourly energy consumption data for past 5 years was given to us.
One government official asks the following questions:-
1. Why are you using deep learning for the project? Why are you not doing data analysis?
2. Which neural network "algorithm" you are using? He wanted to ask which model we are using, but he didn't have a single clue about Neural Networks.
3. Why are you using libraries? Why not your own code?
Here comes the biggest one,
4. Why haven't you developed your own "algorithm" (again, he meant model)? All you have done is used sone library. Where is "novelty" in your project?
I just want to say that if you don't know anything about ML/AI, then don't comment anything about it. And worst thing was, he was not ready to accept the fact that for capturing temporal dependencies where underlying probability distribution ia unknown, deep learning performs much better than traditional data analysis techniques.
After hearing his first question, second one was not a surprise for us. We were expecting something like that. For a few moments, we were speechless. Then one of us started by showing neural network architecture. But after some time, he rudely repeated the same question, "where is the algorithm". We told him every fucking thing used in the project, ranging from RMSprop optimizer to Backpropagation through time algorithm to mean squared loss error function.
Then very calmly, he asked third question, why are you using libraries? That moron wanted us to write a whole fucking optimized library. We were speechless at this question. Finally, one of us told him the "obvious" answer. We were completely demotivated. But it didnt end here. The real question was waiting. At the end, after listening to all of us, he dropped the final bomb, WHY HAVE YOU USED A NEURAL NETWORK "ALGORITHM" WHICH HAS ALREADY BEEN IMPLEMENTED? WHY DIDN'T YOU MAKE YOU OWN "ALGORITHM"? We again stated the obvious answer that it takes atleast an year or two of continuous hardwork to develop a state of art algorithm, that too when gou build it on top of some existing "algorithm". After listening to this, he left. His final response was "Try to make a new "algorithm"".
Needless to say, we were completely demotivated after this evaluation. We all had worked too hard for this. And we had ability to explain each and every part of the project intuitively and mathematically, but he was not even ready to listen.
Now, all of us are sitting aimlessly, waiting for Hackathon to end.😢😢😢😢😢25
Perhaps not "best", but certainly most amusing, so what the heck!
Years ago as an intern, I applied to a large pharmaceutical company. On part of the application form, you had to enter the code of the department you were applying to.
What I *should have* put down was "IT", which is the department that houses all their devs. However, I didn't actually read any of what the codes meant, assumed that was the department for helping people with how to mail merge, and put down "COMPSCI" instead. This was computational sciences - loosely summarised as computational data analysis on various druggable molecules.
I do *not* have any sort of biology or chemistry background, so the interview was rather... interesting, and I muddled through on the basis of getting some more interview practice assuming it was a no go.
To my amazement, got a phone call saying that they'd been thinking they wanted someone more technical on the team, and despite my lack of scientific experience they thought I'd be a good fit. I was unsure as to whether I should accept for a while, but then decided to just go for it - and had a fantastic internship there, working on a great variety of stuff, and learning tons all under a supervisor who I'm still in touch with to this day.
tl;dr - Applied for the wrong job. Coincidentally got it anyway, and miraculously had a fantastic year working there.8
One comment from @Fast-Nop made me remember something I had promised myself not to. Specifically the USB thing.
So there I was, Lieutenant Jr at a warship (not the one my previous rants refer to), my main duties as navigation officer, and secondary (and unofficial) tech support and all-around "computer guy".
Those of you who don't know what horrors this demonic brand pertains to, I envy you. But I digress. In the ship, we had Ethernet cabling and switches, but no DHCP, no server, not a thing. My proposition was shot down by the CO within 2 minutes. Yet, we had a curious "network". As my fellow... colleagues had invented, we had something akin to token ring, but instead of tokens, we had low-rank personnel running around with USB sticks, and as for "rings", well, anyone could snatch up a USB-carrier and load his data and instructions to the "token". What on earth could go wrong with that system?
We got 1 USB infected with a malware from a nearby ship - I still don't know how. Said malware did the following observable actions(yes, I did some malware analysis - As I said before, I am not paid enough):
- Move the contents on any writeable media to a folder with empty (or space) name on that medium. Windows didn't show that folder, so it became "invisible" - linux/mac showed it just fine
- It created a shortcut on the root folder of said medium, right to the malware. Executing the shortcut executed the malware and opened a new window with the "hidden" folder.
Childishly simple, right? If only you knew. If only you knew the horrors, the loss of faith in humanity (which is really bad when you have access to munitions, explosives and heavy weaponry).
People executed the malware ON PURPOSE. Some actually DISABLED their AV to "access their files". I ran amok for an entire WEEK to try to keep this contained. But... I underestimated the USB-token-ring-whatever protocol's speed and the strength of a user's stupidity. PCs that I cleaned got infected AGAIN within HOURS.
I had to address the CO to order total shutdown, USB and PC turnover to me. I spent the most fun weekend cleaning 20-30 PCs and 9 USBs. What fun!
What fun, morons. Now I'll have nightmares of those days again.9
--- GitHub 24-hour outage post mortem ---
As many of you will remember; Github fell over earlier this month and cracked its head on the counter top on the way down. For more or less a full 24 hours the repo-wrangling behemoth had inconsistent data being presented to users, slow response times and failing requests during common user actions such as reporting issues and questioning your career choice in code reviews.
It's been revealed in a post-mortem of the incident (link at the end of the article) that DB replication was the root cause of the chaos after a failing 100G network link was being replaced during routine maintenance. I don't pretend to be a rockstar-ninja-wizard DBA but after speaking with colleagues who went a shade whiter when the term "replication" was used - It's hard to predict where a design decision will bite back and leave you untanging the web of lies and misinformation reported by the databases for weeks if not months after everything's gone a tad sideways.
When the link was yanked out of the east coast DC undergoing maintenance - Github's "Orchestrator" software did exactly what it was meant to do; It hit the "ohshi" button and failed over to another DC that wasn't reporting any issues. The hitch in the master plan was that when connectivity came back up at the east coast DC, Orchestrator was unable to (un)fail-over back to the east coast DC due to each cluster containing data the other didn't have.
At this point it's reasonable to assume that pants were turning funny colours - Monitoring systems across the board started squealing, firing off messages to engineers demanding they rouse from the land of nod and snap back to reality, that was a bit more "on-fire" than usual. A quick call to Orchestrator's API returned a result set that only contained database servers from the west coast - none of the east coast servers had responded.
Come 11pm UTC (about 10 minutes after the initial pant re-colouring) engineers realised they were well and truly backed into a corner, the site was flipped into "Yellow" status and internal mechanisms for deployments were locked out. 5 minutes later an Incident Co-ordinator was dragged from their lair by the status change and almost immediately flipped the site into "Red" status, a move i can only hope was accompanied by all the lights going red and klaxons sounding.
Even more engineers were roused from their slumber to help with the recovery effort, By this point hair was turning grey in real time - The fail-over DB cluster had been processing user data for nearly 40 minutes, every second that passed made the inevitable untangling process exponentially more difficult. Not long after this Github made the call to pause webhooks and Github Pages builds in an attempt to prevent further data loss, causing disruption to those of us using Github as a way of kicking off our deployment processes (myself included, I had to SSH in and run a git pull myself like some kind of savage).
Glossing over several more "And then things were still broken" sections of the post mortem; Clever engineers with their heads screwed on the right way successfully executed what i can only imagine was a large, complex and risky plan to untangle the mess and restore functionality. Github was picked up off the kitchen floor and promptly placed in a comfy chair with a sweet tea to recover. The enormous backlog of webhooks and Pages builds was caught up with and everything was more or less back to normal.
It goes to show that even the best laid plan rarely survives first contact with the enemy, In this case a failing 100G network link somewhere inside an east coast data center.
Link to the post mortem: https://blog.github.com/2018-10-30-...7
I’m so mad I’m fighting back anger tears. This is a long rant and I apologize but I’m so freaking mad.
So a few weeks ago I was asked by my lead staff person to do a data analysis project for the director of our dept. It was a pretty big project, spanning thousands of users. I was excited because I love this sort of thing and I really don’t have anything else to do. Well I don’t have access to the dataset, so I had to get it from my lead and he said he’d do it when he had a chance. Three days later he hadn’t given it to me yet. I approach him and he follows me to my desk, gives me his login and password to login to the secure freaking database, then has me clone it and put it on my computer.
So, I start working on it. It took me about six hours to clean the database, 2 to set up the parameters and plan of attack, and two or three to visualize the data. I realized about halfway through that my lead wasn’t sure about the parameters of the analysis, and I mentioned to him that the director had asked for more information than what he was having me do. He tells me he will speak with director.
So, our director is never there, so I give my lead about a week to speak with her, in the mean time I finish the project to the specifications that the director gave. I even included notes about information that I would need to make more accurate predictions, to draw conclusions, etc. It was really well documented.
Finally, exasperated, and with the project finished but just sitting on my computer for a week, I approached my director on a Saturday when I was working overtime. She confirmed that I needed to what she said in the project specs (duh), and also mentioned she needed a bigger data set than what I was working with if we had one. She told me to speak to my lead on Monday about this, but said that my work looked great.
Monday came and my lead wasn’t there so I spoke with my supervisor and she said that what I was using was the entire dataset, and that my work looked great and I could just send it off. So, at this point 2/3 of my bosses have seen the project, reviewed it, told me it was great, and confirmed that I was doing the right thing.
I sent it off to the director to disseminate to the appropriate people. Again, she looked at it and said it was great.
A week later (today) one of the people that the project was sent to approaches me and tells me that i did a great job and thank you so much for blah blah blah. She then asks me if the dataset I used included blahblah, and I said no, that I used what was given to me but that I’d be happy to go in and fix it if given the necessary data.
She tells me, “yeah the director was under the impression that these numbers were all about blahblah, so I think there was some kind of misunderstanding.” And then implied that I would not be the one fixing the mistake.
I’m being taken off of the project for two reasons: 1. it took to long to get the project out in the first place,
2. It didn’t even answer the questions that they needed answered.
I fucking told them in the notes and ALL THROUGH THE VISUALIZATIONS that I needed additional data to compare these things I’m so fucking mad. I’m so mad.15
UPDATE: devRant Trans-Oceanic Journey Community Project
It was a mere 12 days ago that I asked the question; 'Could devRanters, as a community, build a 21st Century Technology-Laden ‘devRant devie-Stressball-in-a-Bottle’ and send it on a journey across the Atlantic ocean?
I am thrilled to report that devRanters enthusiastically accepted this difficult challenge. A core team quickly formed and a tremendous amount of research and progress has been made in a short period of time. I want to give you a high level-flavor of what we are doing. Please keep in mind we still need your help. We welcome all develops to take part in this journey.
I want to give appreciation to the devRant Founders @dfox and @trogus. Without your support and sponsorship this project would not have been possible. devRant brought us together and it a reality. Devie journeying across the Ocean the Columbus sailed will stir the imagination of children and adults worldwide when we launch on May 1, 2017.
Some of the research and action items in progress:
- Slack and trello environments were created to capture research and foster discussion.
- A Stony Brook University Oceanography Professor suggested the Gulf Stream would be a good pathway across the ocean. We researched it very and agree. The Gulf Stream has been a trans-Atlantic conduit for hundreds of years. We are deciding whether to launch from Cape Hatteras, NC or the Virginia coast. Both have easy access to the rapid currents in the Gulf Stream.
- We are researching every detail of the Gulf Stream to make the journey easier and faster for devie. We have maps and a team member gathered valuable ideas reading a thorough book – ‘The Gulf Stream’.
- We decided on using a highly resilient plastic rather than glass for the bottle material. Plastic is much lighter, faster and glass breaks down more easily. The lightweight enclosure will allow us to take full advantage of waves and ample trade winds. We are still discussing the final design as we want to minimize friction and mimic the non-locomotion fish that migrate thousands of miles riding the Gulf Stream.
-The enclosure might be 3D printed unless we can locate a commercial solution. We have 3D specs and are speaking with some experts. There are advantages and dis-advantages to each solution.
- We will be using Iridiums' RockBLOCK two-way satellite technology to bounce lat-long coordinate pings off their 36 low-orbit satellites. The data will be analyzed by our devRant devie analysis software. IOS and Android public apps being built by the team will display devie's location throughout the journey in.
- Arduino will be used as the brains
- Multiple sensors including temperature and depth are being considered
-A project plan will be published to the team Friday 12/9. Sorry I am a few days late but adding some new ideas.
There are still a lot of challenges we must overcome and we will.
That’s all for now. I will send updates and all ideas / comments are valued.6
Two years ago I moved to Dublin with my wife (we met on tour while we were both working in music) as visa laws in the UK didn’t allow me to support the visa of a Russian national on a freelance artists salary.
After we came to Dublin I was playing a lot to pay rent (major rental crisis here), I play(ed) Double Bass which is a physically intensive instrument and through overworking caused a long term injury to my forearm which prevents me playing.
Luckily my wife was able to start working in Community Operations for the big tech companies here (not an amazing job and I want her to be able to stop).
Anyway, I was a bit stuck with what step to take next as my entire career had been driven by the passion to master an art that I was very committed to. It gave me joy and meaning.
I was working as hard as I could with a clear vision but no clear path available to get there, then by chance the opportunity came to study a Higher Diploma qualification in Data Science/Analysis (I have some experience handling music licensing for tech startups and an MA with components in music analysis, which I spun into a narrative). Seemed like a ‘smart’ thing to do to do pick up a ‘respectable’ qualification, if I can’t play any more.
The programme had a strong programming element and I really enjoyed that part. The heavy statistics/algebra element was difficult but as my Python programming improved, I was able to write and utilise codebase to streamline the work, and I started to pull ahead of the class. I put in more and more time to programming and studied personally far beyond the requirements of the programme (scored some of the highest academic grades I’ve ever achieved). I picked up a confident level of Bash, SQL, Cypher (Neo4j), proficiency with libraries like pandas, scikit-learn as well as R things like ggplot. I’m almost at the end of the course now and I’m currently lecturing evening classes at the university as a paid professional, teaching Graph Database theory and implementation of Neo4j using Python. I’m co-writing a thesis on Machine Learning in The Creative Process (with faculty members) to be published by the institute. My confidence in programming grew and grew and with that platform to lift me, I pulled away from the class further and further.
I felt lost for a while, but I’ve found my new passion. I feel the drive to master the craft, the desire to create, to refine and to explore.
I’m going to write a Thesis with a strong focus on programmatic implementation and then try and take a programming related position and build from there. I’m excited to become a professional in this field. It might take time and not be easy, but I’ve already mastered one craft in life to the highest levels of expertise (and tutored it for almost 10 years). I’m 30 now and no expert (yet), but am well beyond beginner. I know how to learn and self study effectively.
The future is exciting and I’ve discovered my new art! (I’m also performing live these days with ‘TidalCycles’! (Haskell pattern syntax for music performance).
Hey all! I’m new on devRant!12
more buzzword translations with a story (because the last one was pretty well liked):
"machine learning" -> an actual, smart thing, but you generally don't need any knowledge to use it as they're all libraries now
"a bitcoin" -> literally just a fucking number that everyone has
"powerful" -> it's umm… almost working (seriously i hate this word, it really has a meaning of null)
"hacking" -> watching a friend type in their facebook password with a black hoodie on, of course (courtesy of @GeaRSiX)
"cloud-based service" -> we have an extra commodore 64 and you can use it over the internet for an ever-increasing monthly fee
"analysis" -> two options: "it's not working" or "its close enough"
"stress-free workplace" -> working from home without pants
now for a short story:
a few days ago in code.org "apscp" class, we learnt about how to do "top down design" (of course, whatever works before for you was not in option in solving problems). we had to design a game, as the first "step" of "top down design," we had to identify three things we needed to do to make a game.
graphics is literally a png, but what the fuck do you expect for ai?
we have a game right? oh wait! its getting boring. let's just sprinkle some fucking artificial intelligence on it like i put salt on french fries.
this is complete bullshit.
also, one of my most hated commercials:
"iot data and ai from the cloud"
yeah please shut the fuck up
Unintentionally Hilarious joke at work yesterday.
We were doing some data analysis, and I had to dump some stuff into a table for my colleague. So I ran the script and went to the bathroom (no.2).
When I came back, they asked me if the dump is done. And I said without thinking: "I just went." 😂3
So… I’m in my 3rd week as pro bono "tech lead" at a startup with rather high ambitions but amazingly stupid execution.
A product manager (otherwise really nice as a person if he’s for real but I have my doubts…) who could do well in HR or perhaps politics, instead micromanages every aspect of the business including technical.
A project manager who acts as a psychologist: what did you do? how did you feel about it? how do you think others felt about it? did it make your cock hard or more like half-hard? care to write it down on a post it?
Completely tech illiterate CEO and COO, basically some loaded old geezers who felt like doing something else along with clipping roses around their mansions for the rest of their lives, who come up with shit like “have someone make that button bigger I can’t see it”
A bunch of phd’s who seem to make a shit ton of money on the side from buy-to-pass scams basically charging their students for online courses and books they base exams on, pull them our of MS word and they’re completely tech illiterate, and can barely stay awake during team meetings, I get the sense they're there hoping to sell more of their shit that's been outdated since some 20 years ago
A hippie lead over a team for one “UI/UX engineer”, himself, who can’t design a form, but dismisses any ideas that are based on metrics, analysis and some 15+ years of experience and common sense because “he has more experience” although his portfolio only includes this particular business, a few restaurant menus and a couple other unmaintained websites nobody heard about that probably get more traffic from some crackheads’ typos and chinese robots rather than real users
A "developer" from some shit hole country who barely speaks english and can’t figure out how to start a container but somehow manages to put together some js by copy pasting snippets from other people’s code.
Another developer who’s overworked doing most of the heavy lifting without any comments probably because some sad personal circumstance made him accept the shit job he’s doing right now, basically every time I see him I want to give him some money and some food and some pats on the back
A couple guys' wives who basically do nothing but chit chat in "team meetings". They're not even fuckable.
A data model that doesn’t follow any architecture pattern or logic or common sense even
A frontend framework that has nothing to do with the application that would ideally solve the business requirements of their domain
Some over-engineered backend code that gets you digging through 20 fucking files and concerns and libs to figure out what some random method does
Lots of technical debt rising from poor design and implementation decisions
No documented projects, processes, practices, domain model, basically the process I feed my dog with is more documented than this whole fucking joke of a company
Every constructive suggestion gets dismissed with half a page of mostly meaningless political bullshit
… and so on.
Guess who’s getting back to dicking around and posting shit on devrant soon?10
FUCK THE RECRUITERS WHO ASK US TO MAKE AN ENTIRE PROJECT AS A CODE TEST.
Oh you need to scrape this website and then store the data in some DB. Apply sentimental analysis on the data set. On the UI, the user should be able to search the fields that were scraped from the website. Upon clicking it should consume a REST API which you have to create as well. Oh and also deploy it somewhere... Oh I almost forgot, make the UI look good. If you could submit it in one week, we will move towards further rounds if we find you fit enough.
YOU KNOW WHAT, FUCK YOU!
I can apply to 10 others companies in one week and get hired in half the effort than making this whole project for you which you are going to use it on your website YOU SADIST MOTHERFUCK
I CURSE YOUR COMPANY WITH THE ETERNITY OF JS CALLBACK HELL 😡😤😣9
When scrolling through free Udemy courses for fun, my gf says:
"Whaaaat? Data analysis with Panda and Python? Isn't that shit dangerous? Like they go in the jungle and stuff?"
(Disclaimer: She has no background in CS)5
Currently, a classmate and I are working on our technical thesis.
It is all about industry 4.0, IIoT, big data and stuff.
This week, we presented our interim results to our supervisor. He is very pleased with our work and made the following suggestion:
He thinks it would be awesome to publish our work on our own GitHub repository and make it open source because he is convinced that this thesis is able to kind of "set a new standard" in some specific fields of using big data analysis in production processes.
I guess I'm kind of proud :)4
Man, most memorable has to be the lead devops engineer from the first startup I worked at. My immediate team/friends called him Mr. DW - DW being short for Done and Working.
You see, Mr. DW was a brilliant devops engineer. He came up with excellent solutions to a lot of release, deployment, and data storage problems faced at the company (small genetics firm that ships servers with our analysis software on them). I am still very impressed by some of the solutions he came up with, and wish I had more time to study and learn about them before I left that company.
BUT - despite his brilliance, Mr. DW ALWAYS shipped broken stuff. For some reason this guy thinks that only testing a single happiest of happy path scenarios for whatever he is developing constitutes "everything will work as expected!" As soon as he said it was "done", but golly for him was it "done". By fucking God was that never the truth.
So, let me provide a basic example of how things would go:
my team: "Hey DW, we have a problem with X, can you fix this?"
DW: "Oh, sure. I bet it's a problem with <insert long explanations we don't care about we just want it fixed>"
my team: "....uhh, cool! Looking forward to the fix!"
... however long later...
DW: "OK, it's done. Here you go!"
my team: "Thanks! We'll get the fix into the processing pipelines"
... another short time later...
my team: "DW, this thing is broken. Look at all these failures"
DW: "How can that be? It was done! I tested it and it worked!"
my team: "Well, the failures say otherwise. How did you test?"
DW: "I just did <insert super basic thing>"
my team: "...... you know that's, like, not how things actually work for this part of the pipeline. right?"
DW: "..... But I thought it was XYZ?"
my team: "uhhhh, no, not even close. Can you please fix and let us know when it's done and working?"
DW: "... I'll fix it..."
And rinse and repeat the "it's done.. oh wait, it's broken" a good half dozen times on average. But, anyways, the birth of Mr. Done and Working - very often stuff was done, but rarely did it ever work!
I'm still friends with my team mates, and whenever we're talking and someone says something is done, we just have to ask if it's done AND working. We always get a laugh, sadly at the excuse of Mr. DW, but he dug his own hole in this regard.
Little cherry on top: So, the above happened with one of my friends. Mr. DW created installation media for one of our servers that was deployed in China. He tested it and "it was done!" Well, my friend flies out to China for on-site installation. He plugs the install medium in and goes for the install and it crashes and burns in a fire. Thankfully my friend knew the system well enough to be able to get everything installed and configured correctly minus the broken install media, but definitely the most insane example of "it's done!" but sure as he'll "it doesn't work!" we had from Mr. DW.2
Inmates are trying to take over the asylum again.
Got a message from the web team manager deeply concerned because since switching to the new logging framework, the site is significantly slower.
She provided no proof or any data to what 'significantly slower' means.
#1 The 'new logging' has been in place and logging for 5 years. We only recently depreciated the ILogger interface ('new' ILogger interface only has 1 method instead of 5)
#2 The 'old logging' was modified 5 years ago, so even if you were using the 'old' interface, the underlying implementation is still the same.
She tried to push the 'it wasn't this slow before' argument, so I decided to do some fact based analysis.
Knowing they deployed their logging changes couple of weeks ago, I opened up AppDynamics, looked at the average call time to Splunk (along with a few other http calls they are doing)
- caching services - 5ms
- splunk - 30ms
- Order Service - 350ms
- Product Data Service -525ms
Then I look at the data they are logging, for the month of June, over 5 million messages. At 30ms each, that's almost 42 hours spent logging errors...yes errors. Null reference exceptions, Argument exceptions, easily fixable stuff.
So far for the month of July (using the 'new' logging), almost 2.5 million errors. Pretty close so far with June's numbers.
My only suggestion was to fix the bugs in their code so they don't log so many errors.
Her response.."Can we have one of our developers review your logging code? We believe we can find ways to optimize the http requests"
Oh good Lord. I'm not a drinking man..but ...I might start.2
This week I reached a major milestone in a Machine Learning/Music Analysis project that I've been working on for a long time!!
I'm really proud to launch 'The Harmonic Algorithm' as an open source project! It represents the evolution of something that's grown with me through two thesis' (initially in music analysis and later in creative computation) and has been a vessel for my passion in both Music and Computation/Machine Learning for a number of years.
For more info, detailed usage examples (with video clips) and installation instructions for anyone inclined to try it out, have a look at the GitHub repo for the project:
"The Harmonic Algorithm, written in Haskell and R, generates musical domain specific data inside user defined constraints then filters it down and deterministically ranks it using a tailored Markov Chain model trained on ingested musical data. This presents a unique tool in the hands of the composer or performer which can be used as a writing aid, analysis device, for instrumental study or even in live performance."1
TLDR: crappy api + idiot ex client combo rant // devam si duška
I saw a lot of people bitching about APIs that don't return proper response codes and other stuff..
Well let me tell you a story. I used to work on a project where we had to do something like booking, but better..crossbreed with the Off&Away bidding site (which btw we had to rip off the .js stuff and reverse engineer the whole timer thingy), using free versions of everything..even though money wasn't an issue (what our client said). Same client decided to go with transhotel because it was sooooo gooood... OK? Why did noone heard of them then?
Anyhow, the api was xml based.. we had to send some xml that was validated against a schema, we received another that was supposed to be validated againts another schema.. and so on and so on..
The API docs were nonexistent.. What was there, was broken English or Spanish.. Even had some comments like Add This & that to chapter xy.. Of course that chapter didn't even exist yet. :( And the last documentation they had, was really really old..more than a year, with visible gaps, we got the validation schemas not even listed in the docs, let alone described properly.
Yaaay! And that was not everything.. besides wrong and missing data, the API itself caused the 500 server error whenever you were no longer authenticated.
Of course it didn't tell you that your session was dead.. Just pooof! Unhandled crap everywhere!
And the best part?! We handled that login after inspecting what the hell happened, but sent the notification to the company anyways.. We had a conf call, and sent numerous emails explaining to them what a 'try catch' is and how they should handle the not authenticated error <= BTW they should have had a handled xml response for that, we got the schema for it! But they didn't. Anyhow, after two agonizing days talking back and forth they at least set up the server to be available again after the horrified 500 error. Before, it even stopped responding until reset (don't ask me how they managed to do that).
Oh yeah, did I mention this was a worldwide renown company?! Where everybody spoke/wrote English?! Yup, they have more than 700 people there, of course they speak English! <= another one of my ex clients fabulous statements... making me wanna strangle him with his tie.. I told him I am not talking to them because no-one there understood/spoke English and it would be a waste of my time.. Guess who spent almost 3 hours to talk to someone who sounded like a stereotypical Indian support tech guy with a flue speaking Italian?! // no offence please for the referenced parties!!
So yeah, sadly I don't have SS of the fucked up documentation..and I cannot post more details (not sure if the NDA still holds even though they canceled the project).. Not that I care really.. not after I saw how the client would treat his customers..
Anywayz I found on the interwebz some proof that this shitty api existed..
picture + link: https://programmableweb.com/api/...
SubRant: the client was an idiot! Probably still is, but no longer my client..
Wanted to store the credit card info + cvc and owner info etc.. in our database.. for easier second payment, like on paypal (which he wanted me to totally customize the payment page of paypal, and if that wasn't possible to collect user data on our personalized payment page and then just send it over to paypal api, if possible in plaintext, he just didn't care as long as he got his personalized payment page) or sth.... I told the company owner that they are fucking retards if they think they can pull this off & that they will lose all their (potential) clients if they figure that out.. or god forbid someone hacked us and stole the data.. I think this shit is also against the law..
I think it goes without saying what happened next.. called him ignorant stupid fucktard to his face and told him I ain't doing that since our company didn't even had a certificate to store the last 4 numbers.. They heard my voice over the whole firm.. we had fish-tank like offices, so they could all see me yelling at the director..
Guess who got laid off due to not being needed anymore the next day?! It was the best day of my life..so far!! Never have I been happier to lose my job!!
P.S. all that crap + test + the whole backand for analysis, the whole crm + campaign emails etc.. the client wanted done in 6 months.. O.o
P.P.S. almost shat my pants when devRant notified my I cannot post and wanted to copy the message and then everything disappeard.. thank god I have written this in the n++ xD11
Indian outsourcing web companies on their websites be like:
"We provide services in IOT, Machine Learning, Data Analysis, Web security analysis, blah blah"
Me excitingly joins the company and finds out, they all are PHP developers developing unsecured websites.1
First year: intro to programming, basic data structures and algos, parallel programming, databases and a project to finish it. Homework should be kept track of via some version control. Should also be some calculus and linear algebra.
Introduce more complex subjects such as programming paradigms, compilers and language theory, low level programming + logic design + basic processor design, logic for system verification, statistics and graph theory. Should also be a project with a company.
Advanced algos, datastructures and algorithm analysis. Intro to Computer and data security. Optional courses in graphics programming, machine learning, compilers and automata, embedded systems etc. ends with a big project that goes in depth into a CS subject, not a regular software project in java basically.4
Spec'ing out a data analysis tool for several billion transactions and I got this when sending the final spec to the client.
Client: "I don't see why it takes so long, I downloaded this template from theme forest and edited it in dreamweaver, it only took 3 hours"2
If anyone has been keeping up with my data warehouse from hell stories, we're reaching the climax. Today I reached my breaking point and wrote a strongly worder email about the situation. I detailed 3 separate cases of violated referential integrity (this warehouse has no constraints) and a field pulling from THE WRONG FLIPPING TABLE. Each instance was detailed with the lying ER diagram, highlighted the violating key pairs, the dangers they posed, and how to fix it. Note that this is a financial document; a financial document with nondeterministic behavior because the previous contractors' laziness. I feel like the flipping harbinger of doom with a cardboard sign saying "the end is near" and keep having to self-validate that if I was to change anything about this code, **financial numbers would change**, names would swap, description codes would change, and because they're edge cases in a giant dataset, they'll be hard to find. My email included SQL queries returning values where integrity is violated 15+ times. There's legacy data just shoved in ignoring all constraints. There are misspellings where a new one was made instead of updating, leaving the pk the same.
Now I'd just put sorting and other algos, but the data is processed by a crystal report. It has no debugger. No analysis tools. 11 subreports. The thing takes an hour to run and 77k queries to the oracle backend. It's one of the most disgusting infrastructures I've ever seen. There's no other solution to this but to either move to a general programming language or get the contractor to fix the data warehouse. I feel like I've gotten nowhere trying to debug this for 2 months. Now that I've reached what's probably the root issue, the office beaucracy is resisting the idea of throwing out the fire hazard and keeping the good parts. The upper management wants to just install sprinklers, and I'm losing it.
What you are expected to learn in 3 years:
digital signal processing,
signal and control system,
NLP, data algorithm,
Java, C++, Python,
ASP.NET web development,
computer security ,
Android app development,
IOS app development,
3D game development,
introduction to DevOps,
how-to -fix- computer,
Project of being entrepreneur,
and 24 random unrelated subjects of your choices
This is a major called "computer engineering"4
Fuck you Intel.
Fucking admit that you're Hardware has a problem!
"Intel and other technology companies have been made aware of new security research describing software analysis methods that, when used for malicious purposes, have the potential to improperly gather sensitive data from computing devices that are operating as designed. Intel believes these exploits do not have the potential to corrupt, modify or delete data"
With Meltdown one process can fucking read everything that is in memory. Every password and every other sensible bit. Of course you can't change sensible data directly. You have to use the sensible data you gathered... Big fucking difference you dumb shits.
Meltown occurs because of hardware implemented speculative execution.
The solution is to fucking separate kernel- and user-adress space.
And you're saying that your hardware works how it should.
Shame on you.
I'm not saying that I don't tolerate mistakes like this. Shit happens.
But not having the balls to admit that it is because of the hardware makes me fucking angry.5
Full stack developer.
I know what it's supposed to mean, but I feel like it gives discredit to the devs who perfect their area (frontend, backend, db, infrastructure). It's, to me, like calling myself a chef because I can cook dinner..
The depth, analysis and customization of the domain to shape an api to a website is never appreciated. The finicle tweaks on the frontend to make those final touches. Then comes a brat who say they are full stack, and can do all those things. Bullshit. 99.9% of them have never done anything but move data through layers and present it.
Throw these wannabes an enterprise system with monoliths and microservices willy nelly, orchestrate that shit with a vertical slice nginx ssi with disaster recovery, horizontal scaling, domain modeling, version management, a busy little bus and events flowing all decimal points of 2pi. Then, if you fully master everything going on there, I believe you are full stack.
Otherwise you just scraped the surface of what complexities software development is about. Everyone who can read a tutorial can scrape together an "in-out" website. But if your db is looking the same as your api, your highest complexity is the alignment of an infobox, I will laugh loud at your full stack.
And if you told me in an interview that you are full stack, you'd better have 10+ years experience and a good list of failed and successful projects before I'd let you stay the next two minutes..1
So according to Pfizer, ~39,000 people received a second dose of vaccine/placebo. So about 20,000 got the placebo, and out of that group only 85 got infected in the span of 3.5 months, during the second wave. That's 0.425%.
So they either:
* Proved Covid is not infectious
* Manipulated the shit out of their data
How that fuck is this considered valid analysis?!
If I tried to publish a paper based on such unreliable, statistically meaningless data, I would be rejected without a blink.64
Been at a startup for 2-3 weeks, hood some presentations about Linux and Foss stuff but they didn't have any work for anyone and as such didn't pay anything.
So a colleague was like "hey man, I seed you making those presentations and stuff but not getting anything back, I think I might got a job for you".
And now I do small data analysis at another company. And get money.
i want to get my own social network up and running.
so far ive got -
login 100% securely
register (1000% securely)
view someone’s profile (10^7% securely)
to add -
scrypt (maybe bcrypt, however scrypt looks like the better option)
friend a user
track their every move (ill use facebooks and googles apis for that)
to describe my product -
call me when im a gazillionaire
but seriously, im making a social network and i hope its done by wk105 tbh3
After you automate away task into a tool or program, do you tell your boss or keep it to yourself so they remain amazed at your efficiency and productivity?
And for data analysis ones, that you have godly data processing skills as you can somehow read a 1,000,000 line log file and find the root cause and other insights in an hour or minutes?5
2020 seems to be the year of the "dev who has never seen scale."
TypeA -> "Here's a reasoned explanation for a change I think we should make. Here is the current deficiency analysis, here is the desired resolution, here is the course of action and all calculations leading to the resolution + data. This will have x,y,z beneficial result according to our operational metrics."
TypeD -> "Those were words. Why do you need that? Change is bad, learning is worse. This will just slow me down, development speed is all that matters; there is no chance that a poorly considered/factored/checked design could ever require a ground up rewrite or fuck us utterly in the long term. Why do you make my life harder? We could x -> y -> zBUTI haven't done the math and I really don't see the benefit in x, so z is pointless. What even is scale?"
The consequences of the war caused by the ever-widening gap between engineers and developers is low key terrifying.13
TL;DR: academic survey over devRant, 5-7 minutes https://forms.gle/do2KK8cGfv5w6cjY9
We are a group of researchers from Canada, Italy, and the Netherlands, studying communication between software developers. We would like to understand the role devRant plays in developers' professional life and the perceived advantages and disadvantages of the platform.
To this end we created an overview of the topics discussed. The purpose of this survey is to get your opinion on the overview. The results of the survey will be reported in a research manuscript, which will be submitted for a peer-reviewed publication.
We thank you for your participation.
Foutse Khomh, Nicole Novielli, Moses Openja, Alexander Serebrenik, Gias Uddin28
I fixed my big data processing code, I think. If all works as planned, I'll wake up to some processed data that I can do some statistical analysis on... I hope...8
Small company, sole engineer. Non-tech management. Increasingly fancy job titles despite working alone most of the time, with the promise of hiring someone (again) I can actually manage soon.
Backlog of projects/tasks is truly a mindfuck, with new things being added each week. This backlog will never ever get done, and nothing matters anyway because the next idea is "the future", all the time.
While I have influence on some aspects of decision making, it usually ends up being what the boss wants. Actively opposed a project because it's just too big of an undertaking, it was forced through anyway. I'm trying to keep the scope manageable as I'm building it now, and it's hard.
"It's the future, we absolutely have to do this. It will be the biggest thing we've ever done."
Boss's excitement then quickly faded since it's actually in development, now nobody really seems to want to know where it's at, or how it will all work. I need to scope it out, with the knowledge that many decisions boss signed off will be questioned when he actually looks at it. We now have even more "exciting" ideas of utter grandeur. Stuff that I can't even begin to comprehend the complexity of, while struggling to keep a self imposed deadline on the current one.
Every single morning we sit on Zoom for a "valuable" "catch-up". This is absolutely perfect for one thing: Completely destroying whatever drive and focus I have going into the day. Unrelated topics, marketing conversations, even more ideas, ideas for ideas sake, small problems blown out of proportion, the list goes on. I recently argued in detail why it should be scrapped or at least be optional to attend. No luck, it's "valuable".
Today a new idea was announced, and we absolutely have to do it ASAP because it can only be better than the current solution. I raise my concerns, saying it's not as easy as you make it out to be, we should properly think about it. Nope! We'll botch something to prove that it works... So you'll base your decision whether it's good on some half ass botch job that nobody really has the mental capacity to actually pay attention to. What a reliable way to measure!
"Our analytics data isn't useful enough to tell us the impact of things we do. We (you) have to fix this." Over the last 2 or so years, I've been pushing for an overhaul and expansion of our data analysis capabilities for exactly this reason. Integrating different data sources into a unified solution so we can easily see what we're doing, etc. Nope, never happened.
The new project idea which is based on wild assumptions is ALWAYS more important than the groundwork.
Now when I mentioned that this is what I wanted to do all along, it got brushed aside. "We don't need to do anything complicated, just fix this, add that, and it's done. It should be an easy thing to do. This is very important for our decision making." Fine, have it your way.
I'm officially burned out. It's so fucking hard to get myself to focus on my work for more than an hour or two. I started a side project, and even that effort is falling victim to my day-job-induced apathy.
I'm tempted to hand in my resignation without another offer on the table. I just need time to rediscover my passion, and go job hunting from that position, instead of the utter desperation of right now.
If you've read through all this rambling, kudos to you!8
When someone doesn't send raw data with their analysis, I feel like they have something to hide ...1
For starters this is my first post, found devrant the other day you're all hilarious.
I hate math, I hate proofs. I'm in a class "Analysis of Algorithms" and I have understood and do understand the importance of optimizing algorithms and data structures and I understand the algorithms and data structures themselves. That being said, I'm fucking sick of math and proofs and all this bullshit that is probably pretty important but ugggghh, I guess I just have to push through, but writing this out helped.18
During one of our visits at Konza City, Machakos county in Kenya, my team and I encountered a big problem accessing to viable water. Most times we enquired for water, we were handed a bottle of bought water. This for a day or few days would be affordable for some, but for a lifetime of a middle income person, it will be way too much expensive. Of ten people we encountered 8 complained of a proper mechanism to access to viable water. This to us was a very demanding problem, that needed to be sorted out immediately. Majority of the people were unable to conduct income generating activities such as farming because of the nature of the kind of water and its scarcity as well.
Such a scenario demands for an immediate way to solve this problem. Various ways have been put into practice to ensure sustainability of water conservation and management. However most of them have been futile on the aspect of sustainability. As part of our research we also considered to check out of the formal mechanisms put in place to ensure proper acquisition of water, and one of them we saw was tree planting, which was not sustainable at all, also some few piped water was being transported very long distances from the destinations, this however did not solve the immediate needs of the people.We found out that the area has a large body mass of salty water which was not viable for them to conduct any constructive activity. This was hint enough to help us find a way to curb this demanding challenge. Presence of salty water was the first step of our solution.
We came up with an IOT based system to help curb this problem. Our system entails purification of the salty water through electrolysis, the device is places at an area where the body mass of water is located, it drills for a suitable depth and allow the salty water to flow into it. Various sets of tanks and valves are situated next to it, these tanks acts as to contain the salty water temporarily. A high power source is then connected to each tank, this enable the separation of Chlorine ions from Hydrogen Ions by electrolysis through electrolysis, salt is then separated and allowed to flow from the lower chamber of the tanks, allowing clean water to from to the preceding tanks, the preceding tanks contains various chemicals to remove any remaining impurities. The whole entire process is managed by the action of sensors. Water alkalinity, turbidity and ph are monitored and relayed onto a mobile phone, this then follows a predictive analysis of the data history stored then makes up a decision to increase flow of water in the valves or to decrease its flow. This being a hot prone area, we opted to maximize harnessing of power through solar power, this power availability is almost perfect to provide us with at least 440V constant supply to facilitate faster electrolysis of the salty water.
Being a drought prone area, it was key that the outlet water should be cold and comfortable for consumers to use, so we also coupled our output chamber with cooling tanks, these tanks are managed via our mobile application, the information relayed from it in terms of temperature and humidity are sent to it. This information is key in helping us produce water at optimum states, enabling us to fully manage supply and input of the water from the water bodies.
By the use of natural language processing, we are able to automatically control flow and feeing of the valves to and fro using Voice, one could say “The output water is too hot”, and the system would respond by increasing the speed of the fans and making the tanks provide very cold water. Additional to this system, we have prepared short video tutorials and documents enlighting people on how to conserve water and maintain the optimum state of the green economy.
IBM/OPEN SOURCE TECHNOLOGIES
For a start, we have implemented our project using esp8266 microcontrollers, sensors, transducers and low payload containers to demonstrate our project. Previously we have used Google’s firebase cloud platform to ensure realtimeness of data to-and-fro relay to the mobile. This has proven workable for most cases, whether on a small scale or large scale, however we meet challenges such as change in the fingerprint keys that renders our device not workable, we intend to overcome this problem by moving to IBM bluemix platform.
We use C++ Programming language for our microcontrollers and sensor communication, in some cases we use Python programming language to process neuro-networks for our microcontrollers.
Any feedback conserning this project please?8
There was this project where a bunch a scripts had been running for three weeks analysing a bunch of fileshares. The project was in overrun and the analysis wasn't anywhere near done.
I was given complete isolation and a team of people who were instructed to do anything I needed. I had them replicate the data to as many machines as possible and I started scripting the analysis with some sample data. After half a day of collecting laptops, desktops and severs I transfered my scripts to those machines and ran the analysis in 5 hours.
I felt like I saved the project.
A time I (almost) screamed at co-worker?
Too many times to keep up with.
Majority of time its code like ..
using (var connection = new SqlConnection(connectionString))
// data access code that does stuff
catch (Exception e)
// Various ways of dealing with the error such as ..
ShowMessage("An error occured.");
// or do nothing.
Range of excuses
- Users can't do anything about the error, so why do or show them anything?
- I'll fix the errors later
- Handling the errors were not in the end-user specification. If you want it, you'll have to perform a cost/benefit analysis, get the changes approved by the board in writing, placed in the project priority queue ...etc..etc
- I don't know.
- Users were tired of seeing database timeout errors, deadlocks, primary key violations, etc, so I fixed the problem.
On my tip of my tongue are rages of ..
"I'm going to trade you for a donkey, and shoot the donkey!"
"You are about as useful as a sack full of possum heads."
I haven't cast those stones (yet). I'll eventually run across my code that looks exactly like that.1
So I joined a course for big data analysis. And they setup a lab specifically for us. Pulling us away from the usual computer labs
AND GUESS WHAT THEY DON'T EVEN BLOODY HAVE MYSQL INSTALLED. THEY'VE CHARGED ME A FORTUNE AND THEY DIDN'T EVEN CARE TO INSTALL THE BASIC SOFTWARES AND ITS ALREADY BEEN 2 WEEKS WITH THE START OF THE COURSE AND NOTHING.
F**king hate this man!!!!10
Last week I received an invitation to lead the development of a e-commerce redesign, replatforming and data migration. I was excited to work on it, and started the analysis and planning, glad to spend time focusing on quality. But Murphy's law is never asleep - this Monday, I was asked to speed things up and reduce a 4 month project to 1 week.
Best: learned to code, started writing smart-home scripts for home automation and developed biologic and quantitative data analysis scripts in Python and R.
Worst: didn't get paid to develop them and haven't got enough experience for it to be more than a hobby.
[story of your first dev project] - i really think there should be a headline like that for wk rants
Anyways, it was a while back, le college teacher approached my friend and me asking if we wanted to do a project. We said sure, it was a medium sized data analysis project. We got the specs with a lot of formulas, basically implement them all and make a web frontend, thats it. Took like a year but we did it. Few months later teacher is furious because the calculations didnt give him data that he expected (by expect i mean he thought that a distribution formula would accurately yield 200+ data fields from around 4) and blamed it on us. Not the retard other professor who fucked up half of the formulas. Ok.1
So we’ve taken over from a project team that disbanded... read: “cut their contracts because fuck this, I can earn more working for better people”.
Me and one other guy have been tasked with saving this heap of shit.
Obviously the project guys left saying “it’s nearly done, just this one feature”. Because cut contracts are easier to deal with if “everything is almost done”.
We jump on and find that’s not the case at all... this thing, is a beast, a big old stats analysis program... so we’re like “cool, let’s see what’s going o...OH MY GOD”.
The “recalculation” function was core to this POS. The contractors had done it in C# through entity framework... it took 24 hours to run, over a reasonably small data set that was due to double every 2-5 years.
So... here’s the deal, it ran over night.... then failed. And no cunt had noticed. Entity framework “can’t commit because I’m muddled up as fuck, did you really just put the whole db in EF in memory to work with it?” Exception.
Que 6 months of me and my lead doing the job properly.
Anyway, the failure: I ended up in Hospital again with a Crohn’s flare up... about 5 months in.
Fuckall to do with all this nonsense I just wanted to tell a story. it was an interesting/fun project to fix and my lead was a legend... so happy days.
Similar story, different set of contracted devs... they’d been defining requirements with the business users using the term “Risk” which the business users knew as a group of risks.
The domain model had been written RiskGroup<>—
QA personal voice assistant that runs locally without cloud, it’s like never ending project. I look at it from time to time and time pass by. Chat bots arrived, some decent voice algorithms appeared. There is less and less stuff to code since people progress in that area a lot.
I want to save notes using voice, search trough them, hear them, find some stuff in public data sources like wikipedia and also hear that stuff without using hands, read news articles and stuff like that.
I want to spend, more time for math and core algorithms related to machine learning and deep learning.
Problem is once I remember how basic network layers, error correction algorithms work or how particular deep learning algorithm is constructed and why is that, it’s already a week passed and I don’t remember where I started.
I did it couple of times already and every time I remember more then before but understanding core requires me sitting down with pen and paper and math problems and I don’t have time for that.
Now when I’m thinking about it - maybe I should write it somewhere in organized way. Get back to blogging and write articles about what I learned. This would require two times the time but maybe it would help to not forget.
I’m mostly interested in nlp, tts, stt. Wavenet, tacotron, bert, roberta, sentiment analysis, graphs and qa stuff. And now crystallography cause crystals are just organized graphs in 3d.
Well maybe if I’m lucky I retire in the next decade or at least take a year or two years off to have plenty of time to finish this project.
I need to estimate how much ram and CPUs my team will need next year for our apps... That have yet to be built.
We load a lot of data feeds with batch processes running on a few large machines, some can use like 30 GB RAM at times...) which should be a lot less if we get the data real time I hope...
But wondering how to estimate well... I sorta did a worse case analysis where I just multiply and sum # CPUs/memory* nodes*approx apps...
Comes out to be like 600 CPUs and 800 RAM... So wondering if that's ok...
RAM is ok but # of CPUs is way higher bc now all the apps basically run on their own machines...16
I used to be a big security guy, not allowing stuff like most of the social media, not bringing my phone anywhere, carrying a RPi tablet for privacy reasons. Very Stallman stuff.
Recently I noticed that I don't care so much.. I see these things as opportunities, for instance Microsoft products could be benefitial for job opportunities, I have some workout sessions on my phone.
I could restrict myself... but is it worth it just to decline some capitalist/politician's row in a dataset for analysis?
But then again I feel as a society I think we should either do this or request this data to be distributed to us as well.
Should you be playing a game of cards, when the enemy can see your hand? What do u think?6
The futility of working in a bank, or probably any other big corporation.
I was working in a small dev team that made improvements to the applications and processes of the department we were in. I was given the project of creating a small data entry system to track some communications. The comms were currently stored in a spreadsheet and had no validation which made any analysis of the data difficult.
Boring but easy, should have taken about a week.
We had 3 meetings just to define requirements, they took about a month to send me their shitty little spreadsheet so I could actually see what they were recording. Over the next 2 months they kept changing what they wanted. Finally got it finished just before I went on maternity leave.
Went off thinking, that I’ll never have to deal with that clusterfuck again.
Had a baby, went back 9 months later.
First thing I got given was that shitty project back. They hadn’t even tested it - in 9 fucking months! I transferred departments not long after. I have no idea if it ever got done.
And that is on my list of reasons as to why I’ll never work in a large corporate again.3
Time to switch to offline and hide in some dark corner to get work done. Tired of all the IM’s and coming over to my desk from 1 person for “critical” work. If they’re all critical then none of them are truly critical. If you sit on the data for 2 months, and then today is the day it becomes critical and the compliance issue is because of your ineptitude then its a you problem not an IT problem. Then on top of that you submit your data to be loaded in the incorrect request form and spreadsheet format you can go fuck yourself asking this be done in an hour. It could be done in 15 minutes if you had it in the correct format as specified in the 20 meetings over the past year which removed all manual analysis and automated the entire process you idiot. Now I have to get it into the correct format in that hour so I don’t have to do the analysis for you.
I have other things to do besides your etl tickets, like finding the actual problems in our actual critical applications. You know the ones where the VP’s of this giant corporation start calling if they go down.
Sorry for the rambling guys.
One problem for CS education is that the salaries for academia are so low compared to industry that if someone is even vaguely competent, they can at least double their income by working a 'real' job. Now this may be different at higher levels of colleges but generally those folks are such bullshitters they wouldn't last outside of academia.
As what to improve?
Depends if it's a research or vocational course.
Basic knowledge of algorithms, runtime analysis (O notation) and some data structures and you're an instahire.
For research, go wild. Deep dive into the math, algorithmic side. Read up lots of research papers. Try out different programming paradigms. You would aim for a career in academia, AI, quant finance etc...2
i was learning neural networks, started with keras and was on the first tutorial where they started by importing pandas
so i switched to learning data analysis using pandas in Python where they started by importing matplotlib and i realized data visualization is also important and now I'm reading matplotlib docs...🙄11
I want to do something data-science-y.
Gimme project ideas, and where can I get the data for it?
Also, not looking for machine learning, just basic data analysis stuff.
So a team at on-site sent a OOM(Out of Memory) issue in our morning.
Everyone analysed the issue as being code issue since we were bringing too much data into the runtime process. The analysis was done on the heap dumps. The number of records reported by user were 1k. At the end of the day it turns the number of records were actually 100k+.
Why do people jump.to conclusions without thinking about the obvious. :-(1
I think another intriguing job asides programming is engineering (*for some*). A week has past and I've been on the hike assisting my beloved brother on his contracted engineering job while I am less occupied. The job is based on 🗼Tower analysis and It's quite risky as you'd have to climb up to 56 meters high just to take readings of antennas, and fix some other stuffs. The only thing I find intriguing about this job is his love for it, funny enough he also thinks I love the job too and I guess I'm guilty for his thoughts (*Sorry bro, I love the job for you not me*).
With my little experience so far on my *new brotherly job* I noticed the most hectic task isn't going up and down the tower taking readings but at the end of all operations, he'll have to gather the values and snapshots he took while on the tower to prepare reports on msword & excel for the other buttwags at the office (or home I guess)
then archive and sends via mail. Seeing this lengthy process I was forced to ask why he wasn't using any reporting tool like Jotforms or any other equivalent and I was willing to look up some recommendations for him, his reply was: "I'm already used to this form of reporting, its what I was trained with and what the company provided, nevertheless a friend of mine suggested something of such weeks back but I would have to pay monthly fee for its usage which is quite on the high side and I don't think I'd prefer that."
Sounds convincing but not enough, okay here is another deal: You use an android phone right? and at my office we work on system automation (*basically does not know what I do for a living probably thinks I'm a hacker the illegal one*), how about i design you an android app for you to capture the tower data and a PC software for you to auto generate the msword & excel reports, I can get this ready for you in less than 5 nights (*I've got less task on my desk, and was willing to take the timeout to prepare the solution that he needed, all I needed to hear for a kick start was an "Okay" just to be sure he wants it*) I suggested and re-assured but up to this point he still declined my offer and is willing to stick with his current reporting pattern (*Me died*).1
I made a script that performs some analysis on data from a trading card game tournament and I would appreciate your input7
never before have I been happy to be asked to work overtime, but for once, fuck yeah...
Bit of back story, I am tech lead on a massive project that has been run like a complete shit show, the PM who also happens to be the brains behind the project seems to think we are miracle workers and for the first 9/10 months of the project would make significant, like delete a weeks worth of code and start over changes, 3-5 times per week. There are features for the v1 release that have been built in excess of 5 times. I have been saying since October that even without all his constant changes, we will NOT make the deadline, and naturally as is part of my job I argued against every unnecessary feature he tried to implement, eventually he pulled me into a meeting to tell me how much he values my opinion, I need to stop arguing with him and he does not want to work with yes men (I have a rant about that convo already).
I believe our CEO finally started smelling a rat as he insisted on joining our daily stand-ups, during which said PM scripted some lovely stories to disguise the fuckup we are in, and since has assigned another PM to take over and do proper project management and risk analysis.
That is where the email comes in, a lot of the work assigned to me will miss the deadline by a month, honestly I am impressed that it is by so little and so few people will not be missing it, but anyway, he probably spun a few stories there too.
So I spent part of the work compiling the most perfect surgical response as not not actively throw him under the bu, but create a quite a few questions that they hopefully as, as himself and the CEO where cc'd into the mail.
And the jist is, the deadline itself was still impossible and 8 of the 10 tasks assigned to be have ZERO back-end whatsoever, and those tasks are about 80/90% integration to said non-existent back-end, some of those services and data structures have not even been planned yet and we are a week past the deadline and 3 weeks from the just as useless extension.
I work with statistics/data analysis and web development. I study these subjects for almost a decade and now I have 4 years of practical experience.
This information is on my LinkedIn profile and from time to time tech recruiters contact me wanting to have an interview. I always accept because I find it a great way to practice interviews and talking in English, as it isn't my native language.
A remark that I always make to my colleagues wanting to start doing data analysis related work is that it may seem similar to development, but it's not. When you develop, your code work or not. It may be ugly, it may be full of security problems, but you almost always have a clear indication if things are functioning. It's possible to more or less correlate experience using a programming language with knowing how to develop.
Data science is different. You have to know what you are doing because the code will run even if you are doing something totally wrong. You have to know how to interpret the results and judge if they make sense. For this the mathematics and theory behind is as important as the programming language you use.
Ok, so I go to my first interview for a data science position. Then I discover that I will be interview by... a psychologist. A particularly old one. Yeah. Great start.
She proceeds to go through the most boring checklist of questions I ever saw. The first one? "Do you know Python?". At this point I'm questioning myself why I agreed to be interviewed. A few minutes later, a super cringy one: "Can you tell me an example of your amazing analytics skills?". I then proceed to explain what I wrote in the last two paragraphs to her. At this point is clear that she has no idea of what data science is and the company probably googled what they should expect from a candidate.
20 minutes later and the interview is over. A few days later I receive an email saying that I was not selected to continue with the recruitment process because I don't have enough experience.
In summary: an old psychologist with no idea on how data science works says I don't have experience on the subject based on a checklist that they probably google. The interview lasted less than 30 minutes.
Two weeks later another company interviews me, I gave basically the same answers and they absolutely liked what they heard. Since that day I stopped trying to understand what is expected from you on interviews.2
(Day 1 of Database Class)
Database Prof - "Your final project is a program or app that deals with big data and showcases data analysis. Make something that can be used by consumers and has real-world application."
(After the Day of Showcasing Projects)
Database Prof -
(What he should have said on Day 1) - "Make something that makes me laugh with added data."
Bootcamps get you up and running in coding quickly. If you are a programmer, companies are only interested on how quickly, error free and cheaply you produce marketable output. Bootcamps enable this.
More or less you are not more than a former assembly line worker putting parts on a car platform. Your value is not very high as you may be exchanged at any time at their will.
Nevertheless, you can earn money quickly. You trade in your youth and time which might be a dead end in the long-term. Trends go to machine learning, artificial intelligence. They will not need Bootcamp people and code workers.
It is better you set up Bootcamps and sell them versus absolving this. Like selling shovels during the gold rush, but not working in the mud of Alaska by yourself.
Your choice is: Making quick money, which fades anyway; or striving for the long-term future proof career.
C/S degrees from Technical Universities of reputation give to you the right direction under a strategic consideration. Companies which pay well, or freelancing with a solid acknowledged background, will always look for top graduates. People from Bootcamps are just OK for hammering assembly line coding. Even worse with SCRUM in one noisy room under enormous team server pressure controls, counting your lines of code per minute, with pale people all around. And groups of controllers never acknowledging nor trusting your work.
To acquire a serious degree, a Bachelor is nothing. Here, in INDIA, Bachelor now is what a former high school grade was. You must carry a diploma or Masters degree combined with internships at big companies with high brand recognition. This will require 4–6 years of your lifetime. You can support this financially by working part-time freelancing as making some projects front- or back-end web, data analysis and else.
Bootcamp people will lose in the long-term. They are the modern cannon fudder of software production.
It is your choice. Personally, I would never do Bootcamps. Quality and sustainability require time, deep studies and devotion.
I code. I'm not a network or security admin. Did you even read my resume before hiring me? It says "math degree then data analysis then programming degree" Subnets, firewalls and security certs are not my areas of expertise. Given enough time, could I stumble through? Probably. But I'd probably come up with something worthy of mocking on networkadminrant.
Today I had a full-day job interview for a junior data scientist position.
First I met the team which was only like half of everyone because apparently everyone was gone on Fridays. However the few there were really nice.
First task is to do some basic data analysis stuff even though I already spent a week on the coding challenge and sent them all my code/tasks. I log into my machine and create a new virtual environment but can't for the life of me figure out how to use the command line in windows to install packages. Turns out there is some problem with their proxy and they have to log me in on that. Then I am struggling on the keyboard because it's for a language different that my mother tongue and it takes me 3x as long to so the most simple things. All my shortcuts are out the window. Haven't a hard time typing parentheses and brackets. Start freaking out and have a panic attack mid task. I'm sweating bullets. I didn't even make it to the simple visualization tasks much less the models at the end. Time gets called and we all go to lunch and I'm freaking out on the inside the entire time. Angry at myself because I know I am better and just couldn't think.
After lunch I present my code and results from a coding challenge I did weeks prior. People from other teams get invited and I end up getting grilled for 2 hours by 15 people. Questions are flying in from all sides. They ask me almost everything I know about machine learning and some more. Under stress I forgot the name of the optimizer I used and couldn't answer some easy stuff because my mind was racing.
Right now I am on the train home and my body physically hurts. I am disappointed with myself and wish I could have shown up better. Never really froze up like this before.2
My boss is being a stupid cunt. To give you a background we were facing issues with our Collections system. First week December 2019, I and a colleague of mine came up with a new efficient collections architecture. My colleague and I started to Code and create automation scripts mid December and completed it in First week of Jan 2020. This PoC version was supposed to be just between the Dev team(App Dev and Back end, also one from the Ops side to verify the data). I did not receive any feedback on the actual collections system and the data integrity but during this time all they’ve done is take meetings with no real outcome. I raised this and the only email I got is data is looking fine when I know it is not.Now in First week of Feb, he is stressing us to go ahead and deploy the architecture in Production and we have not done any Code Review, Static Code analysis, any real tests on Code and deployment scripts. Have not discussed any metrics for our dashboard and alerting. I have no idea how to handle this cunt. I have even asked for resources to atleast productionalize the code and move ahead the deployment and still no out come. I’ll go in a meeting with him in an hour, I will be very blunt and tell him that whatever he is doing is a foolish way and maybe resign in couple of weeks6
"We are applying deep learning, NLP, machine learning, Big Data frameworks and other technologies to produce outstanding fintech products in areas of robo advisors, stocks and cryptocurrencies analysis, digital assistants, prediction of customer behavior, deep learning analysis of alternative data (satellite images) and other areas."
So I'm in a scenario I'm uncomfortable, need some encouragement from fellow devRanters. (Looong post)
I've been working at this startup for about 10mths (since I graduated). They have been really good to me since the start, and overlooked some fuck ups I did at first.
But now I've been way more experienced , picked it up really quick. And I've basically redesigned several of their admin solutions and data products. Also, I'm basically their entire data analysis team now. I do backend (node, PHP, MySQL) and analysis for them (stats, deep learning, python, big data packaging for clients).
But seeing as I've moved in their company, and have been consulted on several major decisions, as well as built a really good relationship with some of their clients. I still haven't seen a raise, moreover I've been told that I'm expected to work from 8am to 5:30pm (9.5hrs no overtime pay). Which really pisses me off, since I know I'm worth more than what I'm paid (about 40k a year).
My brother (who's also a dev) suggested to tell them that I'm not happy at work due to this. And quit if they don't react well.
How should i bring this up? Should I really quit? This is all new grounds.6
I was studying a lot the last year, i learned a lot about Machine Learning/Deep Learning, Data Gathering, Data Analysis, ETL, Model Architecture Design, Training, Fine Tuning, Backend Development, DataBases, API Development, ORMs, Rest, GraphQL, OAuth, CI/CD, Docker, Deployment to Production environments like Heroku, Git and more stuff i dont remember while writing this. I built and keep adding stuff to my Github Portafolio.
Im not able to get a job. I started looking for jobs as Data Scientists, no response never. I take a look at freelancer sites, nothing seems to fit my skills. And when there is a minimal fit, they always want a Full Stack Web Developer, i dont know Frontend Development, i dont like do it.
Dont know what to do or how to land any job.
My options aeems to be:
1.Learn Frontend Dev and work as Full Stack in underpaying freelance jobs
2.Keep applying to Remote-Only startups, but they still wants people with 3+ years of experience.
i cant work in my city, here are not any company startup hiring no one, we are 30 years in the past here.
What you do in my place?10
I just made my API in my laravel and I understand how it works! It may seem like not a lot, but I got from far.
Just came two years ago in this industry as I worked as a customer service agent for a hostingcompany. I entered a whole new area what I immediatly got into at the time. Mind I already was studying Biomedical labresearch at the same time and was the IT guy in the family. Well, think back then I was just googling and fixing shit most of the time.
I was 21 at the time and began to learn everything I could learn in my position and soon it was not enough and wanted to learn more by working parttime(study already asks a lot of time). I soon applied as Junior System Engineer within the same company without prior education and got the job! And I'm back feeling I entered a new area where you feel you can do so much by just learning how it works. Now I want to learn to develop in PHP so I may make another step further.
Not a rant, but I want to share my experience as labrat starting to someting programming(did some bio-informatics, which was really interesting but with less emphasis on programming but more on data analysis). Still got a gigantic of list I want to learn from languages and frameworks to orchestration systems.
I have found that when companies say or think that they do not have large amounts of data for using even the smallest bit of data acience analysis in terms of generating a model that can assist them that they do have data. No matter the size of the organization, the data is there.
LABELED data, well now that is a other story 🥴1
I currently work in data analysis.
Yesterday one of the colleagues working in a different department came to me asking if I could fix his malfunctioning headphone set :/3
When your IT VP starts speaking blasphemy:
We all know what’s going on with the API. Next week we may see 6x order volumes.
We need to do everything possible to minimize the load on our prod database server.
Here are some guidelines we’re implementing immediately:
· I’m revoking most direct production SQL access. (even read only). You should be running analysis queries and data pulls out of the replication server anyway.
· No User Management activities are allowed between 9AM and 9PM EST. If you’re going to run a large amount of updates, please coordinate with a DBA to have someone monitoring.
· No checklist setup/maintenance activities are allowed at all. If this causes business impact please let me know.
· If you see are doing anything in [App Name] that’s running long, kill it and get a DBA involved.
Please keep the communication level high and stay vigilant in protecting our prod environment!"
RIP most of what I do at work.3
Stop calling data analysis tool AI... Stop calling droping notifications by % of new events on occurance calculation - AI!
There's a team where the leader has some real weird/bad ideas here, it's like, ugh, no. I can't say no because I'm not in position to contest, but geez I so want to.
Like, we wanted to load data with pagination because there could be a lot of them (could reach thousands objects of data easily).
Team Leader: No, no pagination because that ask to call those services several times. Only one call, you load all the data in the same single page and you don't call the service again, so stock them in cache.
So the idea at first looks bad, and after analysis and research, it is real bad, of course.1
I don't know what to choose...
A no-brainer job in data input with security, stability and a chance of promotion or transfer to another area IT related far away in the future...
An intership of one year, in systems analysis, with a 30% better salary but no guarantees after that...4
Me, working hard on a SQL project with a deadline that is half what it should be with no support from the other people on the project and was mostly made with with data I imagined would be in there cos no one could get me any fucking shit done (i.e. effectively designed, built, tested, fixed, upgraded, documented on my own for an entire weekly/monthly/ad-hoc analysis process that would output various reports for internal/external/management)
Manager - man who is a known waste of space but for some reason is in charge of the smallest part of the project, shouldn't have been fucking involved fucking management guzzling stain magnet...
Manager: Hey, do I need to refresh the database?
Me: ............I dunno, do you think we should refresh the database that this entire project is reliant? I mean...why do we need up to date transactions to analyse? Wait....you telling me it's not been being refreshed this ENTIRE time?
Manager: No....you never said I should. So should I?
Me: ..................I never said you should!?!? Are you not in the meetings talking about dependencies?????? Do you think i should have up to date trans or just run this with old stuff????? Why would you not update it!??!!?!?!? Its transactions...... (Desperately trying not to punch through my screen, through his, into his throat)
Manager: I think i'll refresh it and add it to the job?
Me: ....................(goes back to work cursing with music in so I think its quiet but who knows).
Tard, don't know how he even gets to work without someone holding his fucking hand.
Happy ending, I don't work there anymore :p
Sad ending, his spirit of tard follows me to my new jobs and possesses someone (or three sometimes)
How 2018 brings an aww bug.
At my company, we've a reporting application. Which kind of provides analysis of client's weekly business. When you open the application it shows their business trend starting from Sunday to day of week.
As usual their is an endpoint to get the data basis on start date. As soon as date changed to 1st Jan, it stopped showing data. Given that it was a long weekend, no one was available from the tech. Support team got tremendous amount of tickets for this. Later on Tuesday while debugging we got to know while forming the date in the application the logic was like this
- get current year
- get the date and month on start of week
Combine these 2 and request the data. All the time it was fetching data starting from 31 Dec 2018.
want to do software for my final year project but the lecturer's not in office and didnt reply my email during choosing topics, so end up with data analysis. 😴1
personal projects, of course, but let's count the only one that could actually be considered finished and released.
which was a local social network site. i was making and running it for about three years as a replacement for a site that its original admin took down without warning because he got fed up with the community. i loved the community and missed it, so that was my motivation to learn web stack (html, css, php, mysql, js).
first version was done and up in a week, single flat php file, no oop, just ifs. was about 5k lines long and was missing 90% of features, but i got it out and by word of mouth/mail is started gathering the community back.
right as i put it up, i learned about include directive, so i started re-coding it from scratch, and "this time properly", separated into one file per page.
that took about a month, got to about 10k lines of code, with about 30% of planned functionality.
i put it up, and then i learned that php can do objects, so i started another rewrite from scratch. two or three months later, about 15k lines of code, and 60% of the intended functionality.
i put it up, and learned about ajax (which was a pretty new thing since this was 2006), so i started another rewrite, this time not completely from scratch i think.
three months later, final length about 30k lines of code, and 120% of originally intended functionality (since i got some new features ideas along the way).
put it up, was very happy with it, and since i gathered quite a lot of user-generated data already through all of that time, i started seeing patterns, and started to think about some crazy stuff like auto-tagging posts based on their content (tags like positive, negative, angry, sad, family issues, health issues, etc), rewarding users based on auto-detection whether their comments stirred more (and good) discussion, or stifled it, tracking user's mental health and life situation (scale of great to horrible, something like that) based on the analysis of the texts of their posts...
... never got around to that though, missed two months hosting payments and in that time the admin of the original site put it back up, so i just told people to move back there.
awesome experience, though. worth every second.
to this day probably the project i'm most proud of (which is sad, i suppose) - the final version had its own builtin forum section with proper topics, reply threads, wysiwyg post editor, personal diaries where people could set per-post visibility (everyone, only logged in users, only my friends), mental health questionnaires that tracked user's results in time and showed them in a cool flash charts, questionnaire editor where users could make their own tests/quizzes, article section, like/dislike voting on everything, page-global ajax chat of all users that would stay open in bottom right corner, hangouts-style, private messages, even a "pointer" system where sending special commands to the chat aimed at a specific user would cause page elements to highlight on their client, meaning if someone asked "how do i do this thing on the page?", i could send that command and the button to the subpage would get highlighted, after they clicked it and the subpage loaded, the next step in the process would get highlighted, with a custom explanation text, etc...
dammit, now i got seriously nostalgic. it was an awesome piece of work, if i may say so. and i wasn't the only one thinking that, since showing the page off landed me my first two or three programming jobs, right out of highschool. 10 minutes of smalltalk, then they asked about my knowledge, i whipped up that site and gave a short walkthrough talking a bit about how the most interesting pieces were implemented, done, hired XD
those were good times, when I still felt like the programmer whiz kid =D
as i said, worth every second, every drop of sweat, every torn hair, several times over, even though "actual net financial profit" was around minus two hundred euro paid for those two or three years of hosting.
Is a masters in statistics worth it?
A bit of background:
I got my bachelor in actuarial math (statistics for insurance risk), then found machine learning and got a couple of gigs in software development and data engineering. I became my previous employers the go to guy for questions about data integrity and structure.
Now I am heading to a new job that specializes in ML for gambling. And while I love the math, I really see myself doing more software development and system architecture work (with some analysis). I already started this masters program, so I got less than a year to finish, but starting to feel like its a waste of my time, but also, I dont want to just quit it.
just receive a refurbished 4 years old laptop.... then deep clean it.... now having dilemma on picking a distro......
mainly use to run data analysis (r, Python, Java, C++, mySQL, MongoDB and some cloud servers...)
my thinking about a good distro to me, comfortable appearances, customize freedom, community support and constant security update.
any suggestion people??6
So I have question about my resume.
During my college time, I have done two projects related to politics:
One is to analyze the bias of media. What I did is scrape news covers for Trump and Hillary during election year and get sentiment analysis. The result is not surprising that among NY Times, NBC, Fox, Eashington Post, and CNN, Fox news is clearly favoring Trump, since Fox news is a republican news site.
The other project I did was to analyze the speech complexity and sentiment of the election. One of the observation we made was that Hillary and Trump are almost at the same level regarding speech complexity. However, Trump has a more positive sentiment in the speech, which is true consider how much he loves to say make America great again.
Now the question is, when I gave my advisor my resume, she said that I'd better not put those two projects on my resume since they are related to politics.
But, I am applying for a data science master degree. Seriously, I was just collecting the data and the data speaks for himself, why should I take those projects off my resume? I'm very proud of those projects I did as a matter of fact.
So here is the question. Shall I take off those two projects on my resume because they were political or I should leave it thereawarreally need some professional views. Please.1
I have last few months left out of graduation and i don't know what should i learn. There's so much things (web dev , ai/ml, blockchain, android , cloud, ,hybrid apps, gaming, ar/vr, data analysis, security,etc) and as a cs student, i feel i should be knowing them all.
In last 6 years ,
Techs that i liked or got success in :
java, Android,python, data analysis, hybrid apps(flutter)
Techs that i didn't liked or failed in : ai/ml, cloud computing , webdev(css/js) ,hybrid apps(react/angular/ionic/...)
Techs that i didn't tried : security, cryptography, blockchain, open cv , ar/vr, gaming
I am not bound by my likeness or success.
My failures was mainly because i didn't liked those techs and continued further in them. And my success comprises of just launching a few apps, passing in some certification or grabbing an internship opp because of those skills.
But if you think a particular skill is necessary to have as a cs professional then let me know. I just want to earn a lot of money and get out of this mess asap1
Been working on trying to get JMdict (relatively comprehensive Japanese dictionary file) into a database so I can do some analysis on the data therein, and it's been a bit of a pain. The KANJIDIC XML file had me thinking it'd be fairly straightforward, but this thing uses just about every trick possible to complicate what one would think would be a straightforward dictionary file:
* Readings and Spellings/Kanji usage are done in a many-to-many manner, with the only thing tying them together being an arbitrary ID. Not everything is related, however, as there can be certain readings that only apply to specific spellings within the group and vice versa. In short, there's no way to really meaningfully establish a headword fora given entry.
* Definitions are buried within broader Sense groups, which clumsily attach metadata and have the same many-to-many (except when not) structure as the readings/spellings.
Suffice to say, this has made coming up with a logical database schema for it a bit more interesting than usual.
It's at least an improvement over the original format, however, which had a couple different ways of setting up the headword section and could splatter tagging information across any part of a given entry. Fine if you're going to grep the flat file, but annoying if you're looking for something more nuanced.
Was looking online last night to see if anyone had a PHP class written to handle entries and didn't turn anything up, but *did* find this amusing exchange from a while back where the creator basically said, "I like my idiosyncratic format and it works for me. Deal with it!": https://sci.lang.japan.narkive.com/...
Grateful to the creator for producing the dictionary I've used most in my studies over the years, but still...3
Not sure what I wanna do ---
Backend Development or Data Analyst
Even in backend development confused between nodejs, PHP, Python though I don't have proper knowledge of all of them.
For Data Analyst I am looking the way from where I should start.
I am in 3rd year of my college
Can anyone please help me with this?6
Worked entire day on an ML to predict train ticket status for Indian Railways.
While doing the analysis of the data, and trying to check/research which parameters to use and which not, I'm feeling like racist -_-.
I was looking for busy period of the year(read festive seasons) and I removed those festivals which are celebrated by minor groups.
There's more to it, but the results are better now.4
Since a country is considered to be the strongest definition of a group of unified people, consider the following scenario:
Every country makes it mandatory that any foreign company can only sell their software products as free to use softwares, or one time cost software, or a membership software.
These foreign companies are NOT allowed to generate revenue via Advertisement services or data collection/sharing/analysis AT ALL.
The sole right to share/collect data and use Advertisments as a revenue model remains in the hands of domestic companies born, registered and working in that country only.
This would generate an equal chance for both domestic and international companies to grow(domestic companies getting a chance to grow with a better revenue model, while international companies getting a chance to grow by monetising their tools/algorithms and investing in domestic companies) , resulting in countries getting a chance to grow themselves.
Open source will still win, as open source majorly contains tools and technologies for general public use.
Premium tools and frameworks would become even more valuable, and would he shared among countries like the way they share space researches and resources on nuclear technology today.
Privacy will win, as the data of a particular country stayed within a country. Domestic Companies sharing data with other companies (or even their foreign parent companies) would be held against the respective country's laws only, and government would be more involved in protecting its citizens from data theft.
Is it feasible?13
"Dear TitanLannister : You are in the final year. A lot of shit is happening around u. its now time to make a career and take tough decisions. What would you do?"
CHOICE 1: COMPETITIVE
>>>>background : "a lot of super companies like wallmart, fb, amazon, ms, google,.. etc simply takes a straight coding test for fresher placement. They ask tough bad ass level questions, but with right guidance, a hell ton of dedicated hours of coding, and making it to the top of various coding tests could make you a potential candidate"
>>>>+ve points :
- "You got the teachers and professionals with great experience to guide you"
- "a dream job come true.you can go there and join teams that interests you"
- "it was your first exposure to computer world. maybe you would like doing it again, after 4 years"
>>>> -ve points:
- "You have always been an average 70 percentile guy. The task requires 2000-3000 hours of coding an year. it will be hard and you always grow bored out of this pretty quickly"
- "Even If you did that , you stand a lesser chance because your maths is shitty.There are millions running in this race with brains faster than your IDE"
- "your college will riot with you because they expect 75% attendance"
- "You are virtually out of college placements, in which , even though shitty companies come and offer even shittier 4LPA packages($6000 per annum), would take a tough logical/aptitude based test for which you won't be able to prepare"
CHOICE 2: PROFESSIONAL WORK
>>>>background: "you always wanted to create something , and therefore you started taking android based courses. you have been doing android for over 2 years and today you know a lot of things in android. you might be good in other professional lines like web dev, data analytics, ml,ai, etc too if you give time to that"
>>>>+ve points :
- "you will love doing this, you always did"
- "With the support of a good team, you will always be able to complete tasks and build new things quickly"
- "Start ups might offer you the placement, they always need students with some good exposure"
>>>>-ve points :
- "Every established company which provides interesting dev work takes their first round as coding, and do not considers your extra curricular dev work. So you are placing your all hopes in 1 good start up with super offerings that would somehow be amazed by your average profile and offer you a position"
- "start ups are well, startups and may not offer a job security as strong as est. companies"
- "You are probably not as awesome dev as you think you are. for 2 years, you have only learned the concepts , and not launched more than 1 shitty app and a few open source work"
CHOICE 3: NON CODING
>>>>background: "companies coming in college placements have 1-2 rounds of aptitude,logical reasoning , analysis based questions and other non tech tests. There are also online tests available like elitmus,AMCAT, etc which, when cleared with good marks help receive placements from decent established companies like TCS, infosys, accenture,etc"
>>>>+ve points :
- "you will eventually get placed from college, or online tests"
- "there will be a job security, as most of these companies bonds the person for 2-3 years"
>>>> -ve points:
- "You really don't like this. These companies are low profile consultant/services based companies which would put you in any area: from testing to sales, and job offers are again $5000-6000 per annum at max"
- "Since it includes college, the other factors like your average cgpa and 1 backlog will play an opposing role"
- "Again, you are a 70 percentile avg guy. who knows you might not able to crack even these simple tests"
Ugh... I am fucking confused. Please be me, and help.The things that i wrote about myself are true, but the things that i assumed about super companies, start ups or low profile companies might not be correct, these points comes from my limited knowledge ,terrified and confused brain, after all.
So, our lab professor in university for data structures was evaluating my friend's group assignment next to me and my colleague.
Professor: So, what does this method do?
Friend: I feel it would be best to explain our structure first and then how our methods work.
P: No, no. It's okay, I'm seeing it here in my laptop. *Looking at their report which included nothing about the structure of the assignment since she asked specifically only to place the methods and their complexity analysis due to not having time to look and evaluate a full report for each group*
F: Okay, *proceeds to explain*.
P: Okay, and why is your code commented?
F: Because it's good practice and we learned in subject x last semester.
P: Impossible! I teach that subject and I did not teach that in my classes.
F: Okay, but our professor did.
P: *Says nothing*.
They got a 3.25/5 as a grade for it and got really pissed off. I mean, they spent a lot of time making the assignment's structure perfect, optimizing their code and the professor did not care for the structure, just the methods, on the data structures subject.
FYI, they are workaholic, dedicate a lot of time improving their skills and their normal grades are usually >= 4.25.2
Wanted to know what you guys think about "Dev" jobs that also include making slide shows and presentations?
Eg. Imagine working on an analysis engine. And just when the core functionality is working, your boss wants your team to make presentation for every client that it's going to be used for - using the raw data given by the engine.
Instead of maybe adding that function to the engine itself.
Suddenly your work is now 12+hours of MS office instead of 8 hours of coding.
And a year later. You have 10 unfinished skeleton code architectures, poorly documented and 90% of the test cases never written.
And most of the devs who were on the initial project have either left out of frustration or have been fired because apparently fresher's who can not code with a senior coder level proficiency is not performing well.10
Probably working for a scientific research firm and building in house tools for collection and analysis of data...
Any recommendations for books on statistical analysis and data science?
The more survival guide & short manual types, the better.5
I'm taking data structures and algorithm analysis class at college but don't know how to study... I need advice, by the way the language we're using is c++9
I pitched this crazy sentiment analysis story and it got approved.
Which is great... only now, I have to start learning regular expressions to tokenize my data. Regular Expressions are the worst. 😒5
In nearby county to mine, coder was arrested for 3 months for nickname matching leaked terrorist nickname "grower" by coincidence. His coding education was enough reason for arrest.
All hdd/ssd/usb/mobile devices were confiscated for thorough analysis right from the morning by police.
Feeding my security paranoia. Encrypting fully filesystem(LUKS) and my internet traffic (self raised open vpn), wiping fully usbs. I ll be protected from my ISP recording my traffic, and from unauthorised access to my data.6
Has anyone (who does either Data Analysis, ML/DL, NLP) had issues using AMD GPUs?
I'm wondering if it's even worth considering or if it's too early to think about investing in computers with such GPUs.14
Sales guys in my company are unbelievable. They told client that my analysis algorithm runs for about one hour, so that is the time they have to wait for results. I asked if he actually asked what hardware do they have, and if they want all analysis done (its a complex algo) paralell, or just one by one.
Then I suggested to better ask these things, otherwise we end up reckt. Annnnd he said they will question our professionalism if we ask too much questions.
Ok so let me summarize. My algo needs to run maximum 1 hour long, regardless the actual functionality and input data, on any fcking hw you can imagine, or else i am not a pro. Mmmkay.
If I wanted good feedback for my products (especially if they were geared towards developers) I would scrap devRant and do some data analysis on how our products are perceived in raw form. Would be very raw and informative insight indeed because you are at the heart of raging innovation (raging innovation: when a developer is so pissed at a flaw in a piece of software they highlight or fabricate an ingenious feature or solution) and will help not only iron the kinks out but make a better product all together. Also, of course the good aspects would be lauded.
My jupyter notebook has outgrown itself on some real world trading data analysis and its becoming a pain to add to (further) and share.
Need to find better alternatives, web apps where are you?
But i know nothing about it. Learning curve ahead!
I've 7 interactive dashboard plots (from some data) in jupyter-notebook.
- It'd be nicer to have a web app to use them without running notebook from a different location.
- Or running notebooks remotely (running as daemons on host machine).
Any suggestions for a starter ?
rant before requirements, coz rants lead to better requirements.
Top 12 C# Programming Tips & Tricks
Programming can be described as the process which leads a computing problem from its original formulation, to an executable computer program. This process involves activities such as developing understanding, analysis, generating algorithms, verification of essentials of algorithms - including their accuracy and resources utilization - and coding of algorithms in the proposed programming language. The source code can be written in one or more programming languages. The purpose of programming is to find a series of instructions that can automate solving of specific problems, or performing a particular task. Programming needs competence in various subjects including formal logic, understanding the application, and specialized algorithms.
1. Write Unit Test for Non-Public Methods
Many developers do not write unit test methods for non-public assemblies. This is because they are invisible to the test project. C# enables one to enhance visibility between the assembly internals and other assemblies. The trick is to include //Make the internals visible to the test assembly [assembly: InternalsVisibleTo("MyTestAssembly")] in the AssemblyInfo.cs file.
Many developers build a POCO class in order to return multiple values from a method. Tuples are initiated in .NET Framework 4.0.
3. Do not bother with Temporary Collections, Use Yield instead
A temporary list that holds salvaged and returned items may be created when developers want to pick items from a collection.
In order to prevent the temporary collection from being used, developers can use yield. Yield gives out results according to the result set enumeration.
Developers also have the option of using LINQ.
4. Making a retirement announcement
Developers who own re-distributable components and probably want to detract a method in the near future, can embellish it with the outdated feature to connect it with the clients
[Obsolete("This method will be deprecated soon. You could use XYZ alternatively.")]
Upon compilation, a client gets a warning upon with the message. To fail a client build that is using the detracted method, pass the additional Boolean parameter as True.
[Obsolete("This method is deprecated. You could use XYZ alternatively.", true)]
5. Deferred Execution While Writing LINQ Queries
When a LINQ query is written in .NET, it can only perform the query when the LINQ result is approached. The occurrence of LINQ is known as deferred execution. Developers should understand that in every result set approach, the query gets executed over and over. In order to prevent a repetition of the execution, change the LINQ result to List after execution. Below is an example
public void MyComponentLegacyMethod(List<int> masterCollection)
6. Explicit keyword conversions for business entities
Utilize the explicit keyword to describe the alteration of one business entity to another. The alteration method is conjured once the alteration is applied in code
7. Absorbing the Exact Stack Trace
In the catch block of a C# program, if an exception is thrown as shown below and probably a fault has occurred in the method ConnectDatabase, the thrown exception stack trace only indicates the fault has happened in the method RunDataOperation
8. Enum Flags Attribute
Using flags attribute to decorate the enum in C# enables it as bit fields. This enables developers to collect the enum values. One can use the following C# code.
he output for this code will be “BlackMamba, CottonMouth, Wiper”. When the flags attribute is removed, the output will remain 14.
9. Implementing the Base Type for a Generic Type
When developers want to enforce the generic type provided in a generic class such that it will be able to inherit from a particular interface
10. Using Property as IEnumerable doesn’t make it Read-only
When an IEnumerable property gets exposed in a created class
This code modifies the list and gives it a new name. In order to avoid this, add AsReadOnly as opposed to AsEnumerable.
11. Data Type Conversion
More often than not, developers have to alter data types for different reasons. For example, converting a set value decimal variable to an int or Integer
My client wants me to "help increase sales" by applying seo voodoo. For over two years they didn't even have Analytics. Now, after 12 days they're asking me what do I make of the results. Like that's a meaningful amount of data for analysis.3
I'm starting to look at how to get devices to send data to an endpoint for storage and analysis. I'm looking at AWS iot stuff like core and green grass but then I'm thinking that a REST API could also do the job. I don't need to connect devices to each other (in first iteration). Dont think I need any edge stuff either. Anybody have any experience with this?3
everyone needs to be a data scientist/evangelist/superhero or AI enthusiast/developer/super-coder or project head for critical business needs/ or doing analytical analysis for business processes...even school students who are learning just to write English sentences, think they can code easily
AI folks, who think you can code automatically by thinking with no typing..
to them i say2
Hey guys is there anything we as devs could do regarding this covid-19 situation in terms of our technical skills? Like any project or something for making awareness at global or local level, or some status update regarding various things/items that are affected by this pandemic?
I am looking for ideas or currently ongoing projects that i could contribute to. Currently my tech stack is limited to java, python , android and a little bit of basic data analysis, but i am fluid enough to learn and contribute.7
Data structure and Analysis
For experienced ones, add "System Design".
...path for big fat packages.
I starting developing my skills to a pro level from 1 year and half from now. My skillset is focused on Backend Development + Data Science(Specially Deep Learning), some sort of Machine Learning Engineer. I fill my github with personal projects the last 5 months, and im currently working on a very exciting project that involves all of my skills, its about Developing and deploy a Deep Learning Model for Image Deblurring.
I started to look for work two months to now. I applied to dozens of jobs at startups, no response. I changed my strategy a bit, focusing on early stage startups that dont have infinite money for pay all that senior devs, nothing, not even that startups wish to have me in their teams. I even applied to 2 or 3 and claim to do the job for little payment, arguing im not going for money but experience, nothing. I never got a reply back, not an interview, the few that reach back(like 3, from 3 or 4 dozen of startups), was just for say their are not interested on me.
This is frustrating, what i do on my days is just push forward my personal projects without rest. I will be broke in a few months from now if i dont get a job, im still young, i have 21 years, but i dont have economic support from parents anymore(they are already broke). Truly dont know what to do. Currently my brother is helping me with the money, but he will broke in few months as i say.
The worst of all this case is that i feel capable of get things done, i have skills and i trust in myself. This is not about me having doubts about my skills, but about startups that dont care, they are not interested in me, and the other worst thing is that my profile is in high demand, at least on startups, they always seek for backend devs with Machine Learning knowledge. Im nothing for them, i only want to land that first job, but seems to be impossible.
For add to this situation, im from south america, Venezuela, and im only able to get a remote job, because in my country basically has no Tech Industry, just Agencies everywhere underpaying devs, that as extent, dont care about my profile too!!! this is ridiculous, not even that almost dead Agencies that contract devs for very little payment in my country are interested in me! As extra, my economic situation dont allows me to reallocate, i simple cant afford that. planning to do it, but after land some job for a few months. Anyways coronavirus seems to finally set remote work as the default, maybe this is not a huge factor right now.
I try to find job as freelancer, i check the freelancer sites(Freelancer, Guru and so on) every week more or less, but at least from what i see, there is no Backend-Only gigs for Python Devs, They always ask for Fullstack developers, and Machine Learning gigs i dont even mention them.
Maybe im missing something obvious, but feel incredible that someone that has skills is not capable of land even a freelancer job. Maybe im blind, or maybe im asking too much(I feel the latter is not the case). Or maybe im overestimating my self? i think around that time to time, but is not possible, i have knowledge of Rest/GraphQL APIs Development using frameworks like Flask or DJango(But i like Flask more than DJango, i feel awesome with its microframework approach). Familiarized with containerization and Docker. I can mention knowledge about SQL and DBs(PostgreSQL), ORMs(SQLAlchemy), Open Auth, CI/CD, Unit Testing, Git, Soft DevOps Skills, Design Patterns like MVC or MTV, Serverless Environments, Deep Learning Solutions, end to end: Data Gathering, Preprocessing, Data Analysis, Model Architecture Design, Training and Finetunning. Im familiarized with SotA techniques widely used now days, GANs, Transformers, Residual Networks, U-Nets, Sequence Data, Image Data or high Dimensional Data, Data Augmentation, Regularization, Dropout, All kind of loss functions and Non Linear functions. My toolset is based around Python, with Tensorflow as the main framework, supported by other libraries like pandas, numpy and other Data Science oriented utils.
I know lot of stuff, is not that enough for get a Junior Level underpaid job? truly dont get it, what is required for get a job? not even enough for get an interview?
I have some dev friends and everyone seems to be able to land jobs, why im not landing even an interview?
I will keep pushing my Dev career, is that or starve to death. But i will love to read your suggestions! how i can approach this?
i will leave here my relevant social presence:
Thanks in advance!11
Does anyone on here use the 3D software blender. What kind of Python do I need to learn (learned data analysis for internship but 80% sure won't be same as animation Python)1
Is a picture worth a thousand words?
Super fun data driven analysis based on Google's Conceptual Captions Dataset.
#python #dataanalysis #exploratorydataanalysis #statistics #bigdata3
REMINDER TL;DR: academic survey over devRant, 10-15 minutes https://forms.gle/do2KK8cGfv5w6cjY9
We are a group of researchers from Canada, Italy, and the Netherlands, studying communication between software developers. We would like to understand the role devRant plays in developers' professional life and the perceived advantages and disadvantages of the platform.
To this end we created an overview of the topics discussed. The purpose of this survey is to get your opinion on the overview. The results of the survey will be reported in a research manuscript, which will be submitted for a peer-reviewed publication.
We thank you for your participation.
Foutse Khomh, Nicole Novielli, Moses Openja, Alexander Serebrenik, Gias Uddin3
Hi So I need some solid advice from you all wonderful people.
I think i am now ready to look into job side of this world, but have lots of doubts , read my story.
I have been learning android for last 2 years. Most of the time i have been trying to understand how stuff works in android , but i have also gained a few other skills ( python programming, kotlin/flutter basics data analysis basics, testing, some graphic designing, aweful web dev ,etc). But i really want to work with Android. I don't have any specific Salary figure in mind, but i guess my knowledge is better or atleast par with most of the good android developers.
So i want to know how is this fresher/placement thingy work?
1.) GETTING KNOWN? : How can i make some good android based company aware that I am available for hiring? Should i start emailing every android related company that i know of? Should i start listing my profile on recruitment sites like linkedin or internshala? This year it is being said that companies will come for placements. From the status of my college, they are going to give me way to less $ , nd i know am not going to like any of them, but i guess i have to sit for them too.
2.INTERVIEW OR DIRECT PLACEMENTS? A little pre-context: i am currently starting my 4th year in clg. Afaik , 4th year isnt that strict and their can be leniency in terms of attendance. But my college is a place full of political cun*s in the name of directors and HODs and I don't know if they are again going to enforce the old 75% mandatory criteria. Plus if the company is from a different state/country , then my attendance would definitely not suffice.
So mainly i am unsure if somehow a company hires me, i would be able to start immediately. I heard that there are interviews for job recruitment after which the candidate is binded with an agreement to do some months training followed by permanent working after college completion.
This type of agreement is very much suitable for me, since from what my friend tells me, trainings can be lenient and understanding regarding exam preparations nd stuff.
So what do company usually chooses? Binding a fresher on immediate working basis or do they consider graduate completion?
Also, i suck at competitive coding. Do i need to polish myself on that or some company is willing to give me chance on the basis of my other skills 🙈(okay, no kidding , that's a serious question. I need to either work on getting better in competitive or build more apps based on that)
3.) ANDROID OR EVERYTHING? From what i have heard, working as a professional fresher is more like being an allrounder than being a domain specialist. But as i already stated, i really dig android and that's no small framework. I may di other stuff too, but won't interest me nd my output might be less efficient than expected.
So freshers can really be asked to do any stuff? Or can i still be in the area i like being into?
4.) COMPANY OR START-UP? Yeah, this is a general debate starter. Ignoring the business side of the conversation ( job safety vs more salary, experience, etc) the thing that's most important for me is the presence of a team. I want someone to assign me a task, whose vision i could follow, from whom i could learn, and some other people who are supportive and doing the same amount / similar work that am doing . This is so much import8 for me that i can easily ignore other factors for a better team. I once took a call from a startup ceo who hired me, a 2 month old android beginner at that time, as the "lead android developer"
But if am being on a team where i am supposed to do any random stuff that is assigned, then obviously this whole point of "visionary, helpful leader, guiding team, "etc goes moot9
Does ayone knows of a
+ Open Source
Data test generator tool?
I'm on my first work with a development that will go to production, and I'd like to test performance, UI and all of that with random generated data. I know about dbSchema but it's trial pulls me back
Devs can you point me to the topics that i should know in web dev? Kind of being hovering across a lot of different stuff, but don't know the actual path to follow . I kind of wish to become a full stack dev( not just the web) / Systems engineer someday and i am hoping to get a job asap
About me :
- oop/strong typing language guy. Knows enough java/kotlin/android dev to make decent Android apps, with confidence to apply on Android dev jobs
- knows enough python3 to understand its classes/functions. (But not more)
- knows some html css and client side js to make static websites, though my knowledge in this domain is not sequential but rather learning whatever i found interesting, so don't really get confidence to apply on a web dev job when i see an opening
- knows very basics of backend development in flask/php. Although worked with SAAS solutions liked firebase/aws
- knows some basics of data analysis6
How to prepare an academic essay
Today I bring you a Mini Guide on how to elaborate an essay, where you will learn what it is, what its structure is and what it is generally used for. If you are studying or you are going to start studying, surely this brief guide will help you to have greater clarity. At the end of the reading as a gift you can download the format of the structure totally free.
We will begin by defining it as an essay: “Written in which an author develops his ideas without having to show the scholarly apparatus” (RAE 2015). In other words, it is a writing where you plasmas your ideas regarding a specific subject and you do not necessarily have to Be “expert” to write it.
I remember that in college more than once I saved a test with a rating =)
How it is performed?
The essay has an internal structure, this means that at the time of writing must have a coherence based on a structure. But Joe, you should not place the titles of the structure.
Opening or Introduction: In this section you will begin with the presentation of the topic you are going to speak, justifying its importance, and highlighting the considerations of why you approach this topic, among other possible options. Generally many essayists begin with a question, which they develop throughout the document.
Development: Here the action begins! At this point you must give the reasons that justify the main thesis you are proposing. Also it would be great idea to use good hooks for essays in this part. Actually you can find some good tips here https://meowessay.com/blog/... . It will help you a lot with impressing your readers. You can also develop secondary arguments (All those that support, justify or expand controversial data or those that are not obvious, related to the main argument)
Conclusion: In this last you are closing the circle of the essay, where you will be able to make value judgments regarding the subject matter. While it is the conclusion, that does not mean that you have to give the “solution” to the thesis you raised. You can simply give an account of the perspective you have on the subject.
So that you have no problem, it is always good to be sure of the structure of the essay. If you are in college ask for it. But more importantly, you should cite the authors if necessary. There are different ways of citing them, that is why I also recommend that you ask your teacher or university to tell you which would be the one indicated.
What is it for?
Generally it is used in different areas of knowledge, to deal with a current problem, through the analysis and creativity that the essayist can have, which can be approached from different points of view.
Try not to repeat the ideas over and over again.
The simple is always better, so try to be as clear as possible, but always with a touch of seriousness.
Ask for help. Send the document some friends to give you their opinion and if they can understand what you wrote.
The dictionary can be your best friend so you do not repeat the words.6
Question: what's your opinion on Julia? It seems promising, the macro syntax is neat. But would you use it for data analysis/management or general purpose development?
Twitter api restricts me to fetch only 3202 tweets and if I had to fetch the retweeters count is limited to 100 .. is there any open source which will help me get through this problem because the data being collected is not sufficient for analysis
9 Ways to Improve Your Website in 2020
Online customers are very picky these days. Plenty of quality sites and services tend to spoil them. Without leaving their homes, they can carefully probe your company and only then decide whether to deal with you or not. The first thing customers will look at is your website, so everything should be ideal there.
Not everyone succeeds in doing things perfectly well from the first try. For websites, this fact is particularly true. Besides, it is never too late to improve something and make it even better.
In this article, you will find the best recommendations on how to get a great website and win the hearts of online visitors.
Take care of security
It is unacceptable if customers who are looking for information or a product on your site find themselves infected with malware. Take measures to protect your site and visitors from new viruses, data breaches, and spam.
Take care of the SSL certificate. It should be monitored and updated if necessary.
Be sure to install all security updates for your CMS. A lot of sites get hacked through vulnerable plugins. Try to reduce their number and update regularly too.
Ride it quick
Choose a quality hosting provider
In many respects, both the security and the speed of the website depend on your hosting provider. Do not get lost selecting the hosting provider. Other users share their experience with different providers on numerous discussion boards.
Content is king
Content is everything for the site. Content is blood, heart, brain, and soul of the website and it should be useful, interesting and concise. Selling texts are good, but do not chase only the number of clicks. An interesting article or useful instruction will increase customer loyalty, even if such content does not call to action.
Broadcasting should not be one-way. Make a convenient feedback form where your visitors do not have to fill out a million fields before sending a message. Do not forget about the phone, and what is even better, add online chat with a chatbot and\or live support reps.
Refrain from unpleasant surprises
Please mind, self-starting videos, especially with sound may irritate a lot of visitors and increase the bounce rate. The same is true about popups and sliders.
Next, do not be afraid of white space. Often site owners are literally obsessed with the desire to fill all the free space on the page with menus, banners and other stuff. Experiments with colors and fonts are rarely justified. Successful designs are usually brilliantly simple: white background + black text.
With such a dynamic pace of life, it is important to always keep up with trends, and the future belongs to mobile devices. We have already passed that line and mobile devices generate more traffic than desktop computers. This tendency will only increase, so adapt the layout and mind the mobile first and progressive advancement concepts.
Your visitors should be your priority. Use human-oriented terms and concepts to build navigation instead of search engine oriented phrases.
Do not let your visitors get stuck on your site. Always provide access to other pages, but be sure to mention which particular page will be opened so that the visitor understands exactly where and why he goes.
The site can be compared to a house - you always need to monitor the performance of all systems, and there is always a need to fix or improve something. Therefore, a technical audit of any project should be carried out regularly. It is always better if you are the first to notice the problem, and not your visitors or search engines.
As part of the audit, an analysis is carried out on such items as:
● Checking robots.txt / sitemap.xml files
● Checking duplicates and technical pages
● Checking the use of canonical URLs
● Monitoring 404 error page and redirects
There are many tools that help you monitor your website performance and run regular audits.
I hope these tips will help your site become even better. If you have questions or want to share useful lifehacks, feel free to comment below.
It seems like it never came to the mind of the developers of Apache Spark that people might want to use two of their libraries together to perform graph analysis on streamed Twitter data.
Why can't I simply use the streamed batches to create and extend a graph and perform continuous graph Algorithms on it??2
i want to learn some non dev stuff in my free time. i see a tempting bubble outside the tech world, but i don't know how to get into that. i don't know if the following things are related, but i feel that bubble consists of these skills : management (people/product/company, stream) , economics, analysis , charts , graphs , growth ,... etc buzz words. These are attributes attached to "our overlords" i'd say? mainly the PMs , QAs , EMs, stream leads, ceo/ctos, diagrams/charts , data etc. So how to get into that? where to start without again getting swallowed by dev work?
last i remember i tried python and data analysis in my 2nd year of college, but that was somehow turning to ai/ml stuff with ugly maths with their arma/arima models