Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Search - "spark sql"
-
*Interview*
Interviewer: We have an opening. Are you interested to work?
Me: What is that I'll be doing?
I: What technologies and languages do you know?
Me: I know Scala, Java, Spark, Angular, Typescript, blah blah. What is your tech stack?
I: Any experience working on frontend?
Me: Yes. But what do you use for it?
I: Can you work with databases?
Me: I can, on SQL based. What are yours?
I: Can you do big data processing?
Me: I know Spark, if that's what you are asking for. What is it that you actually do?
I: Any experience in cloud development?
Me: Yes. AWS? Azure? GCP?
I: Do you know CI CD?
Me: Excuse me.. I've been asking a lot of questions but you're not paying attention to what I'm asking. Can you please answer the questions I asked.
I: Yes. Go ahead.
Me: What will be my position?
I: A full stack developer.
Me: What technologies do you use in your project?
I: We use all the latest tech.
Me: Like?
I: All latest tech.
Me: You mentioned big data processing?
I: Yes. Processing data from DB and generating reports.
Me: what do you use for that?
I: Java.
Me: Are you planning to rebuild it using Spark or something and deploy in the cloud?
I: No we're not rebuilding it. Just some additions to the existing.
Me: Then what's with cloud? Why did you ask for that?
I: Just to know if you're familiar.
Me: So I'll be working with Java. Okay. What do you use for UI?
I: Flash
Me: 🙄
I sat for a couple of minutes contemplating life.
I: Are you willing to join?
Me: No. Not at all. Thankyou for the offer.5 -
Merging datas using spark sql with two conditions.
my code:
1.merge with first condition
2.merge with second condition
3. 1.unionall(2)
This gives some extra datas(total=10000 and result=10220)
$better way?any way?