Details
-
AboutComputer Science student with a lot of salt to share with the world :)
-
SkillsC++, Assembly Language, Java, Python
-
LocationBelfast
Joined devRant on 5/19/2016
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
Has anyone here worked on news scraping?
I am currently doing my academic project where I need to scrap the news headlines. I have built scrappers for some news sources using their native API. I also tried using newsapi.org, but it returns only 10 results.
If anyone have worked on similar projects or know of their existence, some advice would be highly appreciated.5 -
0. Plan before you code. Document everything. You won't remember either your idea or those clever implementations next week (or next month, or next year...).
1. Don't hack your way through, unless that's what you intend to do. Name your variables, functions etc. neatly: autocomplete exists!
Protip: Sometimes you want to check a quick language feature or a piece of code from one of your modules. Resist the urge to quickly hack in the test into your actual project. Maintain a separate file where you can quickly type in and check what you're looking for without hacking on your project (For example, in Python, you can open a new terminal or IDLE window for those quick tests).
2. Keep a quiet environment where you can focus. Recommend listening to something while coding (my latest fad is on asoftmurmur.com). Don't let anything distract you and throw your contextual awareness out of whack.
3. Rubber ducks work. Really. Talking out a complex piece of logic, or that regex or SQL query aids your mind greatly in grasping the concept and clearing the idea. Bounce off code and ideas with a friend or colleague to catch errors and oversights faster. Read more here: https://en.wikipedia.org/wiki/...
4. Since everyone else is saying this (and because it merits saying), USE VERSION CONTROL. Singular most important thing to software development aside from planning and documenting.
5. Remember to flout all of the above once in a while and just make a mess of a project where you have fun throwing everything around all over the place. You'll make mistakes that you never thought were possible by someone of your caliber :) That's how you learn.
Have fun, keep learning!3