Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
I had someone request something pretty similar. (No mention of an API though.)
Wrote it in selenium and let it run for half a week. Site was slow as frozen tamales.
The same script worked flawlessly for all but like 15 entries (out of 500+?). For some reason selenium just refused to work with them, saying it couldn’t find elements that were obviously there, the whatever expired, Mercury was on retrograde, someone insulted its mother (it was me), etc. idfk. No sense.
Never got paid for it, so I never gave them the data. 🤷🏻♀️ -
@sipu261988 Cram it, botface.
When i try to justify the inconsistency of Selenium Tests.
Boss"So, we need to get some data about the users using the APIs from this list of sites."
Me"Alright, sounds feasible enough"
Navigating to first site.
M"Hold on, where's the API?"
B"What do you mean? You're looking at it."
M"This is a website with a search bar, not an API"
B"Same thing. Get to scrapping that data."
M"I-It's written in a JS framework to be reactive in a half-assed way."
B"We need that data"
M"The data is not even consistent!"
B"That's why we need to join it with all these different sources."
The API was a lie. None of the sites had anything remotely similar to an API.
Having to use bloody selenium with chrome driver to scrap all the information because of course, it has to be done programatically every week from now on.
I just hope no captcha of any kind is installed before I finish this project.
rant
selenium
chrome driver