Got-scraping doesn't throw an error for error 4xx. #82
Answered
by
B4nan
meotimdihia
asked this question in
Q&A
|
Is this intent? it is different with |
Answered by
B4nan
Dec 5, 2022
Replies: 1 comment
|
Yes, intentional, I don't recall the exact reason now, but most probably to have more similar behavior to browsers. The library is mainly developed for making requests in http crawlers in crawlee. |
0 replies
Answer selected by
meotimdihia
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Yes, intentional, I don't recall the exact reason now, but most probably to have more similar behavior to browsers. The library is mainly developed for making requests in http crawlers in crawlee.