Step-by-step tutorials for you to get started with web scrapingDownload Octoparse
Why does Octoparse stop after clicking “Next” ?Tuesday, December 03, 2019
The latest version for this tutorial is available here. Go to have a check now!
While scraping data from the web, Octoparse takes the reloading as the signal to execute the action, such as "Click item" and "Click to paginate". For the web page using AJAX, it updates new contents without reloading. As there is no reloading, Octoparse doesn't receive the signal to act and would be stuck in the last step. As a result, we may get zero, or much fewer extracted data than we expect.
To fix this issue, you will need to set up “AJAX Load” timeout under “Advanced Options” of the “Click Item” action.
AJAX timeout setting in Octoparse is based on the time it takes for web pages to load. You can also set up a higher value as long as there is enough time for the next page to load.
In fact, many websites update new data with AJAX configuration such as Amazon, eBay, Google etc. Sometimes, if Octoparse takes so long to scrape data after clicking an item, you should also configure AJAX timeout to make sure there’s enough time for detail pages to load.
If you do have questions about AJAX, read more details here
- Most popular tutorials
- Scrape data on Instagram
- Scrape business information from Yelp
- Scrape tweets from Twitter
- Scrape listings from Airbnb
- Scrape hotel data from Booking.com