Опублікувати конкурс

Scraping Google Trends

Статус: Closed
Винагорода: $10
Надіслані роботи: 1

Стисле описання конкурсу

First 3 to propose a solution by private chat will each be rewarded $50, with author rights to my site (as administrator, I will change settings on request). The first index-able multi-page group of google trending (top 500 words, and their related queries, which is usually one to several dozen keywords), with html for trends graph (usually last 7 days/month), incorporated with my own template, image and words, will win $500.

I use WP Scraper, and I will soon use Scrapebox. I got the following message from the WP Scraper owner:

Hey Jim,

I tried every way I could think of.... but I could not scrape google trends.

I was able to scrape pandabusters. All I had to do was check load javascripts option on the top right of the live scrape page.

URL Used: (pandabusters.com)

Its not your IP that's been blocked...( IP changers are NOT the answer) .. they have some type of security that's stopping the scrape. You will run into sites now and then that just cant be scraped.

What security can possibly stop us when it is always possible to make robots mimic everything humans do? Is it the fact that Google Trends, like Facebook, loads indefinitely, scrolling downward, so the scrape content loader just "crashes"? HELP! I will fund dearly any good scraper/security expert to come up with a fix, and we can corner the market for high volume outbound, inbound, high demand, low competition, up-trending keywords-links/spun-html, guaranteed multi-tier pyramids with cooperation, for the ultimate high volume "anti-spam", always changing original content, where spins, data merge with multiple "scrape template" spins can create "if you build it, they will come" hyper-business, large volumes of new pages on our sites daily, and many corresponding spun Web-threads obsessive-compulsive PBN and Web 2.0 engagements indefinitely...

Публічна роз'яснювальна дошка

marmikb4u
- 6 роки(ів) тому
First of all here, you need a google trend, and in that you want to use a scrapping tool, but why you want to use a scrapping tool?
I recommend use of API, there is already an API available that provides, JSON data of trends.

Or use the direct link like this "https://trends.google.com/trends/fullscreen/all/US"

Here, the content is continuously changing, and the document format is angular format, so the traditional scrapper will not work as expected or fail as written there..

This will require an API that works well

Or use some link like this:
https://trends.google.com/trends/hottrends
This updates every hour, and should work well with Scrapper:
first 3 elements div with class "hottrends-single-trend-container"
i.e. "Div.hottrends-single-trend-container"

Hope this comments will help solve the requirement.
- 6 роки(ів) тому