I'm trying to setup a Google custom search engine for scraping, but my CSE can't be scraped as the default, wich are inbuild in Scrapebox.
My CSE has an url like
and delivers search results in browser correctly as JSON, beginning with
If i setup it into Scrapebox and run "test engine" i get 400 bad request.
The default one, built in CSE has an url like
, and delivers in browser search results too, but beginning with
Both of test engine and scraping of this default CSE are successful.
Where i'm wrong with setting of my CSE? What and where should i change CSE settings to be able to scrape it?
thanks / regards
My CSE has an url like
Code:
https://www.googleapis.com/customsearch/v1?q=cloud&cx=MyCxNumber&googlehost=www.google.de&hl=de&start=1&rsz=filtered_cse&key=MyApiNumberCode:
{
"kind": "customsearch#search",
"url": {
"type": "application/json",The default one, built in CSE has an url like
Code:
https://www.googleapis.com/customsearch/v1element?key=AIzaSyCVAXiUzRYsML1Pv6RwSG1gunmMikTzQqY&rsz=filtered_cse&num=20&hl=en&prettyPrint=false&source=gcsc&gss=.com&start={PAGENUM}&cx=004415538554621685521:vgwa9iznfuo&q={KEYWORD}&googlehost=www.google.com&callback=google.search.Search.apiary17764Code:
// API callback
google.search.Search.apiary17764({
"cursor": {
"currentPageIndex": 0,
"estimatedResultCount": "129000000", ... etcWhere i'm wrong with setting of my CSE? What and where should i change CSE settings to be able to scrape it?
thanks / regards