If you are looking to automate some SEO / SERP processing in Google, it’s not long before you look to see how to automate the Google search API, and in this case, I’m using C#
Don’t even attempt to screen-scrape google, they will spot very quickly, and you’ll have wasted time doing HTML parsing for nothing, use the official API.
Now, the official API has one huge caveat, it is only useful for searching within a set number of specified sites. This means you can’t use it to determine, is my website in position #1 for keyword “Y”, but it can be used to check what pages of your site, or a competitor’s site are indexed.
This caveat rules out 99% of standard use cases, so feel free to close the page now, if it rules out your case. – Although I have seen that it is possible to include an entire Top Level Domain in the Custom search engine, like “*.es” (spain)
So, step 1 is to create a custom search engine, you do this from https://cse.google.com, and when it is created, copy the “cx” parameter, you will need this later
Step 2, go to https://developers.google.com/custom-search/v1/introduction/?apix=true and then press “Get A Key”, you’ll need this in step 3
Step 3, build up a URL as follows;
Where cx and key are from step 1 and 2 above respectively.
Q is the search query
Start is a number from 0 to 90 Which represents the start position in the search results. You cannot return more than 100 results using this API.
To request, you just use some code like this
var apiUrl =
searchTerm + “&start=” + start;
var response = http.Request(apiUrl);
var jResponse = JObject.Parse(response);
And hopefully that helps someone!