"daily limit exceeded" when using Google custom search API

11,076

According to this post:

The first 100 queries per day are free. Any more, then you have to pay $5 per 1000 queries, for up to 10,000 queries per day, just enable billing to do so. Each query returns a maximum of 10 results, so you can retrieve 1000 URL’s from your search per day for free.

Share:
11,076
yvetterowe
Author by

yvetterowe

Updated on June 05, 2022

Comments

  • yvetterowe
    yvetterowe about 2 years

    I wanna crawl 200 results for about 2000 queries but it gives me a "daily limit exceeded" error.

    I want to confirm how many results can we crawl per day. Is there any solution that can solve this problem? Or the only way is to crawl a small part of queries each day...?

    My code to crawl google is as follow:

    def crawl(query_list):
        http = httplib2.Http()
    
        # Construct the service object for the interacting with the CustomSearch API.
        service = discovery.build('customsearch', 'v1',  developerKey='my api key', http=http)
    
        res_list = []
        for query in query_list:
            json_res = service.cse().list(q = query, cx = 'my search engine id', num = 200,).execute()
            res_list.append(json_res)
    

    Thank you!