DuckDuckGo API - How to get more results?

Question

The default search using the DuckDuckGo API returns only the results on the first page (around 25 I guess). Is there any way to get more results or navigate to the 2nd, 3rd pages of the search results?

Websites like Faroo have a parameter called s (which stands for start) which can be set to 1 if we want the first 10 results, to 11 if we want the next 10 results and so on. Is there something like that for DuckDuckGo, too?

Solution 1

According to DuckDuckGo Search API documentation, all the available parameters are:

q: query

format: output format (json or xml)

If format=='json', you can also pass:

callback: function to callback (JSONP format) pretty: 1 to make JSON look pretty (like JSONView for Chrome/Firefox)

no_redirect: 1 to skip HTTP redirects (for !bang commands).

no_html: 1 to remove HTML from text, e.g. bold and italics.

skip_disambig: 1 to skip disambiguation (D) Type.

In particular, note that:

This API does not include all of our links, however. That is, it is not a full search results API or a way to get DuckDuckGo results into your applications beyond our instant answers.

Solution 2

TL/DR; - Install TamperMonkey, add the short script below (full instructions follow) and the browser will lazy-load the next page(s) automatically as you scroll.

After coming to this answer via Google and not finding the information I was seeking, I wrote this small TamperMonkey script to do the job. I post it here for future googlers.

The below userscript will work with Chrome, Firefox and Opera. Instructions for installation follow below the script, and a brief explanation of what TamperMonkey is follows below that.

This script is inspired by, and named similarly to (in honor of), Endless Google by Tumpio.

// ==UserScript==
// @name         Endless DuckDuckGo
// @namespace    http://tampermonkey.net/
// @match        https://duckduckgo.com/?q=*
// @require      http://ajax.googleapis.com/ajax/libs/jquery/2.2.4/jquery.min.js
// @grant        none
// ==/UserScript==

(function() {
    'use strict';
    $(window).scroll(function(){
        var els = document.querySelectorAll('.result.result--more');
        if (els.length){
            var elmore = document.querySelectorAll('.result--more__btn.btn.btn--full');
            if (elmore.length){
                elmore[0].click();
            }
        }
    });
})();  //

 

How To Install the Above Script:

1. Install the TamperMonkey extension for Chrome (or the "add-in" for Firefox).

2. You will see the TamperMonkey icon appear at the top of the browser

3. Do a search on DuckDuckGo

4. Click on the TamperMonkey icon and from the drop-down menu, choose Dashboard

5. Along the tabs at the top of the Dashboard page, click on the [+] icon at left of the tab strip

6. The TamperMonkey editor will open up with a blank UserScript template. Delete that entire sample script and replace it with the script from this post.

7. Save [Ctrl] + [s]

8. Run another DuckDuckGo search and scroll down the page... True happiness is yours.

 

What Is TamperMonkey:

A good overview is here.

TamperMonkey is a browser extension, and there is a version of TamperMonkey for each major browser. You probably already use the AdBlock or uBlock browser extensions (if not, WHY NOT?), this is just another extension like those. Anyway, to install for Chrome or Brave, go to the Chrome Web Store and search for TamperMonkey by Jan Binoc. Install it. (Yes, it's safe - there are hundreds of thousands of users, mostly coders). Please consider donating - Jan deserves your support (and no, I don't know him, and yes I donated.)

Before TamperMonkey, there was another extension called GreaseMonkey that did the same thing but only worked on Firefox. However, the GreaseMonkey authors stopped maintaining it or something, and Jan Binoc stepped up to the plate with TamperMonkey.

TamperMonkey allows us to inject our own code into ANY webpage, to programmatically manipulate the web page on our local computers. How does that work? Simplistic Explanation: When you view a web page, you never actually view it "directly from the web server" - your browser first downloads a local copy of the web page code to your browser's cache folder and displays it to you from there. Therefore, TamperMonkey can intercept the page as it loads from cache (on your local hard drive) into the browser and modify it before it is displayed. That explanation is super-simplistic and not fully technically accurate, but in essence that is exactly how it works, and why TamperMonkey works. Most Importantly: The above few lines explain why the page does not change for anyone else - just for you, on your own computer.

TamperMonkey is an excellent reason to learn a bit of javascript/css/html. Using it, you can do stuff like hiding or re-arranging images on a webpage, removing clutter from a page, totally reformatting a page, etc. For example, one of my fav News sites has lots of clutter. So, I go to their RSS feed page, which acts like a great index of articles, but that also has too much stuff I don't want to see (mainly unnecessary thumbnail images and too-narrow columns). I wrote a short TM script to hide all the images and widen the columns and now, instead of seeing 5 or 6 article summaries per screen, I see ~ 20.

The absolute best, most concise, primer for html/css/js that I've ever seen is on Lynda.com. (You might already have access via your local library card - I was greatly surprised to find out that I do.) There is a series by Emma Saunders called D3.js Essential Training for Data Scientists. The course begins with two short tutorials (Recalling HTML Basics (4m) and Understanding HTML5 (3m) ) in html/css/js that are worth a university course tuition by themselves. Why can't everyone teach like this? Anyway, that's all you need - those first two (3 and 4 min) videos. Now, go tweak a webpage.

(Final disclaimer: No, I don't know Emma Saunders either, nor do I have anything to do with either Binoc's or Saunders' products in any way. I'm just a run-of-the-mill user and fan.)

Comments

[chovy]

checkout searx, it includes a json format. its a meta search engine.