Firefox - reading out urls of opened tabs from the command-line

Question

I sometimes have quite a big range of tabs open in Firefox and I prefer it to save them to a file, rather then using the build-in bookmarks.

Therefore I (manually) copy the urls from the about:preferences page, save them to a file and process the file with: tr '|' '\n' in a little bash script.

Later when I want to reopen the tabs from the textfile I run this little loop:

#!/bin/bash

# usage: $bash Open-tabs.sh file-with-bookmarks.txt

 while read -r line; do
     firefox -new-tab "$line" 2>/dev/null &
     sleep 2
 done < "$1"

and it opens all tabs with a delay of 2 seconds.

I would like to know if there is a way, I can read-out the urls of the opened tabs from the command line, so I could include it to my script?

Solution 1

Source(Changed file path) : Get all the open tabs

This snippet gets the current firefox tab url's. It uses the recovery.js[onlz4] file in your profile folder. That file is updated almost instantly, however it will not always be the correct url.

Get all the open tabs:

python -c '
import io, json, pathlib as p
fpath = next(iter(p.Path("~/.mozilla/firefox").expanduser().glob("*.default/sessionstore-backups/recovery.js*")))
with io.open(fpath, "rb") as fd:
    if fpath.suffix == ".jsonlz4":
        import lz4.block as lz4
        fd.read(8)  # b"mozLz40\0"
        jdata = json.loads(lz4.decompress(fd.read()).decode("utf-8"))
    else:
        jdata = json.load(fd)
    for win in jdata.get("windows"):
        for tab in win.get("tabs"):
            i = tab["index"] - 1
            print(tab["entries"][i]["url"])
'

Solution 2

this works for Firefox 57+. You'll need lz4 (via pip). The file header is gathered from the length of b'mozLz40\0'. Use an environment variable for the filepath if you want to use it in a oneliner, replace with \n and \t accordingly and merge lines.

export opentabs=$(find ~/.mozilla/firefox*/*.default/sessionstore-backups/recovery.jsonlz4);

python3 <<< $'import os, json, lz4.block
f = open(os.environ["opentabs"], "rb")
magic = f.read(8)
jdata = json.loads(lz4.block.decompress(f.read()).decode("utf-8"))
f.close()
for win in jdata["windows"]:
    for tab in win["tabs"]:
        i = int(tab["index"]) - 1
        urls = tab["entries"][i]["url"]
        print(urls)'

Solution 3

Some of these answers reference the "[random chars].default" directory. Starting with version 67, users can have profiles for different update channels (e.g., release, beta, nightly, etc.).

On my Ubuntu 18 system, this directory was "[random chars].default-release". I still had a "[...].default" directory but it was mostly empty. Keep that in mind if you get an error that "sessionstore-backups" can't be found.

Solution 4

I recommend using https://github.com/balta2ar/brotab for this purpose:

pip install brotab
brotab install

Install the web extension as well: https://addons.mozilla.org/en-US/firefox/addon/brotab/

Restart Firefox, and you can use brotab list and parse it as so:

bt list | awk -F'\t' '{
    print "Downloading "$2
    system("curl --silent --output \""$2"\" \""$3"\"")
}'

View more solutions

Comments

[nath]

is it possible, the path on debian is ~/.mozilla/firefox/*.default/sessionstore-backups/recovery.j‌​s?

[Hunter.S.Thompson]

Yes sorry I edited the question with that but I forgot the sessionrestore-backups before recovery.js. Thank you for pointing it out. Edited Answer

[nath]

I always get Traceback (most recent call last): File "<stdin>", line 2, in <module> IOError: [Errno 2] No such file or directory: '~/.mozilla/firefox/*.default/sessionstore-backups/recovery.‌​js' with cat I can read the file...

[Hunter.S.Thompson]

I edited the answer. copy and paste it, but instead of <username> add your username.

[Hunter.S.Thompson]

Python probably does not understand the ~/ syntax being /home/<username>.

[Gilles Quenot]

To get the latest needed file from firefox : export opentabs=$(ls -t ~/.mozilla/firefox*/*/sessionstore-backups/recovery.jsonlz4 | sed q)

[Gilles Quenot]

Can you explain magic = f.read(8) ? You don't use magic var later.

[wbob]

the first 8 bytes in recovery.jsonlz4 are part of mozillas custom file format - passing the length of the magic to read(8) causes the next read() within json.loads to continue after the header, making it a valid decompressable lz4 stream. For the reasoning to choose a custom file header, you can find more details in this github gist linking to the relevant firefox source.

[SebMa]

@wbob Now I get UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf2 in position 12: invalid continuation byte, can you help ?

[wbob]

@Sebma I updated to lz4 2.0.0 and got the error too, your edit fixed it, thank you. I went ahead and merged the module imports

[nath]

@GillesQuenot using: ls -t ~/.mozilla/firefox*/*/sessionstore-backups/recovery.jsonlz4 | sed q I get: ls: cannot access '/home/user/.mozilla/firefox*/*/sessionstore-backups/recover‌​y.jsonlz4': No such file or directory

[nath]

@wbob using: find ~/.mozilla/firefox*/*.default/sessionstore-backups/recovery.‌​jsonlz4 I get: find: /home/user/.mozilla/firefox*/*.default/sessionstore-backups/‌​recovery.jsonlz4': No such file or directory

[nath]

got it, on my system it is called .../recovery.js still I get the error: Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'js'

[wbob]

@nath absence of a jsonlz4 file hints at a firefox version prior to 57 - then the answer by raymii/huntersthompson is to be prefered and this one will not work. Nonetheless: your traceback reads like the import line somehow got cut up in the middle of "json" - as this is what python will throw for an "import js" statement.

[nath]

@wbob ah thanks you are right I'm running Firefox 52.8.1 (32-Bit)

[Will]

jq can be used instead of python: jq '.windows | .[].tabs | .[].entries | .[].url' $(find ~/.mozilla/ -name recovery.js) but the tab list seems very off/outdated

[SebMa]

@wbob On firefox > 57, I get UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 8: ordinal not in range(128) What is the encoding of the recovery.jsonlz4 file ?

[SebMa]

@wbob I've uncompressed the recovery.jsonlz4 file with the mozlz4a.py script and it turns out the recovery.json is utf-8 encoded, I don't understand why it does not work.

[SebMa]

@wbob My mistake, I had forgotten the magic = f.read(8) instruction by the time I used your script.

[ankostis]

Didn't work since firefox uses its own .jsonlz4 formatted files nowdays - so i ported the code to Python-3 and the locating of the recovery-file is now automated, so now the code work as-is.