dnf - how to show which mirror url has been chosen?

Question

Below is the part of bash log, the full log can be achieved at https://gist.github.com/limkokhole/f2a423112aa005f10862:

[xiaobai@xiaobai hello]$ dnf --verbose download --source readline
cachedir: /var/cache/dnf
Loaded plugins: noroot, needs-restarting, reposync, copr, playground, kickstart, Query, generate_completion_cache, builddep, download, config-manager, protected_packages, system-upgrade, debuginfo-install
DNF version: 0.6.4
repo: using cache for: spot-chromium
not found deltainfo for: Copr repo for chromium owned by spot
not found updateinfo for: Copr repo for chromium owned by spot
repo: using cache for: rpmfusion-nonfree-updates-testing
not found deltainfo for: RPM Fusion for Fedora 21 - Nonfree - Test Updates
not found updateinfo for: RPM Fusion for Fedora 21 - Nonfree - Test Updates
repo: using cache for: updates-source
...
not found deltainfo for: RPM Fusion for Fedora 21 - Nonfree - Updates Source
not found updateinfo for: RPM Fusion for Fedora 21 - Nonfree - Updates Source
readline-6.3-5.fc21.src.rpm                                                                                                             623 kB/s | 2.4 MB     00:03    
[xiaobai@xiaobai hello]$ 

I've tried dnf info too, the url http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html is not a repo url:

[xiaobai@xiaobai hello]$ dnf info readline
[sudo] password for xiaobai: 
Using metadata from Thu Dec 31 19:18:09 2015 (6:13:33 hours old)
Installed Packages
Name        : readline
Arch        : i686
Epoch       : 0
Version     : 6.3
Release     : 5.fc21
Size        : 446 k
Repo        : @System
Summary     : A library for editing typed command lines
URL         : http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html
License     : GPLv3+
Description : The Readline library provides a set of functions that allow users to
            : edit command lines. Both Emacs and vi editing modes are available. The
            : Readline library includes additional functions for maintaining a list
            : of previously-entered command lines for recalling or editing those
            : lines, and for performing csh-like history expansion on previous
            : commands.

Name        : readline
Arch        : x86_64
Epoch       : 0
Version     : 6.3
Release     : 5.fc21
Size        : 483 k
Repo        : @System
Summary     : A library for editing typed command lines
URL         : http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html
License     : GPLv3+
Description : The Readline library provides a set of functions that allow users to
            : edit command lines. Both Emacs and vi editing modes are available. The
            : Readline library includes additional functions for maintaining a list
            : of previously-entered command lines for recalling or editing those
            : lines, and for performing csh-like history expansion on previous
            : commands.

[xiaobai@xiaobai hello]$ 

How can i know which url or domain has been chosen to download this readline-6.3-5.fc21.src.rpm ? --verbose doesn't seem to show the link. And how can i get this url information from dnf history in future after exit the bash session ?

Solution 1

Thanks for @muru to point out the librepo responsible for download.

The current workaround is define a debug_function and pass it to set_debug_log_handler (refer to download_packages.py ) in repo.py:

def download_payloads(payloads, drpm):
    # download packages
    drpm.err.clear()
    targets = [pload.librepo_target() for pload in payloads]
    errs = _DownloadErrors()
    try:

        #START my custom code
        def debug_function(msg, _):
            print("##hole## msg:", msg)
        librepo.set_debug_log_handler(debug_function)
        #END my custom  code

        librepo.download_packages(targets, failfast=True)
    except librepo.LibrepoException as e:
        errs.fatal = e.args[1] or '<unspecified librepo error>'
    ...

This repo.py file can be located by manually:

[xiaobai@xiaobai log]$ python -c 'import sys, dnf.repo; print(sys.modules["dnf.repo"])'
<module 'dnf.repo' from '/usr/lib/python2.7/site-packages/dnf/repo.py'>
[xiaobai@xiaobai log]$ 

[UPDATE] In Fedora 24, the path is /usr/lib/python3.5/site-packages/dnf/repo.py.

And now i'm able to get the url http://ftp.jaist.ac.jp/pub/Linux/Fedora/releases/21/Everything/source/SRPMS/r/readline-6.3-5.fc21.src.rpm:

[xiaobai@xiaobai test]$ dnf download --source readline
[sudo] password for xiaobai: 
Using metadata from Thu Dec 31 19:18:09 2015 (1 day, 11:59:10 hours old)
...
##hole## msg: select_next_target: Selecting mirror for: r/readline-6.3-5.fc21.src.rpm
##hole## msg: select_suitable_mirror: Skipping rsync url: rsync://ftp.jaist.ac.jp/pub/Linux/Fedora/releases/21/Everything/source/SRPMS/
##hole## msg: prepare_next_transfer: URL: http://ftp.jaist.ac.jp/pub/Linux/Fedora/releases/21/Everything/source/SRPMS/r/readline-6.3-5.fc21.src.rpm
##hole## msg: prepare_next_transfer: Resume ignored, existing file was not originaly being downloaded by Librepo
##hole## msg: lr_download: Downloading started
##hole## msg: lr_headercb: Server returned Content-Length: "2493152" (converted 2493152/2493152 expected)                             ] ---  B/s |   0  B     --:-- ETA
##hole## msg: check_transfer_statuses: Transfer finished: r/readline-6.3-5.fc21.src.rpm (Effective url: http://ftp.jaist.ac.jp/pub/Linux/Fedora/releases/21/Everything/source/SRPMS/r/readline-6.3-5.fc21.src.rpm)
##hole## msg: check_finished_trasfer_checksum: Checksum (sha256) 521bd47a3293e694190a237921a9954b20fa41d0e8e38183d186452d4cc62ac8 is OK
readline-6.3-5.fc21.src.rpm                                                                                                             1.4 MB/s | 2.4 MB     00:01    
##hole## msg: lr_download_packages: Restoring an old SIGINT handler
[xiaobai@xiaobai test]$ 

Of course, it could be better if dnf log this url for future reference, either retrieve from command dnf history or file /var/log/dnf.log.

Solution 2

For dnf itself, this is a deferred feature request. Apparently dnf does not know the URLs. See comments 4 and 5:

I apologize, I/we have misunderstood the motivation. FYI, DNF actually does not know the URLs. It just knows the URL of the "metalink" and the file name (and some other metadata) of the package. The underlying library librepo is responsible for choosing the best mirror, composing the URL and downloading the file. It means, BTW, that we would need some callback in the library to provide the information to you. Just FYI...

---

The problem is that we don't know the URL from metalinks, it is taken care on lower level by librepo. If anyone shares the same use case and would like to see this feature, post a comment and you'll get higher chance to have this feature.

Comments

[Admin]

I don't think it is possible at this point, unless you would be tracking network requests using wireshark or tcpdump. I might be wrong, but it would not be the firs feature that just does not exist in DNF.

[林果皞]

Are you testing it on Fedora 25 ? I just test it in Fedora 24 but prepare_next_transfer: doesn't reflect the latest downloaded src.rpm. And i need root to open this log.

[林果皞]

Erm, the correct command should be sudo download --source readline in order to get src.rpm. It did shows *.src.rpm in /var/log/dnf.log but it's not a URL. And the correct host did appear in list of urls (Mirrows from metalink) in dnf.librepo.log, but it doesn't states which host has been chosen. I tested it in Fedora 24.

[X3MBoy]

sudo cat /var/log/dnf.librepo.log | grep Effective\ url