How to sign files with Ubuntu command-line tools and my own keys?

Question

I want to sign some Python code files I wrote, because they are plug-in modules for one of my projects. For distributing it, I want that the user can be sure a plug-in is verified and safe (because written by me or somebody I trust) and was not modified.

The software is just a open-source hobby project, therefore I don't want to spend any money for buying an official certificate. Instead, I assume that the main program is always valid and can be trusted without additional verification. If somebody downloads it from any other location than my GitHub repository, it's their fault.

From what I've read, signing is usually done by creating an asymmetric key pair, calculating a strong cryptographic hash value (e.g. SHA-512) from the code file, encrypting the hash using my private key and storing that signature in a separate file to be shipped with the original code file.
The main program will then have to decrypt the signature using the public key that is saved in plain-text in the main program's source code, calculate the same hash function of the code file and then compare it to the decrypted one. If they match, the plug-in can be trusted.

So now my question:

How do I easily create a strong asymmetric key pair with Ubuntu tools and how do I easily calculate a cryptographic hash value of a file?
Automatizing the signing process in a script (using always the same key) would be great.

Solution 1

Much of this answer is ripped off from the Arch Wiki and the GnuPG documentation. Any recommendations in this answer are purely my opinion, and should be taken with a tonne of salt.

Creating a PGP key

GUI

1. Open the Passwords and Keys app (aka seahorse), and click + (or go to File->New, or press CtrlN) to see:

   

2. Select PGP key, and enter your details. I'm impersonating Byte Commander:

   

   RSA and 2048 bits are fine for most purposes. If you only want to use it for signing, pick the RSA (sign only) option from the drop-down menu, but you shouldn't need to - this can be handled using subkeys. You can leave a comment. Keeping an expiry date on your key is also useful. Click Create.

3. Enter a suitably long password (and I mean long, my example is short, IMO), and click Ok:

   

   Seahorse doesn't seem to have any feedback, unlike the CLI. Wait around for a while, doing whatever you want to do, while it gathers entropy and creates a key. It might take a while. After which, you'll see they key in the PGP Keys section:

   

CLI

To generate a key from the command line, just run gpg --gen-key. It will ask you for the same details that the GUI did:

$ gpg --gen-key 
gpg (GnuPG) 1.4.16; Copyright (C) 2013 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits
Please specify how long the key should be valid.
         0 = key does not expire
      <n>  = key expires in n days
      <n>w = key expires in n weeks
      <n>m = key expires in n months
      <n>y = key expires in n years
Key is valid for? (0) 1y
Key expires at Tuesday 27 September 2016 03:45:19 PM IST
Is this correct? (y/N) y

You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and E-mail Address in this form:
    "Heinrich Heine (Der Dichter) <[email protected]>"

Real name: Byte Commander
E-mail address: [email protected]
Comment: 
You selected this USER-ID:
    "Byte Commander <[email protected]>"

Change (N)ame, (C)omment, (E)-mail or (O)kay/(Q)uit? o
You need a Passphrase to protect your secret key.

passphrase not correctly repeated; try again.
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, use the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy!  (Need 186 more bytes)
.....+++++
+++++
We need to generate a lot of random bytes. It is a good idea to perform
some other action (type on the keyboard, move the mouse, use the
disks) during the prime generation; this gives the random number
generator a better chance to gain enough entropy.

Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy!  (Need 80 more bytes)
....+++++

Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy!  (Need 83 more bytes)
...+++++
gpg: key 8AE670A6 marked as ultimately trusted
public and secret key created and signed.

gpg: checking the trustdb
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 2u
gpg: next trustdb check due at 2016-09-26
pub   2048R/8AE670A6 2015-09-28 [expires: 2016-09-27]
      Key fingerprint = 82D9 0644 B265 8E75 1E01  538B B479 3CF4 8AE6 70A6
uid                  Byte Commander <[email protected]>
sub   2048R/0E2F4FD8 2015-09-28 [expires: 2016-09-27]

Note how GnuPG tells us it needs more entropy. Wish seahorse did too. But then again, it feels like GnuPG is acting like Oliver Twist. :P

Publishing your key

Now, we need to get our public key out there, so people can verify things using it.

GUI

Head back to the list of PGP keys in the seahorse app (see last screenshot). Select the key(s) you want to export, and in the Remote menu, pick Sync and Publish Keys:

The Sync button will be disabled if you haven't selected a server to publish to. Do so, by clicking on the Key Servers button:

I picked Ubuntu's server.

Now, you can click the Sync button and have it published to Ubuntu's keyserver (sorry for the spam, Ubuntu!).

CLI

With the CLI, you need the key ID of the key you want to publish. It's the very last line of the output when creating the key (8AE670A6). If you don't remember what it is, just run gpg --list-keys. To publish:

$ gpg  --keyserver pgp.mit.edu --send-keys 8AE670A6
gpg: sending key 8AE670A6 to hkp server pgp.mit.edu

Sorry, MIT.

Signing

I don't know of a convenient GUI method of signing a document, yet.

Once you have created the file you want to sign, head to the terminal. Try gpg --list-keys:

$ gpg --list-keys       
/home/muru/.gnupg/pubring.gpg
---------------------------
pub   2048R/F7878B0C 2015-09-28 [expires: 2016-09-26]
uid                  Byte Commander <[email protected]>
sub   2048R/345B9A4F 2015-09-28 [expires: 2016-09-26]

You can sign the file using two methods:

Signing with encryption

$ gpg --sign --output examples.sig examples.desktop 

You need a passphrase to unlock the secret key for
user: "Byte Commander <[email protected]>"
2048-bit RSA key, ID F7878B0C, created 2015-09-28

gpg: Invalid passphrase; please try again ...

You need a passphrase to unlock the secret key for
user: "Byte Commander <[email protected]>"
2048-bit RSA key, ID F7878B0C, created 2015-09-28

If you're on a desktop session, chances are you'll be greeted with a graphical password prompt. For example, in GNOME:

If the recipient has your public key, they can verify it, or get the decrypted content:

$ gpg --verify examples.sig
gpg: Signature made Monday 28 September 2015 03:25:00 PM IST using RSA key ID F7878B0C
gpg: Good signature from "Byte Commander <[email protected]>"
$ gpg --decrypt examples.sig
[Desktop Entry]
Version=1.0
Type=Link
Name=Examples
Name[aa]=Ceelallo
...
URL=file:///usr/share/example-content/
Icon=folder
X-Ubuntu-Gettext-Domain=example-content

gpg: Signature made Monday 28 September 2015 03:25:00 PM IST using RSA key ID F7878B0C
gpg: Good signature from "Byte Commander <[email protected]>"

Signing with cleartext

You might not want to encrypt the contents, for example, when sending a mail. In which case, use the --clearsign option:

$ gpg --clearsign examples.desktop 

You need a passphrase to unlock the secret key for
user: "Byte Commander <[email protected]>"
2048-bit RSA key, ID F7878B0C, created 2015-09-28

$ cat examples.desktop.asc 
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

[Desktop Entry]
Version=1.0
Type=Link
Name=Examples
Name[aa]=Ceelallo
...
URL=file:///usr/share/example-content/
Icon=folder
X-Ubuntu-Gettext-Domain=example-content

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJWCRAaAAoJEGUZkqX3h4sMBWsH/1yw+G0v5Ck+T3PBS90SkvC8
5C0FJeGVr0AgYQohhsE3zEGQ7nn53N7JsvNlF6VccvN99DZIp18JbrJ+qs5hWjtg
KU/ACleR5dvVrJgfjppkuC8Q3cAudvqciKlLjA7Xycr3P49oCNCy8k/ue2TrgCvS
mMb5IS/kqpO7wrOMBAR0c/2CjQsA91S1/YK7DbuUqeNgEzW1grsI7XZPhiDGpAib
D20HWrbdLhklAEJuo1EvuOIggW6MF6ksxDoVapsUzQalD0TWEq6OnvzIS5qhITrc
XaDPQJpiHyCyINnL5aZCUwr2uon7osJ+2a8Ahp1REpzIZTdND9jA5NWSel5+yAs=
=ZrtB
-----END PGP SIGNATURE-----

Signing, with a separate file for signature (detached signature)

Lastly, for some files, you can't have the signature in the document. For example, packaging files, or the metadata for a repository, both have content of a specific nature that don't easily allow embedded signatures. In this case, you use the --detached-sig option:

$ gpg --output examples.desktop.sig --detach-sign examples.desktop

You need a passphrase to unlock the secret key for
user: "Byte Commander <[email protected]>"
2048-bit RSA key, ID F7878B0C, created 2015-09-28

$ gpg --verify examples.desktop.sig examples.desktop
gpg: Signature made Monday 28 September 2015 03:35:55 PM IST using RSA key ID F7878B0C
gpg: Good signature from "Byte Commander <[email protected]>"

Note

In encryption+signing and in detached signatures, the output of gpg is binary. You can have GnuPG output base64-encoded data using the --armor option (ASCII-armoured).

Automation

To script signing, you could:

• use an empty passphrase for the key
• depending on your version of GnuPG, send the passphrase via stdin. See this Unix & Linux post for some options.

Solution 2

1. Create a asymmetric key with gpg

   gpg --gen-key
   

2. Use gpg to sign your file (your private key is used)

   gpg --output foo.sig --detach-sig foo.py
   

3. Test the signature file (your public key is used)

   gpg --verify foo.sig foo.py
   

   Example output

   % gpg --verify foo.sig foo.py 
   gpg: Signature made Mo 28 Sep 2015 12:46:04 CEST using RSA key ID 89B30DEC
   gpg: Good signature from "Your Name <[email protected]>"
   
   % echo "bad" >> foo.py
   
   % gpg --verify foo.sig foo.py
   gpg: Signature made Mo 28 Sep 2015 12:46:04 CEST using RSA key ID 89B30DEC
   gpg: BAD signature from "Your Name <[email protected]>"
   

Comments

[Byte Commander]

Very nice and detailed answer. But about the example keys you created... I don't know if I'm too happy about the fact that a set of unused keys for my pseudonym are laying around. If I want to create real keys with that name later and use them, people searching for them might get confused. I guess your example keys and my real keys will be able to coexist because of different email-addresses and therefore get a different ID, but still... Is it possible that you might delete those keys again?

[muru]

@ByteCommander the term is revoke. I'll look into revoking them. For that matter, I should add a section on revoking keys.

[muru]

@ByteCommander sorry, I deleted the key I published (and hadn't made a revocation certificate (classic n00b mistake), so I can't revoke that key. It does, however, have an expiry of one year, so, well, any confusion will be resolved in a year.

[Byte Commander]

Okay, well - that happens. Not sure whether I should be annoyed or just laugh about that stupid mistake... I think for my project in question I'll rather just use the Python module rsa which easily allows signing data without struggling with all the GPG stuff. That might become very useful once I start publishing and packaging software for real, but is probably just too much for my purpose at the very moment. So until then it will probably be expired! :)

[muru]

@ByteCommander both, I suppose. Though I'd suggest you use your real name when creating your key, and leave the nick for the comment field.

[Byte Commander]

That will depend which of my two personalities writes the code! ;D I am quite schizophrenic when it comes to my real and online identity...