Encrypting files

Assuming a situation where you want to share data with another colleague who also holds a CÉCI account, is part of the same project (UNIX group) and with whom want to share data that you must encrypt for one reason or another.

You can do that with gocryptfs. The process is as follows:

  1. You install gocryptf (simplest way is to download the pre-built executable from GitHub)
  2. You create a vault directory in a shared filesystem with a secret password and correct UNIX permissions for sharing
  3. You mount the cleartext version in a temporary directory with the gocryptfs command
  4. You share the secret password with your colleagues
  5. They install gocryptfs
  6. They mount the cleartext version in a temporary directory with the gocryptfs command

From there files you create in the temporary directory are encrypted and synchronised with them. Conflicts are handled by the shared filesystem.

Example

Download gocryptfs

wget https://github.com/rfjakob/gocryptfs/releases/download/v2.4.0/gocryptfs_v2.4.0_linux-static_amd64.tar.gz
tar xvzf gocryptfs_v2.4.0_linux-static_amd64.tar.gz
chmod +x gocryptfs
mv gocryptfs [some directory in your PATH]

Create a vault and choose a password. Here the vault will be the directory $CECIHOME/SecretFolder on the CÉCI common storage.

[dfr@lm4-f001 ~]$ mkdir $CECIHOME/SecretFolder

Initialise a vault in that directory. You will need to choose a password and make sure to store the master key in a sage and secure location.

[dfr@lm4-f001 ~]$ gocryptfs -init $CECIHOME/SecretFolder
Choose a password for protecting your files.
Password:
Repeat:

Your master key is:

    1a88a6b1-8f072fe8-7aac5356-1d025115-
    7574f7c3-627cbbdb-12b96ca8-09bfb39a

If the gocryptfs.conf file becomes corrupted or you ever forget your password,
there is only one hope for recovery: The master key. Print it to a piece of
paper and store it in a drawer. This message is only printed once.
The gocryptfs filesystem has been created successfully.
You can now mount it using: gocryptfs /CECI/home/ucl/pan/dfr/SecretFolder MOUNTPOINT

The contents of the vault is just two configuration files at this stage.

[dfr@lm4-f001 ~]$ ls $CECIHOME/SecretFolder
gocryptfs.conf  gocryptfs.diriv

Mount vault in a temporary cleartext directory (here ./Tests/ClearFolder). You will need to enter the secret password.

[dfr@lm4-f001 ~]$ gocryptfs $CECIHOME/SecretFolder ./Tests/ClearFolder
Password:
Decrypting master key
Filesystem mounted and ready.

Now the cleartext folder is actually a clear-text view of the vault. Any file written there is encrypted on the fly in the vault directory.

[dfr@lm4-f001 ~]$ echo test > ./Tests/ClearFolder/test.txt
[dfr@lm4-f001 ~]$ ls ./Tests/ClearFolder
test.txt
[dfr@lm4-f001 ~]$ ls $CECIHOME/SecretFolder
e6AxIMr4RuztuwpA-o_uOQ  gocryptfs.conf  gocryptfs.diriv

To remove the temporary cleartext directory, use the fusermount command:

[dfr@lm4-f001 ~]$ fusermount -u ./Tests/ClearFolder
[dfr@lm4-f001 ~]$ ls ./Tests/ClearFolder
[dfr@lm4-f001 ~]$ ls $CECIHOME/SecretFolder
e6AxIMr4RuztuwpA-o_uOQ  gocryptfs.conf  gocryptfs.dir

Sharing with a colleague

To share the encrypted vault with a colleague, you will have to

  1. share the vault (either read-only or read-write), and
  2. give them the secret password.

Note

Please review section Sharing files among CÉCI users before proceding with this tutorial.

If your colleague only needs read access to the vault, it is sufficient to set the UNIX permissions to world readable for the vault and “traversable” for the parent directories. Everyone will be able to read the contents of the vault, but will only see encrypted files.

If you colleague needs write access to the files, you will need to request the creation of a common UNIX group / project.

Warning

Do not set world-writable permissions on the vault. Even though other users will not be able to read clear files, they will be able to destroy the vault!

Make sure your colleague is able to write to the vault with the proper chgrp and chmod commands, e.g.

[dfr@lm4-f001 ~]$ mkdir $CECIHOME/SecretFolder && chgrp [common group] && chmod 770

Then, they can install the gocryptfs executable, create a temporary cleartext directory and mount the ecrypted vault:

[bvr@lm4-f001 ~]$ gocryptfs /CECI/home/ucl/pan/dfr/SecretFolder ./Tests/ClearFolder
Password:
Decrypting master key
Filesystem mounted and ready.
[dfr@lm4-f001 ~]$ ls ./Tests/ClearFolder
test.txt

Finally, they must use the fusermount command to remove the temporary cleartext directory when they are done working.

Using gocryptfs in a job

While you could decrypt the vault prior to submitting jobs, it is better practice to decrypt the vault in the submisison script, and to create the temporary cleartext folder on a temporary location that is cleaned up after the job, such as $LOCALSCRATCH.

The gocryptfs command requires the user to enter the secret password in order to decrypt the vault. But in a submission script, you will have to pass the secret password in another way:

The easy way: You can use -passfile FILE argument and specify a file that contains the secret password.

[dfr@lm4-f001 ~]$ gocryptfs -passfile $HOME/secret $CECIHOME/SecretFolder $LOCALSCRATCH/ClearFolder

Make sure in that case that the secret file has restrictive permissions.

The better way is to encrypt the secret password with Slurm’s munge utility and pass it through the -extpass parameter.

First encrypt the secret with

[dfr@lm4-f001 ~]$ munge -s "secretpassword" > $HOME/secretpasswordencrypted
[dfr@lm4-f001 ~]$ cat $HOME/secretpasswordencrypted
MUNGE:AwQFAAAOxRZL23a5wkqLWv0/+oL2BPY9V7kmYntbW0kvvw9EzitZwIATgj2xTbW9+vFv+INagbKPDZjWy6/tdCS0UnAR4O33GGpH9lm6sl5CydUIjjHiVU7EsWqHDilOt2vTOMgTLMNGgy49Vzbzty8No0kA:

and, in your submission script, use

[dfr@lm4-f001 ~]$ gocryptfs -quiet -extpass "echo $(cat $HOME/secretpasswordencrypted | unmunge | tail -1)" /CECI/home/ucl/pan/dfr/SecretFolder ./Tests/ClearFolder