github.com/StackExchange/blackbox/v2@v2.0.1-0.20220331193400-d84e904973ab/Version2-Ideas.md (about)

     1  # Ideas for BlackBox Version 2
     2  
     3  I'm writing this to solicit feedback and encourage discussion.
     4  
     5  Here are my thoughts on a "version 2" of BlackBox.  This is where
     6  I list ideas that would require major changes to the system. They
     7  might break backwards compatibility, though usually not.
     8  
     9  BlackBox grew from a few simple shell scripts used at StackOverflow.com
    10  to a larger system used by dozens (hundreds?) of organizations. Not
    11  all the design decisions were "forward looking".
    12  
    13  These are the things I'd like to change someday.
    14  
    15  [TOC]
    16  
    17  ## Change the commmand names
    18  
    19  There should be one program, with subcommands that have names that make more sense:
    20  
    21  * `blackbox admin add <key>`
    22  * `blackbox admin list`
    23  * `blackbox admin remove <key>`
    24  * `blackbox cat <filename> ...`
    25  * `blackbox decrypt <filename> ...`
    26  * `blackbox diff <filename> ...`
    27  * `blackbox edit <filename> ...`
    28  * `blackbox encrypt <filename> ...`
    29  * `blackbox file add <filename> ...`
    30  * `blackbox file list`
    31  * `blackbox file remove <filename> ...`
    32  * `blackbox info`
    33  * `blackbox init`
    34  * `blackbox reencrypt`
    35  * `blackbox shred --all|<filename> ...`
    36  * `blackbox status --all|<filename> ...`
    37  
    38  Backwards compatibility: The old scripts will be rewritten to use the new commands.
    39  
    40  ## Change the "keyrings" directory
    41  
    42  The name `keyrings` was unfortunate.  First, it should probably begin with a `.`.  Second, it stores more than just keyrings.  Lastly, I'm finding that in most cases we want many repos to refer to the same keyring, which is not supported very well.
    43  
    44  A better system would be:
    45  
    46  1. If `$BLACKBOX_CONFIG` is set, use that directory.
    47  2. If the repo base directory has a file called `.blackbox_external`, read that file as if you are reading `$BLACKBOX_CONFIG`
    48  3. If the repo base directory has a `keyrings` directory, use that.
    49  4. If the repo base directory has a `.blackbox` directory, use that.
    50  
    51  Some thoughts on `.blackbox_external`:
    52  I'm not sure what the format should be, but I want it to be simple and expandable.  It should support support `../../dir/name` and `/long/path`.  However some day we may want to include a Git URL and have the system automatically get the keychain from it. That means the format has to be something like directory:../dir/name so that later we can add git:the-url.
    53  
    54  NOTE: Maybe `.blackbox_external` should be `.blackbox/BLACKBOX_CONFIG`?
    55  
    56  Backwards compatibility: `keyrings` would be checked before `.blackbox`.
    57  
    58  ## System Test
    59  
    60  There needs to be a very complete system test.  The `make test` we
    61  have now is great for something written in bash.
    62  
    63  It should be easy to make tests. Perhaps a directory of files, each
    64  specifying a test.  We could make a little language for writing tests.
    65  
    66      # This test becomes the user "alice" and verifies that she
    67      # can encrypt a file, and decrypt it, with full fidelity.
    68      BECOME alice a
    69      BASH echo "foo contents" >foo.txt
    70      SHOULD_NOT_EXIST foo.txt.gpg
    71      BASH blackbox encrypt foo.txt
    72      SHOULD_NOT_EXIST foo.txt
    73      SHOULD_EXIST foo.txt.gpg
    74      BASH_WITH_PASSWORD a blackbox decrypt foo.txt
    75      SHOULD_EXIST foo.txt.gpg
    76      SHOULD_EXIST foo.txt
    77      SHOULD_CONTAIN foo.txt "foo contents\n"
    78  
    79  ## Plug-in support
    80  
    81  There should plug-ins support for:
    82  
    83  Repo type:
    84  
    85  * Git -- Using /usr/bin/git or git.exe
    86  * Subversion
    87  * Mercurial
    88  * None (repoless)
    89  * Autodetect
    90  
    91  Encryption software:
    92  
    93  * GnuPG -- using /usr/bin/gpg{,2} or gpg.exe
    94  * golang.org/x/crypto/openpgp
    95  
    96  ## JSON or .txt
    97  
    98  The files in .blackbox are mostly .txt files.  Instead we should
    99  define a .json format, and only read the .txt file is the .json file
   100  doesn't exist.
   101  
   102  
   103  ## Repo-less mode
   104  
   105  I can't imagine storing files that aren't in a repo. I just put everything in repos lately. I use it more than I use NFS.  That said, I have received feedback that people would like the ability to disable automatic committing of files.
   106  
   107  I prefer the file commits to be automatic because when they were manual, people often accidentally committed the plaintext file instead of the GPG file.  Fixing such mistakes is a PITA and, of yeah, a big security nightmare.
   108  
   109  That said, I'm willing to have a "repo-less" mode.
   110  
   111  When this mode is triggered, no add/commit/ignore tasks are done.  The search for the keyrings directory still uses `$BLACKBOX_CONFIG` but if that is unset it looks for `.blackbox_config` in the current directory, then recursively `..` until we hit `/`.
   112  
   113  I think (but I'm not sure) this would benefit the entire system because it would force us to re-think what VCS actions are done when.
   114  
   115  I think (but I'm not sure) that a simple way to implement this would be to add an environment variable that overrides the automatic VCS detection. When set to "none", all VCS operations would basically become no-ops.  (This could be done by writing a plug-in that does nothing for all the `vcs_*` calls)
   116  
   117  
   118  Backwards compatibility: This would add a `none` VCS, not remove any existing functionality.
   119  
   120  
   121  ## Is "bash" the right language?
   122  
   123  `bash` is fairly universal. It even exists on Windows.  However it is not the right language for large systems. Writing the acceptance tests is quite a bear.  Managing `.gitignore` files in bash is impossible and the current implementation fails in many cases.
   124  
   125  `python` is my second favorite language. It would make the code cleaner and more testable. However it is not installed everywhere.  I would also want to write it in Python3 (why start a new project in Python2?) but sadly Python3 is less common.  It is a chicken vs. egg situation.
   126  
   127  `go` is my favorite language. I could probably rewrite this in go in a weekend. However, now the code is compiled, not interpreted. Therefore we lose the ability to just `git clone` and have the tools you want.  Not everyone has a Go compiler installed on every machine.
   128  
   129  The system is basically unusable on Windows without Cygwin or MINGW.  A rewrite in python or go would make it work better on Windows, which currently requires Cygwin or MinGW (which is a bigger investment than installing Python). On the other hand, maybe Ubuntu-on-Windows makes that a non-issue.
   130  
   131  As long as the code is in `bash` the configuration files like `blackbox-files.txt` and `blackbox-admins.txt` have problems.  Filenames with carriage returns aren't supported.  If this was in Python/Go/etc. those files could be json or some format with decent quoting and we could handle funny file names better. On the other hand, maybe it is best that we don't support funny filenames... we shouldn't enable bad behavior.
   132  
   133  How important is itto blackbox users that the system is written in `bash`?
   134  
   135  
   136  ## Ditch the project and use git-crypt
   137  
   138  People tell me that git-crypt is better because, as a plug-in, automagically supports `git diff`, `git log` and `git blame`.
   139  
   140  However, I've never used it so I don't have any idea whether git-crypt is any better than blackbox.
   141  
   142  Of course, git-crypt doesn't work with SVN, HG, or any other VCS.  Is blackbox's strong point the fact that it support so many VCS systems?  To be honest, it originally only supported HG and GIT because I was at a company that used HG but then changed to GIT.  Supporting anything else was thanks to contributors. Heck, HG support hasn't even been tested recently (by me) since we've gone all git where I work.
   143  
   144  How important is this to BlackBox users?