github.com/rajasrinivasan/spm@v0.0.0-20200125100127-755649755f3f/README.md (about)

     1  # spm - Secure Package Manager
     2  
     3  ## Background
     4  
     5  Secure Package Manager (spm) evolved out of a need to distribute software updates to connected embedded devices with no practical user interfaces. It is derived from an actual functioning system, retaining only the features that are of general applicability.
     6  
     7  The goal was to be able to update the application software, distribute data files, and change configurations; being built utilizing the embedded linux platform yocto, eventually the goal incorporated the upgrade of the Operating System itself. This subsystem belongs in a larger context of a network distribution infrastructure and a resilient data transfer, installation and subsequent activation.
     8  
     9  High level requirements for such a subsystem then is to:
    10  - collect a set of files to be installed on a target system
    11  - enumerate the set of instructions on how to utilize the set of files
    12  - package them in an encrypted form - requiring a previously shared key to decrypt
    13  - a mechanism to authenticate each of the files at the destination
    14  - driving the execution of the installation steps
    15  
    16  ## Installation
    17  
    18  ### Systemwide Configuration
    19  
    20  A configuration file is used to specify parameters applicable to all the packages. An example is:
    21  
    22  ```
    23  #-----------------------------------------------------------------------
    24  #     This is an example configuration file systemwide for spm.
    25  #     Typical location $HOME/.spm.yaml
    26  #     can be overridden with the --config flag
    27  #-----------------------------------------------------------------------
    28  #     Environment Variables
    29  #     SPM_PKGPASSWORD   - the password for the encrytion of the spm file
    30  #-----------------------------------------------------------------------
    31  pubpkg: https://drive.google.com/
    32  pubart: https://drive.aws.com/
    33  
    34  package:
    35      format: tgz
    36      workarea: /tmp
    37  ```
    38  
    39  Of the parameters above, the workarea is sometimes overwritten to point to different partitions - in the case of embedded systems with limited storage e.g. on sdcards.
    40  
    41  ## Configuration of individual packages
    42  
    43  For each package that needs built, spm accepts a configuration file similar to:
    44  
    45  ```
    46  
    47  package:
    48      name: ServicePack
    49      version: 1.2
    50  
    51  contents:
    52      - from: /Volumes/Dev1/Ref/Books/acsac.pdf 
    53        to: /tmp/acsac.pdf 
    54  
    55  
    56  preinstall:
    57      - go version
    58      - ls -l /tmp
    59  
    60  postinstall:
    61      - ls /tmp
    62      - python --version
    63  ```
    64  ### Section: package
    65  Mostly intended for documentation.
    66  ### Section: contents
    67  The pair of from and to can be repeated any number of times in the contents section.
    68  ### Sections: preinstall and postinstall
    69  Each entry is a shell command applicable in the target system - typically a linux system. Depending on the context during installation, the commands may have to provide complete paths.
    70  
    71  ## Usage
    72  
    73  ../bin/spm
    74  Secure package manager helps prepare and distribute packages of applications 
    75  and/or data.
    76  
    77  Usage:
    78    spm [command]
    79  
    80  Available Commands:
    81    build       Build a secure package
    82    help        Help about any command
    83    install     Install the package
    84  
    85  Flags:
    86        --config string   config file (default is $HOME/.spm.yaml)
    87    -h, --help            help for spm
    88        --keep            keep workarea
    89  
    90  Use "spm [command] --help" for more information about a command.
    91  
    92  ### Building a package
    93  
    94  ../bin/spm build --help
    95  Create a secure package based on the configuration file provided.
    96  Optionally push the artifact(s) to a distribution server. 
    97  The first argument is the package spec file (ex spec.yaml)
    98  Output package name is the second argument
    99  
   100  Usage:
   101    spm build [flags]
   102  
   103  Flags:
   104    -h, --help   help for build
   105  
   106  Global Flags:
   107        --config string   config file (default is $HOME/.spm.yaml)
   108        --keep            keep workarea
   109  
   110  ### Installing a package
   111  
   112  ../bin/spm install --help
   113  Install the package provided first verifying the integrity of the artifacts. Argument
   114          is the package (.spm)
   115  
   116  Usage:
   117    spm install [flags]
   118  
   119  Flags:
   120    -h, --help   help for install
   121        --show   extract and show the contents. do not install. Implies --keep
   122  
   123  Global Flags:
   124        --config string   config file (default is $HOME/.spm.yaml)
   125        --keep            keep workarea
   126  
   127  ## Example Usage
   128  
   129  ### Package Configuration
   130  In the following package, one file is packaged and distributed to the target system at a specific location. There are a few
   131  shell commands specified to be executed before the file installation (Preinstall) and another set to execute after the file installations(Postinstall).
   132  
   133  ```
   134  package:
   135      name: ServicePack
   136      version: 1.2
   137  
   138  contents:
   139      - from: /Volumes/Dev1/Ref/Books/acsac.pdf 
   140        to: /tmp/acsac.pdf 
   141  
   142  preinstall:
   143      - go version
   144      - ls -l /tmp
   145  
   146  postinstall:
   147      - ls /tmp
   148      - python --version
   149  ```
   150  
   151  ### Build a package
   152  ```
   153  ../bin/spm build systest/sp.yaml systest/sp.spm
   154  Home dir is /Users/rajasrinivasan
   155  Using config file: /Users/rajasrinivasan/Prj/go/spm/example/.spm.yaml
   156  Pkg publish url=https://drive.google.com/ Artifacts=https://drive.aws.com/
   157  Pkg Password Thisisagoodpassword Workarea /tmp
   158  2020/01/22 14:46:10 Building package for configuration file systest/sp.yaml
   159  2020/01/22 14:46:10 Workarea created /tmp/spm400690855
   160  2020/01/22 14:46:10 Created dir /tmp/spm400690855/tmp/spm400690855/contents and /tmp/spm400690855/artifacts
   161  Loaded package File: systest/sp.yaml Name : ServicePack
   162  2020/01/22 14:46:10 Copying file /Volumes/Dev1/Ref/Books/acsac.pdf to /tmp/spm400690855/contents
   163  2020/01/22 14:46:10 Generating keys Private: /tmp/spm400690855/work/private.pem and Public: /tmp/spm400690855/contents/public.pem
   164  2020/01/22 14:46:11 Created keypair /tmp/spm400690855/work/private.pem and /tmp/spm400690855/contents/public.pem
   165  2020/01/22 14:46:11 Content file /tmp/spm400690855/contents/acsac.pdf
   166  2020/01/22 14:46:11 Content file /tmp/spm400690855/contents/public.pem
   167  2020/01/22 14:46:11 Files: [/tmp/spm400690855/contents/acsac.pdf /tmp/spm400690855/contents/public.pem]
   168  2020/01/22 14:46:11 Signing using /tmp/spm400690855/work/private.pem of 2 files
   169  2020/01/22 14:46:11 Loading private key /tmp/spm400690855/work/private.pem
   170  2020/01/22 14:46:11 Signing /tmp/spm400690855/contents/acsac.pdf creating /tmp/spm400690855/contents/acsac.pdf.sig
   171  2020/01/22 14:46:11 Datahash: 7f5b4e683df4120ddb5a2937259255e8cef209e16cd6fba948f964959e6c4eb5
   172  Signature: 2d114a1617cc2e007fceb25691e788780a35b5fdc650e33f5dd30f27d9026e58a0b59cb1d26a4771993550f09a7005db33d5e0e8eedf710560ecafec1f4b3d306110afebc887fc6b69b6ac0ecd5ee3395468daf5c730e2f65fdffaecd89247fbd2f88309d7eeb79949b0e6afa26f333cec292bb32fe7a2ec374fc7d24a7146aeec6ce29ac380185502d2d71fcdc1e186e7b16ef9d69f773b2b5667d1e5721f88adba84372de42dd9b27d9a77b0fa8ca15af8fa63bab988e579f83809a2998893448fec59e3387627a842d407719c3d445b0645319ee9e87e1972d46ef052e2f79e633b570e948693ac3ca3d3b567585a88ca34d1b4cc16de4ccf65c7be560cfb
   173  2020/01/22 14:46:11 Signing /tmp/spm400690855/contents/public.pem creating /tmp/spm400690855/contents/public.pem.sig
   174  2020/01/22 14:46:11 Datahash: bf54b4450a7438a38e769cd14199d747fb6f6c58e5757c348a81d9f5f7c0d179
   175  Signature: 247921b321eda649e1964f495faeb5de89318f6bffb428c4dcc402c6d4538fd0e39664199e445010103c45c9a0dd1d4a6476082e7ff931e0174de295acb71eecf0442a3a977a7cc76f347e81cab49feadf40de8e417fc059211f5fd6b3499c4f8d86faeaa8dbe605f40d0af4836272ad45aec7a67c3c935a1af02b3c2ff5b28f5d3e65f0b1de9fc86e9bb504b3014d4ce5936a948da224376d2912367c30d4c1ee5554930d4f32afae713fa8d85c334e9f3a99aaf5bdc4c98836173ec0cf2330ff8e3aa61190e40fd9327ed88afe029ecdb7a64002a720210891296588f01d900405050685095e4524fe212166ea99082400a9c9d645448dd5d5e2960e65dd02
   176  2020/01/22 14:46:11 Unique Id created b27d94be-d489-4f2f-9dca-9b954564843f
   177  2020/01/22 14:46:11 Saved manifest /tmp/spm400690855/contents/Packagefile
   178  2020/01/22 14:46:11 Signing /tmp/spm400690855/contents/Packagefile with /tmp/spm400690855/work/private.pem to generate /tmp/spm400690855/contents/Packagefile.sig
   179  2020/01/22 14:46:11 Loading private key /tmp/spm400690855/work/private.pem
   180  2020/01/22 14:46:11 Signing /tmp/spm400690855/contents/Packagefile creating /tmp/spm400690855/contents/Packagefile.sig
   181  2020/01/22 14:46:11 Datahash: f7f851d7031c00aa71d22028040023db6df6b4b2e29fe4664892f17b711c2f7b
   182  Signature: 3e8e185cd031516b4bed7516ac75a91a3ae6924a08161445bb4d59f7186718b2b168be90a764e8d4e4bca4bf3c0b971db4a26981f53ae169dc789423df5119188af5b46cbaec89047200a33978323b56800a094e14dd4d255950b5f24a1e2d82d31298c5f7f4fde9240b99ea7539e34ea1d4705ae6a04c44c5bd64136b8c65f2f66383f3a0d3f406d8c45f73f31a3a058cd8c9f32f31f9f5eabb2e6ed3d33021c9f5a15a962921da1d948a65bd61390fc9cf57795d11cbc56b5383098c59f2313dee0d906229d4c6cfb2074a218660907802fb6978c7e3959ad6c8610a3cb42cce76b3ecaf992be94fe5c6cee08749718c46af9a11e38ab8109f03aaccf8d285
   183  2020/01/22 14:46:11 Signed the Package file. Generated /tmp/spm400690855/contents/Packagefile.sig
   184  2020/01/22 14:46:11 Created /tmp/spm400690855/work/sp.spm
   185  2020/01/22 14:46:11 Adding Packagefile Size 390
   186  2020/01/22 14:46:11 Adding Packagefile.sig Size 256
   187  2020/01/22 14:46:11 Adding acsac.pdf Size 123519
   188  2020/01/22 14:46:11 Adding acsac.pdf.sig Size 256
   189  2020/01/22 14:46:11 Adding public.pem Size 418
   190  2020/01/22 14:46:11 Adding public.pem.sig Size 256
   191  2020/01/22 14:46:11 Created /tmp/spm400690855/work/sp.spm
   192  2020/01/22 14:46:11 Encrypt from: /tmp/spm400690855/work/sp.spm to systest/sp.spm passphrase Thisisagoodpassword
   193  2020/01/22 14:46:11 IV: d1aca762acc9cd49ca7d33b9558d9de8
   194  2020/01/22 14:46:11 Created systest/sp.spm
   195  2020/01/22 14:46:11 Removed /tmp/spm400690855
   196  ```
   197  
   198  ### Install the above package
   199  ```
   200  ../bin/spm install systest/sp.spm
   201  Home dir is /Users/rajasrinivasan
   202  Using config file: /Users/rajasrinivasan/Prj/go/spm/example/.spm.yaml
   203  Pkg publish url=https://drive.google.com/ Artifacts=https://drive.aws.com/
   204  Pkg Password Thisisagoodpassword Workarea /tmp
   205  2020/01/22 14:47:17 Installing package /Users/rajasrinivasan/Prj/go/spm/systest/sp.spm
   206  2020/01/22 14:47:17 Workarea created /tmp/spm839508907
   207  2020/01/22 14:47:17 Created dir /tmp/spm839508907/tmp/spm839508907/contents and /tmp/spm839508907/artifacts
   208  2020/01/22 14:47:17 Decrypt from: /Users/rajasrinivasan/Prj/go/spm/systest/sp.spm to /tmp/spm839508907/work/sp.spm passphrase Thisisagoodpassword
   209  2020/01/22 14:47:17 32 bytes read for password
   210  2020/01/22 14:47:17 16 bytes read for IV
   211  2020/01/22 14:47:17 IV: d1aca762acc9cd49ca7d33b9558d9de8
   212  2020/01/22 14:47:17 110232 bytes read
   213  2020/01/22 14:47:17 110232 bytes written
   214  2020/01/22 14:47:17 Decrypted /Users/rajasrinivasan/Prj/go/spm/systest/sp.spm to create /tmp/spm839508907/work/sp.spm
   215  2020/01/22 14:47:17 Extracting Packagefile
   216  2020/01/22 14:47:17 Extracting Packagefile.sig
   217  2020/01/22 14:47:17 Extracting acsac.pdf
   218  2020/01/22 14:47:17 Extracting acsac.pdf.sig
   219  2020/01/22 14:47:17 Extracting public.pem
   220  2020/01/22 14:47:17 Extracting public.pem.sig
   221  Loaded package File: /tmp/spm839508907/contents/Packagefile Name : ServicePack
   222  2020/01/22 14:47:17 Authenticating /tmp/spm839508907/contents/acsac.pdf signature /tmp/spm839508907/contents/acsac.pdf.sig publickey file /tmp/spm839508907/contents/public.pem
   223  2020/01/22 14:47:17 Loading public key /tmp/spm839508907/contents/public.pem
   224  2020/01/22 14:47:17 Public key file /tmp/spm839508907/contents/public.pem parsed
   225  2020/01/22 14:47:17 Verified the signature /tmp/spm839508907/contents/acsac.pdf.sig of file /tmp/spm839508907/contents/acsac.pdf
   226  2020/01/22 14:47:17 Authenticating /tmp/spm839508907/contents/Packagefile signature /tmp/spm839508907/contents/Packagefile.sig publickey file /tmp/spm839508907/contents/public.pem
   227  2020/01/22 14:47:17 Loading public key /tmp/spm839508907/contents/public.pem
   228  2020/01/22 14:47:17 Public key file /tmp/spm839508907/contents/public.pem parsed
   229  2020/01/22 14:47:17 Verified the signature /tmp/spm839508907/contents/Packagefile.sig of file /tmp/spm839508907/contents/Packagefile
   230  2020/01/22 14:47:17 Executing Preinstall steps
   231  2020/01/22 14:47:17 go version go1.12.1 darwin/amd64
   232  
   233  2020/01/22 14:47:17 lrwxr-xr-x@ 1 root  wheel  11 Feb  8  2019 /tmp -> private/tmp
   234  
   235  2020/01/22 14:47:17 
   236  2020/01/22 14:47:17 
   237  2020/01/22 14:47:17 Executing Postinstall steps
   238  2020/01/22 14:47:17 0E54B0DC-3D67-4903-99AE-F0D43B3655D2
   239  0F0C0FE4-C809-42F5-A2B8-BFB1A097224E
   240  2121E7F2-A1FC-4A1A-9BB1-7B1B3A919591
   241  4ADB67E3-6507-4735-8226-E4B2AC35E3B6
   242  612A3519-AB12-4F2F-9492-473A95074FA7
   243  9C96170E-6DD2-4027-98D5-023DF5261272
   244  9E4652A4-5EAE-4D2B-AAFD-10A030963615
   245  AE0F797D-7E7D-4C33-A489-727249DB451B
   246  BF9CEB3F-CFDC-46E4-AD51-D21CAE11203E
   247  DC263519-51BB-46E1-BF23-08B90B157DAB
   248  F21554BE-CD85-450C-938C-DCA11A67D796
   249  FB48D565-A389-47AB-B70A-E14EDC97CF23
   250  acsac.pdf
   251  adobesmuoutp3XmNBJ
   252  adobesmuoutp8HNNgv
   253  adobesmuoutpVfsj1m
   254  adobesmuoutpkqlY3T
   255  com.apple.launchd.6lDvSEZYde
   256  com.apple.launchd.JKHYdsQzV3
   257  ext
   258  powerlog
   259  spm839508907
   260  
   261  2020/01/22 14:47:17 Python 2.7.16
   262  
   263  2020/01/22 14:47:17 Removed /tmp/spm839508907
   264  ```
   265  
   266  ## Design Choices
   267  
   268  ### Digital Signatures for individual files
   269  
   270  spm generates a public and private key pair for every invocation. The private key is used to generate signature files for each of the content files. Then the private key file is discarded but the public key is saved in the package. Any tampering of any files then will be detected when the contents are authenticated with the signature files. Further details can be gleaned from sign.go.
   271  
   272  ### Container File
   273  
   274  All the contents are packaged up in a compressed tar file ie .tgz. The detailed format can be gleaned from pack.go.
   275  
   276  ### Encryption of the container file
   277  
   278  The current implementation uses the Output Feedback Mode [OFB](https://csrc.nist.gov/publications/detail/sp/800-38a/final). Further details can be gleaned from crypt.go.