github.com/graybobo/golang.org-package-offline-cache@v0.0.0-20200626051047-6608995c132f/x/talks/2014/camlistore.slide (about)

     1  Camlistore: Android, ARM, App Engine, anywhere.
     2  
     3  25 June 2014
     4  Tags: camlistore
     5  
     6  Brad Fitzpatrick
     7  Gopher, Google
     8  @bradfitz
     9  bradfitz@golang.org
    10  http://bradfitz.com/
    11  http://camlistore.org/
    12  
    13  * Me
    14  
    15  - bradfitz
    16  - LiveJournal, memcached, OpenID, ...
    17  - Google
    18  - Go Standard Library, etc
    19  - Camlistore
    20  
    21  * Camlistore
    22  
    23  * Camlistore
    24  
    25  - "Your personal storage system for life"
    26  - 4 year+ open source project:
    27  .link http://camlistore.org
    28  - All in Go.
    29  - Early user of Go's standard library. Helped shape it.
    30  
    31  * Camlistore
    32  
    33  - stores, indexes, searches, syncs, shares content
    34  - can optionally model POSIX filesystems
    35  - lightweight object storage model, stores all mutations
    36  
    37  Stores on:
    38  
    39  - local disk, S3, Google Cloud Storage
    40  - MySQL, Postgres, SQLite, LevelDB,
    41  - MongoDB, App Engine,
    42  - itself,
    43  - encrypted, sharded, replicated, load balanced,
    44  - ...
    45  
    46  * Camlistore screenshots
    47  
    48  * Web UI
    49  
    50  .image camlistore/cam-mix-types.png _ 1000
    51  
    52  * Location search
    53  
    54  .image camlistore/cam-moscow.png 580 _
    55  
    56  * Panos
    57  
    58  .image camlistore/cam-pano.png _ 1000
    59  
    60  * Paris + Portrait
    61  
    62  .image camlistore/cam-paris-portrait.png _ 1000
    63  
    64  * Non-images
    65  
    66  .image camlistore/cam-other.png _ 1000
    67  
    68  * FUSE
    69  
    70  .image camlistore/cam-fuse.png _ 1000
    71  
    72  * Android
    73  
    74  .image camlistore/cam-android.png 600 _
    75  
    76  * Camlistore Demo
    77  
    78  * Camlistore does much, in many places
    79  
    80  * Camlistore
    81  
    82  does:
    83  
    84  - a web & API server
    85  - a FUSE filesystem (OS X, Linux)
    86  - command-line tools
    87  
    88  on:
    89  
    90  - desktops, personal servers
    91  - Raspberry Pi, etc (ARM)
    92  - Android
    93  - App Engine
    94  - Cloud (EC2, GCE)
    95  
    96  * Desktops, personal servers
    97  
    98  - Linux, FreeBSD, Mac, Windows, ...
    99  
   100    $ go {get,build,install,test} camlistore.org/{cmd,pkg,server}/...
   101  
   102  - Occasional use of +build windows, etc.
   103  - Nothing fancy.
   104  
   105  * Raspberry Pi
   106  
   107  - Most ARM devices are very slow
   108  - Cross compile from any OS, architecture on a fast machine:
   109  
   110    $ GOARCH=arm GOOS=linux go build camlistore.org/server/camlistored
   111  
   112  - ... then copy binary to the slow ARM device.
   113  - Popular helper tool:
   114  
   115  .link https://github.com/davecheney/golang-crosscompile
   116  
   117  - For debugging: ChromeOS + crouton is nice, pretty fast
   118  
   119  * App Engine
   120  
   121  - Just upload the code.
   122  - App Engine APIs require context plumbing.
   123  - Ugly hacks avoid context plumbing, but don't.
   124  .link https://developers.google.com/appengine/docs/go/reference#Context
   125  .link http://godoc.org/camlistore.org/pkg/context
   126  .link http://godoc.org/code.google.com/p/go.net/context
   127  - Restructure code to not assume long-running goroutines; handlers only
   128  - No unsafe, syscall, etc: use build tags on files:
   129  
   130    // +build !appengine
   131  
   132  * Cloud
   133  
   134  - Same as desktop.
   135  - Static binaries make deployment easy
   136  - Even easier: 
   137  - Embed resources (CSS, JS, HTML, images) with fileembed
   138  - Camlistore has: camlistore.org/pkg/fileembed + genfileembed
   139  
   140  * fileembed
   141  
   142  Definition:
   143  
   144    package fileembed
   145    func (f *Files) Open(filename string) (http.File, error)
   146  
   147  Use in Camlistore:
   148  
   149    package server
   150    uistatic.Files = &fileembed.Files{
   151        DirFallback: ui.uiDir,
   152        Listable:    true,
   153        // In dev_appserver, allow edit-and-reload without
   154        // restarting. In production, though, it's faster to just
   155        // slurp it in.
   156        SlurpToMemory: uistatic.IsProdAppEngine,
   157    }
   158  
   159  - Just reload in dev; embedded in binary in production.
   160  
   161  * Camlistore's genfileembed
   162  
   163    $ genfileembed ./path/to/dir
   164  
   165  - generates zindex.html.go
   166  
   167  * Embedding resources
   168  
   169  - Many options in the Go community
   170  - Coming to Go 1.4 built-in
   171  
   172  * Web server
   173  
   174  - Obvious
   175  - net/http is great: no need for Apache, nginx, etc
   176  - WebSockets support is great and easy
   177  
   178  * FUSE
   179  
   180  - Native filesystem on OS X & Linux
   181  - hanwen/gofuse (then only Linux) -> rsc/fuse (then only Mac) -> Camlistore fork (both) -> bazil.org (community maintained)
   182  - Easy to write fast filesystems in Go
   183  
   184  * Android
   185  
   186  - child process
   187  - on Application start, unzip arm binary to Unix (not media) filesystem 
   188  .link https://camlistore.googlesource.com/camlistore/+/master/clients/android/src/org/camlistore/UploadApplication.java UploadApplication.java
   189  .link https://camlistore.googlesource.com/camlistore/+/master/clients/android/src/org/camlistore/UploadThread.java UploadThread.java
   190  - build with Docker (demo)
   191  .link https://camlistore.googlesource.com/camlistore/+/master/clients/android/devenv/Dockerfile Dockerfile
   192  .link https://camlistore.googlesource.com/camlistore/+/master/clients/android/Makefile Makefile
   193  .link https://camlistore.googlesource.com/camlistore/+/master/clients/android/build-in-docker.pl build-in-docker.pl
   194  
   195  * Command-line tools
   196  
   197    $ cmdtool [--global=foo] subcmd [--subcommand opts]
   198  
   199  .link http://camlistore.org/pkg/cmdmain/
   200  .link https://camlistore.googlesource.com/camlistore/+/master/cmd/camtool/describe.go camtool's describe.go
   201  .link https://github.com/codegangsta/cli Popular: codegangsta/cli
   202  
   203  * In summary
   204  
   205  Go is awesome and runs lots of places, easily.
   206  
   207  - Web, CLI, FUSE, x86, ARM, Android, App Engine