github.com/olivere/camlistore@v0.0.0-20140121221811-1b7ac2da0199/website/talks/2011-05-07-Camlistore-Sao-Paolo/index.html (about) 1 <!DOCTYPE html> 2 3 <!-- 4 Google I/O 2011 HTML slides template 5 URL: http://code.google.com/p/io-2011-slides/ 6 --> 7 8 <html> 9 <head> 10 <title>Camlistore</title> 11 12 <meta charset='utf-8' /> 13 <script src='slides.js'></script> 14 </head> 15 16 <style> 17 /* Your individual styles here, or just use inline styles if that’s 18 what you want. */ 19 .smaller { 20 font-size: 80%; 21 } 22 23 ul li ul { 24 margin-top: 1.5em; 25 margin-bottom: 1em; 26 } 27 ul li ul li { 28 margin-top: 1em; 29 font-size: 80%; 30 } 31 ul li ul.dense li { 32 margin-top: 0em; 33 margin-bottom: 0em; 34 font-size: 80%; 35 } 36 h1.center { 37 text-align: center; 38 font-style: italic; 39 } 40 </style> 41 42 <body style='display: none'> 43 44 <section class='slides layout-regular'> 45 46 <!-- Your slides (<article>s) go here. Delete or comment out the 47 slides below. --> 48 49 <article> 50 <h1> 51 Camlistore 52 </h1> 53 <p> 54 Brad Fitzpatrick 55 <br> 56 2011-05-07 57 </p> 58 59 <p><i><small>(use arrow keys or PgUp/PgDown to move slides)</small></i></p> 60 </article> 61 62 63 <article> 64 <h3> 65 Who am I? 66 </h3> 67 <ul class='nobuild'> 68 <li> 69 Brad Fitzpatrick <brad@danga.com> 70 </li> 71 <li>Perl Hacker since 1994</li> 72 <li>Projects: 73 <table><tr valign='top'> 74 <th>Danga / 6A (Perl)</th> 75 <th>Google</th> 76 </tr> 77 <td class='nobuild'> 78 <div>LiveJournal</div> 79 <div>memcached</div> 80 <div>Perlbal</div> 81 <div>MogileFS</div> 82 <div class='blue'>OpenID</div> 83 </td> 84 <td class='nobuild'> 85 <div><nobr>Social Graph API (<span class='blue'>XFN / FOAF</a>)</nobr></div> 86 <div class='blue'>WebFinger</div> 87 <div class='blue'>PubSubHubbub</div> 88 <div>Android</div> 89 <div>Go</div> 90 </td> 91 </table> 92 <div style='font-size: 70%; margin-top: 1em'>* <span class='blue'>decentralized social</span></div> 93 </li> 94 </ul> 95 </article> 96 97 <article> 98 <h3> 99 But why am I in Brazil? 100 </h3> 101 <ul class='nobuild'> 102 <li> 103 <i>"Hey, want to come speak at a Perl conference in Brazil?"</i> 104 </li> 105 <li>"Yes, totally, but... I don't write much Perl these days. :-("</li> <!-- " --> 106 <li style="margin-top: 2em"><i>"You could speak on memcached."</i></li> 107 <li>"But that's an old topic, no?"</li> 108 <li style="margin-top: 2em"><i>"You have any new project you're excited about?"</i></li> 109 </li> 110 </ul> 111 </article> 112 113 114 <article> 115 <h1 align='center'> 116 Camlistore! 117 </h1> 118 </article> 119 120 <article> 121 <h3> 122 Camlistore 123 </h1> 124 <ul> 125 <li>New open source project</li> 126 <li>Almost a year old</li> 127 <li>Still in development</li> 128 <li>Starting to be useful :-)</li> 129 <li>Hard to easily describe...</li> 130 </article> 131 132 <article> 133 <q> 134 Camlistore is a way to store, sync, share, model and back up content 135 </q> 136 <div class='author'> 137 camlistore.org 138 </div> 139 </article> 140 141 <article> 142 <h3> 143 Motivation 144 </h3> 145 <ul> 146 <li>I've written too many Content Management Systems 147 <ul> 148 <li>blogs, comments, photos, emails, backups, scanned paperwork, ...</li> 149 <li>is a scanned photo a scan, a photo, or a blog post? who cares.</li> 150 <li>write <b>one CMS to rule them all</b></li> 151 <li>... or at least a good framework for higher-level CMSes</li> 152 </ul> 153 </li> 154 </ul> 155 </article> 156 157 158 <article> 159 <h3> 160 Motivation (cont) 161 </h3> 162 <ul> 163 <li>I still want to help solve the Decentralized Social Network Problem 164 <ul> 165 <li>protocols, not companies</li> 166 <li>gmail, hotmail: hosted versions of SMTP, IMAP</li> 167 <li>... but I can run my own SMTP/IMAP server if I want.</li> 168 <li>... or change my SMTP/IMAP provider</li> 169 </ul> 170 </li> 171 </ul> 172 </article> 173 174 <article> 175 <h3> 176 Motivation (cont) 177 </h3> 178 <ul> 179 <li>I wanted something conceptually simple.</li> 180 <li>HTTP interfaces, not language-specific</li> 181 <li>I use lots of machines; don't want to think about sync or conflicts.</li> 182 <li>Data archaeology: should be easy and obvious to 183 reconstruct in 20 or 100 years</li> 184 </ul> 185 </article> 186 187 <article> 188 <h3> 189 The Product 190 </h3> 191 <ul> 192 <li>one private dumping ground to store anything</li> 193 <li>backups, filesystems, objects, photos, likes, bookmarks, shares, my website, ...</li> 194 <li>live backup my phone</li> 195 <li>live replicate / sync my dumping group between my house & laptop & Amazon & Google</li> 196 <li>web UI (ala gmail, docs.google.com, etc) or FUSE filesystem</li> 197 <li>Easy for end-users; powerful for dorks</li> 198 </ul> 199 </article> 200 201 <article> 202 <h3> 203 Security Model 204 </h3> 205 <ul> 206 <li><i><b>your</b></i> private repo, for life</li> 207 <li>everything private by default</li> 208 <li>grant access to specific objects/trees with friends or the world</li> 209 <li>web UI or CLI tools let you share</li> 210 </ul> 211 </article> 212 213 <article> 214 <h1 class='center'> 215 So what's with the silly name? 216 </h1> 217 </article> 218 219 <article> 220 <h3> 221 Camlistore 222 </h3> 223 <ul> 224 <li>Content-</li> 225 <li>Addressable</li> 226 <li>Multi-</li> 227 <li>Layer-</li> 228 <li>Indexed</li> 229 <li>Storage</li> 230 </ul> 231 </article> 232 233 <article> 234 <h3> 235 Content-Addressable 236 </h3> 237 <ul> 238 <li>At the core, everything is stored & addressed by its digest (e.g. SHA1, MD5, etc)</li> 239 <li>e.g. <tt class='smaller'>"sha1-0beec7b5ea3f0fdbc95d0dd47f3c5bc275da8a33"</tt> for the blob <tt class='smaller'>"foo"</tt></li> 240 <li>Great properties: 241 <ul> 242 <li>no versions of content: change it changes the new digest too</li> 243 <li>no versions: no sync conflicts</li> 244 <li>no versions: perfect caching (have it or don't)</li> 245 </ul> 246 </li> 247 </ul> 248 </article> 249 250 <article> 251 <h3> 252 Multi-Layer, Indexed 253 </h3> 254 <ul> 255 <li>Unix philosophy: small pieces with well-defined interfaces that can be chained or composed</li> 256 <li>Camlistore pieces include: 257 <ul class='dense'> 258 <li style='margin-top: 1em'>Blob storage: memory, disk, S3, Google, MySQL index, etc</li> 259 <li>Schema</li> 260 <li>Signing</li> 261 <li>Replication</li> 262 <li>Indexing: (e.g. replicate from disk to MySQL index)</li> 263 <li>Search</li> 264 <li>HTML UI</li> 265 </ul> 266 </li> 267 268 </ul> 269 </article> 270 271 <article> 272 <h3>Logically</h3> 273 <img src='arch.png' width='100%'/> 274 </article> 275 276 <article> 277 <h3>In reality</h3> 278 <ul> 279 <li>End-users: use a hosted version</li> 280 <li>Dorks: single server binary with all the logical pieces</li> 281 </ul> 282 </article> 283 284 <article> 285 <h2> 286 From the bottom up... 287 </h2> 288 </article> 289 290 <article> 291 <h2> 292 Blob Server 293 </h2> 294 </article> 295 296 <article> 297 <h3>Blob Server: how dumb it is</h3> 298 <ul> 299 <li>"Blob" == zero or more bytes. <i class='red'>no</i> meta-data</li> 300 <li>private operations, to owner of data only:</li> 301 <ul> 302 <li class='green'>get(blobref) → blob</li> 303 <li class='green'>stat(blobref+) → [(blobref, size), ...]</li> 304 <li class='green'>put(blobref, blob)</li> 305 <li class='green'>enumerate(..) → [(blobref, size)...] (sorted by blobref)</li> 306 </ul> 307 <li>no public (non-owner) access</li> 308 <li>HTTP interface: <tt>GET /camli/sha1-xxxxxxx HTTP/1.1</tt></li> 309 <li><span class='green'>delete(blobref)</span> is disabled by default, privileged op for GC or replication queues only</li> 310 </ul> 311 </article> 312 313 <article> 314 <h3>Blob Server: seriously, no metadata</h3> 315 <ul> 316 <li>no filenames</li> 317 <li>no "mime types"</li> 318 <li>no "{create,mod,access} time"</li> 319 <li>size is implicit</li> 320 <li>blob: just some bytes</li> 321 <li>metadata? layers above.</li> 322 </ul> 323 </article> 324 325 <article> 326 <h1 class='center'> 327 Uh, what can you do with that? 328 </h1> 329 </article> 330 331 <article> 332 <h3>Uh, what can you do with that?</h3> 333 <ul> 334 <li>with just a blob server?</li> 335 <li>not much</li> 336 <li>but let's start with an easy example...</li> 337 </ul> 338 </article> 339 340 <article> 341 <h1 class='center'> 342 Filesystem Backups 343 </h1> 344 </article> 345 346 <article> 347 <h3>Filesystem Backups</h3> 348 <ul> 349 <li>previous project: brackup 350 <ul> 351 <li>good: Perl, slide/dice/encrypt S3 backup, content-addressed, good iterative backups</li> 352 <li>bad: large (several MB) "backup manifest" text files 353 </ul> 354 </li> 355 356 <li>fossil/venti, git, etc: directories content-addressed by content of their children, hash trees, etc</li> 357 <li>git: "tree objects", "commmit objects", etc</li> 358 <li>Camlistore: "schema blobs"</li> 359 </ul> 360 </article> 361 362 <article> 363 <h3>Schema: how to model your content</h3> 364 <ul> 365 <li>Camlistore defines <i>one possible</i> schema</li> 366 <li>but blobserver doesn't know about it all</li> 367 <li>tools generate schema,</li> 368 <li>indexer + search understand the schema.</li> 369 </ul> 370 </article> 371 372 <article> 373 <h3>Schema Blobs</h3> 374 <ul> 375 <li>so if all blobs are just dumb blobs of bytes with no metadata,</li> 376 <li>how do you store metadata?</li> 377 <li>as blobs themselves!</li> 378 </ul> 379 </article> 380 381 <article> 382 <h3>Minimal Schema Blob</h3> 383 384 <section> 385 <pre>{ 386 "camliVersion": 1, 387 "camliType": "whatever" 388 }</pre> 389 </section> 390 <p>Whitespace doesn't matter. Just must be valid JSON in its 391 entirety. Use whatever JSON libraries you've got.</p> 392 393 <p>That one is named<br/><tt class='smaller'>sha1-19e851fe3eb3d1f3d9d1cefe9f92c6f3c7d754f6</tt></p> 394 <p>or perhaps: <tt class='smaller'>sha512-2c6746aba012337aaf113fd63c24d994a0703d33eb5d6ed58859e45dc4e02dcf<br/>dae5c4d46c5c757fb85d5aff342245fe4edb780c028a6f3c994c1295236c931e</tt></p> 395 396 </article> 397 398 <!-- END --> 399 400 <article> 401 <h3>Schema blob; type "file"</h3> 402 <section><pre>{"camliVersion": 1, 403 <span style='background: #fff'>"camliType": "file",</span> 404 "fileName": "foo.dat", 405 "unixPermission": "0644", 406 ..., 407 "size": 6000133, 408 "contentParts": [ 409 {"blobRef": "sha1-...dead", "size": 111}, 410 {"blobRef": "sha1-...beef", "size": 5000000, "offset": 492 }, 411 {"size": 1000000}, 412 {"blobRef": "digalg-blobref", "size": 22}, 413 ] 414 }</pre></section> 415 </article> 416 417 <article> 418 <h3>Schema blob; type "directory"</h3> 419 <section><pre>{"camliVersion": 1, 420 <span style='background: #fff'>"camliType": "directory",</span> 421 "fileName": "foodir", 422 "unixPermission": "0755", 423 ..., 424 "entries": <span style='background: #fff'>"sha1-c3764bc2138338d5e2936def18ff8cc9cda38455"</span> 425 }</pre></section> 426 </article> 427 428 <article> 429 <h3>Schema blob; type "static-set"</h3> 430 <section><pre>{"camliVersion": 1, 431 <span style='background: #fff'>"camliType": "static-set",</span> 432 "members": [ 433 "sha1-xxxxxxxxxxxx", 434 "sha1-xxxxxxxxxxxx", 435 "sha1-xxxxxxxxxxxx", 436 "sha1-xxxxxxxxxxxx", 437 "sha1-xxxxxxxxxxxx", 438 "sha1-xxxxxxxxxxxx", 439 ] 440 }</pre></section> 441 </article> 442 443 <article> 444 <h3> 445 Backup a directory... 446 </h3> 447 <section><pre>$ camput --file $HOME 448 sha1-8659a52f726588dc44d38dfb22d84a4da2902fed</pre></section> 449 450 <p>(like git/hg/fossil, that identifier represents everything down.)</p> 451 452 <p>Iterative backups are cheap, easy identifier to share, etc</p> 453 454 <p>But how will you remember that identifier? (later)</p> 455 </article> 456 457 <article> 458 <h3> 459 But what about mutable data? 460 </h3> 461 <ul> 462 <li>immutable data is easy to represent & reference</li> 463 <ul> 464 <li><tt class='smaller'>sha1-8659a52f726588dc44d38dfb22d84a4da2902fed</tt> is an immutable snapshot</li> 465 </ul> 466 <li>how to represent mutable data in an immutable, content-addressed world?</li> 467 <li>how to share a reference to a mutable object when changing an object mutates its name?</li> 468 </ul> 469 </article> 470 471 <article> 472 <h1 class='center'> 473 Objects & "Permanodes" 474 </h1> 475 </article> 476 477 <article> 478 <h3> 479 Terminology 480 </h3> 481 <ul> 482 <li><span class='red'>blob</span>: just dumb, immutable series of bytes</li> 483 <li><span class='red'>schema blob</span>: a blob that's a valid JSON object w/ camliVersion & camliType</li> 484 <li><span class='red'>signed schema blob</span> aka "<span class='red'>claim</span>": a schema blob with an embedded OpenPGP signature</li> 485 <li><span class='red'>object</span>: something mutable. represented as an anchor "<span class='blue'>permanode</span>" + a set of mutations (<span class='blue'>claims</span>)</li> 486 <li><span class='red'>permanode</span>: a stable reference. an anchor. just a <span class='blue'>signed schema blob</span>, but of almost no content...</li> 487 </ul> 488 </article> 489 490 <article> 491 <h3> 492 Permanode 493 </h3> 494 <section><pre><span style='font-weight: bold' class='blue'>$ camput --permanode</span> 495 sha1-ea799271abfbf85d8e22e4577f15f704c8349026 496 497 <span style='font-weight: bold' class="blue">$ camget sha1-ea799271abfbf85d8e22e4577f15f704c8349026</span> 498 <span style="background: #ff7">{"camliVersion": 1, 499 "camliSigner": "sha1-c4da9d771661563a27704b91b67989e7ea1e50b8", 500 <span style='font-weight: bold'>"camliType": "permanode"</span>, 501 "random": "oj)r}$Wa/[J|XQThNdhE"</span> 502 ,"camliSig":"iQEcBAABAgAGBQJNRxceAAoJEGjzeDN/6vt8ihIH/Aov7FRIq4dODAPWGDwqL 503 1X9Ko2ZtSSO1lwHxCQVdCMquDtAdI3387fDlEG/ALoT/LhmtXQgYTt8QqDxVdu 504 EK1or6/jqo3RMQ8tTgZ+rW2cj9f3Q/dg7el0Ngoq03hyYXdo3whxCH2x0jajSt4RCc 505 gdXN6XmLlOgD/LVQEJ303Du1OhCvKX1A40BIdwe1zxBc5zkLmoa8rClAlHdqwo 506 gxYFY4cwFm+jJM5YhSPemNrDe8W7KT6r0oA7SVfOan1NbIQUel65xwIZBD0ah 507 CXBx6WXvfId6AdiahnbZiBup1fWSzxeeW7Y2/RQwv5IZ8UgfBqRHvnxcbNmScrzl 508 p3V3ZoY"}</pre></section> 509 510 </article> 511 512 <article> 513 <h3> 514 Backup a directory... 515 </h3> 516 <section><pre><span style='font-weight: bold'>$ camput --file $HOME</span> 517 sha1-8659a52f726588dc44d38dfb22d84a4da2902fed 518 <span style='font-weight: bold'>$ camput --permanode --file $HOME</span> 519 sha1-ea799271abfbf85d8e22e4577f15f704c8349026 520 <span style='font-weight: bold'>$ camput --permanode --name="Brad's home directory" --file $HOME</span> 521 sha1-ea799271abfbf85d8e22e4577f15f704c8349026</pre></section> 522 <ul> 523 <li>all the file data blobs, file/dir schema blobs,</li> 524 <li>a new permanode, owned by you</li> 525 <li>a mutation: permanode's content attribute == directory root</li> 526 <li>a mutation: permanode's name attribute == "Brad's home directory"</li> 527 </ul> 528 </article> 529 530 <article class='fill'> 531 <p><img src="fsbackup.png" height="100%"/></p> 532 </article> 533 534 <article> 535 <h3>Aside: Garbage Collection</h3> 536 <ul> 537 <li>Permanodes are (optionally) GC roots,</li> 538 <li>or anything signed by you.</li> 539 <li>If not a blob isn't reachable by a signed root, can be deleted.</li> 540 <li>If you want to keep a plain "dumb" blob, you should create a "keep" claim for it, or a permanode.</li> 541 </ul> 542 </article> 543 544 <article> 545 <h1 class='center'> 546 Modeling non-filesystem objects 547 </h1> 548 </article> 549 550 <article> 551 <h3>Example: a photo gallery</h3> 552 <ul> 553 <li>Photos are objects</li> 554 <li>Galleries (sets) are objects</li> 555 <li>Photos are members of galleries</li> 556 <li>Photos & galleries have attributes (single-valued: "title", multi-valued: "tag")</li> 557 <li>Photos might be updated over time: 558 <ul> 559 <li>EXIF GPS updated, cropping, white balance</li> 560 <li>don't want to break links!</li> 561 </ul> 562 </li> 563 </ul> 564 </article> 565 566 <article class='fill'> 567 <p><img src="blobjects.png" width="100%"/></p> 568 </article> 569 570 <article> 571 <h1 class='center'> 572 How to make sense of that? 573 </h1> 574 </article> 575 576 <article> 577 <h1 class='center'> 578 Indexing & Search 579 </h1> 580 </article> 581 582 <article> 583 <h3> 584 Indexing: summary 585 </h3> 586 <p style='margin-top: 2em'>For each blob, build an index of: 587 <ul> 588 <li>directed graph of inter-blob references</li> 589 <li>(permanode, time) => resolved attributes</li> 590 <li>(permanode, time) => set memberships</li> 591 <li>etc...</li> 592 </ul> 593 </article> 594 595 <article> 596 <h3> 597 Indexing & Replication 598 </h3> 599 <ul> 600 <li>indexing is real-time, no polling</li> 601 <li>MySQL index speaks the blob server protocol</li> 602 <li>just replicate <i>to</i> the index (MySQL, etc) just like other blob servers (Amazon S3, etc)</li> 603 </ul> 604 <center><img src='repl.png' /></center> 605 </article> 606 607 <article> 608 <h3> 609 Replication Implementation 610 </h3> 611 <ul> 612 <li>cold bootstrap: <tt class='green'>enumerate()</tt> (sorted) all blobs from <tt class='red'>src</tt> and <tt class='red'>dst</tt>, copy all blobs that <tt>dst</tt> doesn't have.</tt> 613 <li>more efficient: use multiple machines, starting at <tt class='blue'>sha1-0*</tt>, <tt class='blue'>sha1-1*</tt>, <tt class='blue'>sha1-2*</tt>, ... etc</li> 614 <li>once in-sync, for each <tt class='red'>(src, dst)</tt> replication pair, keep a <tt class='red'>src_to_dst_QUEUE</tt> namespace on <tt class='red'>src</tt>,</li> 615 <li>all new blobs to <tt class='red'>src</tt> also go into <tt class='red'>src_to_dst_QUEUE</tt> (refcount, hardlink, etc)</li> 616 <li>real-time watch <tt class='red'>src_to_dst_QUEUE</tt> & replicate & delete from the queue. or re-enumerate just the queue. 617 </ul> 618 </article> 619 620 <article> 621 <h3> 622 Search 623 </h3> 624 <ul> 625 <li>Permanodes created by $who, sorted by date desc, type "photo", tagged "funny"</li> 626 <li>My recent backups with attribute "hostname" == "camlistore.org",</l> 627 <li>All friends' galleries in which this photo appears,</li> 628 <li>etc...</li> 629 </ul> 630 <p>...similar to your email, or docs.google.com. "My stuff" or "My bookmarks".</p> 631 </article> 632 633 <article> 634 <h3> 635 Privacy Model 636 </h3> 637 <ul> 638 <li>all your blobs & objects & searches are private</li> 639 <li>nothing is public by default</li> 640 </ul> 641 </article> 642 643 <article> 644 <h1> 645 What if you want to share with friends, or globally publish something? 646 </h1> 647 </article> 648 649 <article> 650 <h3> 651 Sharing & Share Blobs 652 </h3> 653 <p>the act of sharing involves creating a new <span class='red'>share claim</span>, just another blob, signed.</p> 654 <p>here is: <a href="http://camlistore.org:3179/camli/sha1-071fda36c1bd9e4595ed16ab5e2a46d44491f708"><tt class='smaller'>sha1-071fda36c1bd9e4595ed16ab5e2a46d44491f708</tt></a>:</p> 655 <section><pre>{"camliVersion": 1, 656 "authType": <span style="background: #fff">"haveref"</span>, 657 "camliSigner": "sha1-f019d17dd308eebbd49fd94536eb67214c2f0587", 658 "camliType": "share", 659 "target": "<a style="background: #fff" href="http://camlistore.org:3179/camli/sha1-0e5e60f367cc8156ae48198c496b2b2ebdf5313d">sha1-0e5e60f367cc8156ae48198c496b2b2ebdf5313d</a>", 660 "transitive": <span style="background: #fff">true</span> 661 ,"camliSig":"iQEcBAABAgAGBQJNQJGuAAoJEIUeCLJL7Fq1EuAIAL/nGoX8caGaANnam0bcIQT7C61wXMRW4qCCaFW+w67ys5z4ztfnTPKwL9ErzMF8Hd32Xe/bVcF6ZL38x/axqI7ehxN8lneKGQNoEdZDA9i752aAr0fkAba6eDehoOj9F4XxOzk3iVrq445jEXtu/+twamHV3UfRozWK1ZQb57dM+cRff47M/Y6VIBRSgW2BrABjuBs8G6PiKxycgh1mb+RL8f9KG+HB/yFuK37YJqZ0zU2OTRp6ELiOgTxbeg99koV9Duy4f4mQgxQgli46077Sv/ujzIeVbmdFL3OenGEzQnyKG0fhf8fa5WkED0XfH7zibAHLiSq3O7x11Q0406U==ANug"}</pre></section> 662 Target w/ ?via= parameter: <a href="http://camlistore.org:3179/camli/sha1-0e5e60f367cc8156ae48198c496b2b2ebdf5313d?via=sha1-071fda36c1bd9e4595ed16ab5e2a46d44491f708">sha1-0e5e60f?via=sha1-071fda</a> & <a href="http://camlistore.org:3179/camli/sha1-3dc1d1cfe92fce5f09d194ba73a0b023102c9b25?via=sha1-071fda36c1bd9e4595ed16ab5e2a46d44491f708,sha1-0e5e60f367cc8156ae48198c496b2b2ebdf5313d">next hop</a> 663 </article> 664 665 <article> 666 <h3> 667 Sharing Details & Implementation 668 </h3> 669 <ul> 670 <li>blobserver is private-only. the <span class='red'>frontend</span> mediates access to the world, checks authentication, or lack thereof.</li> 671 <li>all non-owner requests must present a share blob's blobref as an access token</li> 672 <li>that share blob dictates: 673 <ul> 674 <li>what sort of authenticatation is required (or "<tt class="green">haveref</tt>" for none, like a secret link)</li> 675 <li>which blob(s) are granted access (the "<tt class='green'>transitive</tt>" option)</li> 676 </ul> 677 </li> 678 <li>requests for a blob must include the path to get there, from the share root</li> 679 </ul> 680 </article> 681 682 <article> 683 <h3> 684 What can be shared 685 </h3> 686 <ul> 687 <li>Share a single blob, 688 <li>Share a subtree, 689 <li>Share a <i>search query</i> and its results' reachable blobs</li> 690 <li>... give out [world, girlfriend] access to all pictures you take on your phone, in real-time</li> 691 </ul> 692 </article> 693 694 <article> 695 <h2> 696 Project Status 697 </h2> 698 </article> 699 700 <article> 701 <h3> 702 Project Status 703 </h3> 704 <ul> 705 <li>Blobstore, Go (any OS), can store on disk, s3, mysqlindex 706 <li>Blobstore, Python (App Engine only) can store on Google 707 <li>Perl tests for two blob stores 708 <li>Android uploader (Java) 709 <li>Bunch of Go libraries / command-line tools: sync, put, get 710 <li>FUSE filesystem (read-only, currently) 711 <li>Search: basics working. more queries looks easy now. 712 <li>Simple, self-contained everything binary (blob storage, sharing, search, index, frontend) for early adopters: ~95% 713 <li>Web UI / JavaScript APIs: in progress 714 </ul> 715 </article> 716 717 <article> 718 <h3> 719 In Review 720 </h3> 721 <ul> 722 <li>You own all your blobs; everything is private by default.</li> 723 <li>Mutable objects are made of mutation claim blobs.</li> 724 <li>Sync is trivial: either you have it or you don't</li> 725 <li>Some blobs are signed</li> 726 <li>Indexing & search to find your blobs / roots</li> 727 <li>To share you must create a declaration of sharing ...</li> 728 <li>... and the system will only allow access if such claims exist.</li> 729 <li>Decentralized, but hostable. You can run your own server (with no central 730 company or point of control), but you can also let somebody else do it for 731 you, like email.</li> 732 </ul> 733 </article> 734 735 <article> 736 <h3> 737 Thank you! 738 </h3> 739 740 <p class='smaller'>Brad Fitzpatrick, <a href="mailto:brad@danga.com">brad@danga.com</a>; Want to help? More info: <a href="http://camlistore.org/">camlistore.org</a></p> 741 <img src='arch.png' width='100%'/> 742 </article> 743 744 </section> 745 746 </body> 747 </html>