github.com/olivere/camlistore@v0.0.0-20140121221811-1b7ac2da0199/doc/protocol/blob-upload-protocol.txt (about)

     1  Uploading a single blob is done in two parts:
     2  
     3  1) Optional: see if the server already has it with a HEAD request to
     4     its blob URL, or do a multi-stat request with a single blob. If the
     5     server has it, you're done. Blobs are content-addressable and have
     6     no metadata or version, so if the server has it, it has the right
     7     version.
     8  
     9  2) PUT that blob to its blob URL (or do a multipart batch put of a
    10     single blob)
    11  
    12  
    13  When uploading multiple blobs (the common case), the fastest option
    14  depends on whether or not you're using a modern HTTP transport
    15  (e.g. SPDY).  If your client and server don't support SPDY, you want
    16  to use the batch stat and batch upload endpoints, which hopefully can
    17  die when the future finishes arriving.
    18  
    19  If you have SPDY, uploading 100 blobs is just like uploading 100
    20  single blobs, but all at once. Send all your 100 HEAD requests at
    21  once, wait 1 RTT for all 100 replies, and then send then the <= 100
    22  PUT requests with the blobs that the server didn't have.
    23  
    24  If you DON'T have SPDY on both sides, you want to use the batch stat
    25  and batch upload endpoints, described below.
    26  
    27  ============================================================================
    28  Preupload request:
    29  ============================================================================
    30  
    31  (see blob-stat-protocol.txt)
    32  
    33  ============================================================================
    34  Batch upload request:
    35  ============================================================================
    36  
    37  Things to note about the request:
    38  
    39     * You do a POST to $BLOB_ROOT/camli/upload where $BLOB_ROOT is the
    40       blobserver's root path. You can get the path from
    41       performing "discovery" on the server and getting the
    42       "blobRoot" value. See discovery.txt.
    43  
    44     * You MUST provide a "name" parameter in each multipart part's
    45       Content-Disposition value.  The part's name matters and is the
    46       blobref ("digest-hexhexhexhex") of your blob.  The bytes MUST
    47       match the blobref and the server MUST reject it if they don't
    48       match.
    49  
    50     * You (currently) MUST provide a Content-Type for each multipart
    51       part.  It doesn't matter what it is (it's thrown away), but it's
    52       necessary to satisfy various HTTP libraries.  Easiest is to just
    53       set it to "application/octet-stream" Server implementions SHOULD
    54       fail if you clients forget it, to encourage clients to remember
    55       it for compatibility with all blob servers.
    56  
    57     * You (currently) MUST provide a "filename" parameter in each
    58       multipart's Content-Disposition value, unique per blob, but it
    59       will also be thrown away and exists purely to satisfy various
    60       HTTP libraries (mostly App Engine).  It's recommended to either
    61       set this to an increasing number (e.g. "blob1", "blob2") or just
    62       repeat the blobref value here.
    63  
    64     * The total size of a batch upload HTTP request, including headers
    65       and body (including MIME bits) should not exceed 32 MB.  (A
    66       single blob can be at most 16 MB, but will in practice be much
    67       smaller: claims will be at most ~1 KB, and file chunks are
    68       typically at most 64 KB or 256 KB)
    69  
    70  Some of these requirements may be relaxed in the future.
    71  
    72  Example:
    73  
    74  POST /$BLOB_SERVER_PATH_ROOT/camli/upload HTTP/1.1
    75  Host: upload-server.example.com
    76  Content-Type: multipart/form-data; boundary=randomboundaryXYZ
    77  
    78  --randomboundaryXYZ
    79  Content-Disposition: form-data; name="sha1-9b03f7aca1ac60d40b5e570c34f79a3e07c918e8"; filename="blob1"
    80  Content-Type: application/octet-stream
    81  
    82  (binary or text blob data)
    83  --randomboundaryXYZ
    84  Content-Disposition: form-data; name="sha1-deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"; filename="blob2"
    85  Content-Type: application/octet-stream
    86  
    87  (binary or text blob data)
    88  --randomboundaryXYZ--
    89  
    90  -----------------------------------------------------
    91  Response (status may be a 200 or a 303 to this data)
    92  -----------------------------------------------------
    93  
    94  HTTP/1.1 200 OK
    95  Content-Type: text/plain
    96  
    97  {
    98     "received": [
    99        {"blobRef": "sha1-9b03f7aca1ac60d40b5e570c34f79a3e07c918e8",
   100         "size": 12312},
   101        {"blobRef": "sha1-deadbeefdeadbeefdeadbeefdeadbeefdeadbeef",
   102         "size": 29384933}
   103     ]
   104  }
   105  
   106  Response keys:
   107  
   108     received         required   Array of {"blobRef": BLOBREF, "size": INT_bytes}
   109                                 for blobs that were successfully saved. Empty
   110                                 list in the case nothing was received.
   111     errorText        optional   String error message for protocol errors
   112                                 not relating to a particular blob.
   113                                 Mostly for debugging clients.  
   114  
   115  If connection drops during a POST to an upload URL, you should re-do a
   116  stat request to verify which objects were received by the server
   117  and which were not.  Also, the URL you received from stat before
   118  might no longer work, so stat is required to a get a valid upload
   119  URL.
   120  
   121  For information on resuming truncated uploads, read blob-upload-resume.txt
   122