github.com/artpar/rclone@v1.67.3/librclone/README.md (about) 1 # librclone 2 3 This directory contains code to build rclone as a C library and the 4 shims for accessing rclone from C and other languages. 5 6 **Note** for the moment, the interfaces defined here are experimental 7 and may change in the future. Eventually they will stabilise and this 8 notice will be removed. 9 10 ## C 11 12 The shims are a thin wrapper over the rclone RPC. 13 14 The implementation is based on cgo; to build it you need Go and a GCC compatible 15 C compiler (GCC or Clang). On Windows you can use the MinGW ports, e.g. by installing 16 in a [MSYS2](https://www.msys2.org) distribution (you may now install GCC in the newer 17 and recommended UCRT64 subsystem, however there were compatibility issues with previous 18 versions of cgo where, if not force rebuild with go build option `-a` helped, you had 19 to resort to the classic MINGW64 subsystem). 20 21 Build a shared library like this (change from .so to .dll on Windows): 22 23 go build --buildmode=c-shared -o librclone.so github.com/artpar/artpar/librclone 24 25 Build a static library like this (change from .a to .lib on Windows): 26 27 go build --buildmode=c-archive -o librclone.a github.com/artpar/artpar/librclone 28 29 Both the above commands will also generate `librclone.h` which should 30 be `#include`d in `C` programs wishing to use the library (with some 31 [exceptions](#include-file)). 32 33 The library will depend on `libdl` and `libpthread` on Linux/macOS, unless 34 linking with a C standard library where their functionality is integrated, 35 which is the case for glibc version 2.34 and newer. 36 37 You may add arguments `-ldflags -s` to make the library file smaller. This will 38 omit symbol table and debug information, reducing size by about 25% on Linux and 39 50% on Windows. 40 41 Note that on macOS and Windows the mount functions will not be available unless 42 you add additional argument `-tags cmount`. On Windows this also requires you to 43 first install the third party utility [WinFsp](http://www.secfs.net/winfsp/), 44 with the "Developer" feature selected, and to set environment variable CPATH 45 pointing to the fuse include directory within the WinFsp installation 46 (typically `C:\Program Files (x86)\WinFsp\inc\fuse`). See also the 47 [mount](/commands/rclone_mount/#installing-on-windows) documentation. 48 49 On Windows, when you build a shared library, you can embed version information 50 as binary resource. To do that you need to run the following command **before** 51 the build command. 52 53 ``` 54 go run bin/resource_windows.go -binary librclone.dll -dir librclone 55 ``` 56 57 ### Documentation 58 59 For documentation see the Go documentation for: 60 61 - [RcloneInitialize](https://pkg.go.dev/github.com/artpar/artpar/librclone#RcloneInitialize) 62 - [RcloneFinalize](https://pkg.go.dev/github.com/artpar/artpar/librclone#RcloneFinalize) 63 - [RcloneRPC](https://pkg.go.dev/github.com/artpar/artpar/librclone#RcloneRPC) 64 - [RcloneFreeString](https://pkg.go.dev/github.com/artpar/artpar/librclone#RcloneFreeString) 65 66 ### Linux C example 67 68 There is an example program `ctest.c`, with `Makefile`, in the `ctest` 69 subdirectory. It can be built on Linux/macOS, but not Windows without 70 changes - as described next. 71 72 ### Windows C/C++ guidelines 73 74 The official [C example](#linux-c-example) is targeting Linux/macOS, and will 75 not work on Windows. It is very possible to use `librclone` from a C/C++ 76 application on Windows, but there are some pitfalls that you can avoid by 77 following these guidelines: 78 - Build `librclone` as shared library, and use run-time dynamic linking (see [linking](#linking)). 79 - Do not try to unload the library with `FreeLibrary` (see [unloading](#unloading)). 80 - Deallocate returned strings with API function `RcloneFreeString` (see [memory management](#memory-management)). 81 - Define struct `RcloneRPCResult`, instead of including `librclone.h` (see [include file](#include-file)). 82 - Use UTF-8 encoded strings (see [encoding](#encoding)). 83 - Properly escape JSON strings, beware of the native path separator (see [escaping](#escaping)). 84 85 #### Linking 86 87 Use of different compilers, compiler versions, build configuration, and 88 dependency on different C runtime libraries for a library and the application 89 that references it, may easily break compatibility. When building the librclone 90 library with MinGW GCC compiler (via go build command), if you link it into an 91 application built with Visual C++ for example, there will be more than enough 92 differences to cause problems. 93 94 Linking with static library requires most compatibility, and is less likely to 95 work. Linking with shared library is therefore recommended. The library exposes 96 a plain C interface, and by using run-time dynamic linking (by using Windows API 97 functions `LoadLibrary` and `GetProcAddress`), you can make a boundary that 98 ensures compatibility (and in any case, you will not have an import library). 99 The only remaining concern is then memory allocations; you should make sure 100 memory is deallocated in the same library where it was allocated, as explained 101 [below](#memory-management). 102 103 #### Unloading 104 105 Do not try to unload the library with `FreeLibrary`, when using run-time dynamic 106 linking. The library includes Go-specific runtime components, with garbage 107 collection and other background threads, which do not handle unloading. Trying 108 to call `FreeLibrary` will crash the application. I.e. after you have loaded 109 `librclone.dll` into your application it must stay loaded until your application 110 exits. 111 112 #### Memory management 113 114 The output string returned from `RcloneRPC` is allocated within the `librclone` 115 library, and caller is responsible for freeing the memory. Due to C runtime 116 library differences, as mentioned [above](#linking), it is not recommended to do 117 this by calling `free` from the consuming application. You should instead use 118 the API function `RcloneFreeString`, which will call `free` from within the 119 `librclone` library, using the same runtime that allocated it in the first 120 place. 121 122 #### Include file 123 124 Do not include `librclone.h`. It contains some plain C, golang/cgo and GCC 125 specific type definitions that will not compile with all other compilers 126 without adjustments, where Visual C++ is one notable example. When using 127 run-time dynamic linking, you have no use of the extern declared functions 128 either. 129 130 The interface of librclone is so simple, that all you need is to define the 131 small struct `RcloneRPCResult`, from [librclone.go](librclone.go): 132 133 ```C++ 134 struct RcloneRPCResult { 135 char* Output; 136 int Status; 137 }; 138 ``` 139 140 #### Encoding 141 142 The API uses plain C strings (type `char*`, called "narrow" strings), and rclone 143 assumes content is UTF-8 encoded. On Linux systems this normally matches the 144 standard string representation, and no special considerations must be made. On 145 Windows it is more complex. 146 147 On Windows, narrow strings are traditionally used with native non-Unicode 148 encoding, the so-called ANSI code page, while Unicode strings are instead 149 represented with the alternative `wchar_t*` type, called "wide" strings, and 150 encoded as UTF-16. This means, to correctly handle characters that are encoded 151 differently in UTF-8, you will need to perform conversion at some level: 152 Conversion between UTF-8 encoded narrow strings used by rclone, and either ANSI 153 encoded narrow strings or wide UTF-16 encoded strings used in runtime function, 154 Windows API, third party APIs, etc. 155 156 #### Escaping 157 158 The RPC method takes a string containing JSON. In addition to the normal 159 escaping of strings constants in your C/C++ source code, the JSON needs its 160 own escaping. This is not a Windows-specific issue, but there is the 161 additional challenge that native filesystem path separator is the same as 162 the escape character, and you may end up with strings like this: 163 164 ```C++ 165 const char* input = "{" 166 "\"fs\": \"C:\\\\Temp\"," 167 "\"remote\": \"sub/folder\"," 168 "\"opt\": \"{\\\"showHash\\\": true}\"" 169 "}"; 170 ``` 171 172 With C++11 you can use raw string literals to avoid the C++ escaping of string 173 constants, leaving escaping only necessary for the contained JSON. 174 175 ## Example in golang 176 177 Here is a go example to help you move files : 178 179 ```go 180 func main() { 181 librclone.Initialize() 182 syncRequest: = syncRequest { 183 SrcFs: "<absolute_path>", 184 DstFs: ":s3,env_auth=false,access_key_id=<access>,secret_access_key=<secret>,endpoint='<endpoint>':<bucket>", 185 } 186 187 syncRequestJSON, err: = json.Marshal(syncRequest) 188 if err != nil { 189 fmt.Println(err) 190 } 191 192 out, status: = librclone.RPC("sync/copy", string(syncRequestJSON)) 193 fmt.Println("Got status : %d and output %q", status, out) 194 } 195 196 ``` 197 198 ## gomobile 199 200 The `gomobile` subdirectory contains the equivalent of the C binding but 201 suitable for using with [gomobile](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile) 202 using something like this. 203 204 gomobile bind -v -target=android -javapkg=org.rclone github.com/artpar/artpar/librclone/gomobile 205 206 The command generates an Android library (`aar`) that can be imported 207 into an Android application project. Librclone will be contained 208 within `libgojni.so` and loaded automatically. 209 210 ```java 211 // imports 212 import org.rclone.gomobile.Gomobile; 213 import org.rclone.gomobile.RcloneRPCResult; 214 215 // initialize rclone 216 Gomobile.rcloneInitialize(); 217 218 // call RC method and log response. 219 RcloneRPCResult response = Gomobile.rcloneRPC("core/version", "{}"); 220 Log.i("rclone", "response status: " + response.getStatus()); 221 Log.i("rclone", "output: " + response.getOutput()); 222 223 // Clean up when finished. 224 Gomobile.rcloneFinalize(); 225 ``` 226 227 This is a low level interface - serialization, job management etc must 228 be built on top of it. 229 230 iOS has not been tested (but should probably work). 231 232 Further docs: 233 234 - [gomobile main website](https://pkg.go.dev/golang.org/x/mobile/cmd/gomobile) 235 - [gomobile wiki](https://github.com/golang/go/wiki/Mobile) 236 - [go issue #16876](https://github.com/golang/go/issues/16876) where the feature was added 237 - [gomobile design doc](https://docs.google.com/document/d/1y9hStonl9wpj-5VM-xWrSTuEJFUAxGOXOhxvAs7GZHE/edit) for extra details not in the docs. 238 239 ## python 240 241 The `python` subdirectory contains a simple Python wrapper for the C 242 API using rclone linked as a shared library with `ctypes`. 243 244 You are welcome to use this directly. 245 246 This needs expanding and submitting to pypi... 247 248 ## Rust 249 250 Rust bindings are available in the `librclone` crate: https://crates.io/crates/librclone 251 252 ## PHP 253 254 The `php` subdirectory contains how to use the C library librclone in php through foreign 255 function interface (FFI). 256 257 Useful docs: 258 - [PHP / FFI](https://www.php.net/manual/en/book.ffi.php) 259 260 ## TODO 261 262 - Async jobs must currently be cancelled manually at the moment - RcloneFinalize doesn't do it. 263 - This will use the rclone config system and rclone logging system. 264 - Need examples showing how to configure things, 265