github.com/10XDev/rclone@v1.52.3-0.20200626220027-16af9ab76b2a/docs/content/commands/rclone_dedupe.md (about)

     1  ---
     2  title: "rclone dedupe"
     3  description: "Interactively find duplicate files and delete/rename them."
     4  slug: rclone_dedupe
     5  url: /commands/rclone_dedupe/
     6  # autogenerated - DO NOT EDIT, instead edit the source code in cmd/dedupe/ and as part of making a release run "make commanddocs"
     7  ---
     8  # rclone dedupe
     9  
    10  Interactively find duplicate files and delete/rename them.
    11  
    12  ## Synopsis
    13  
    14  
    15  By default `dedupe` interactively finds duplicate files and offers to
    16  delete all but one or rename them to be different. Only useful with
    17  Google Drive which can have duplicate file names.
    18  
    19  In the first pass it will merge directories with the same name.  It
    20  will do this iteratively until all the identical directories have been
    21  merged.
    22  
    23  The `dedupe` command will delete all but one of any identical (same
    24  md5sum) files it finds without confirmation.  This means that for most
    25  duplicated files the `dedupe` command will not be interactive.  You
    26  can use `--dry-run` to see what would happen without doing anything.
    27  
    28  Here is an example run.
    29  
    30  Before - with duplicates
    31  
    32      $ rclone lsl drive:dupes
    33        6048320 2016-03-05 16:23:16.798000000 one.txt
    34        6048320 2016-03-05 16:23:11.775000000 one.txt
    35         564374 2016-03-05 16:23:06.731000000 one.txt
    36        6048320 2016-03-05 16:18:26.092000000 one.txt
    37        6048320 2016-03-05 16:22:46.185000000 two.txt
    38        1744073 2016-03-05 16:22:38.104000000 two.txt
    39         564374 2016-03-05 16:22:52.118000000 two.txt
    40  
    41  Now the `dedupe` session
    42  
    43      $ rclone dedupe drive:dupes
    44      2016/03/05 16:24:37 Google drive root 'dupes': Looking for duplicates using interactive mode.
    45      one.txt: Found 4 duplicates - deleting identical copies
    46      one.txt: Deleting 2/3 identical duplicates (md5sum "1eedaa9fe86fd4b8632e2ac549403b36")
    47      one.txt: 2 duplicates remain
    48        1:      6048320 bytes, 2016-03-05 16:23:16.798000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
    49        2:       564374 bytes, 2016-03-05 16:23:06.731000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
    50      s) Skip and do nothing
    51      k) Keep just one (choose which in next step)
    52      r) Rename all to be different (by changing file.jpg to file-1.jpg)
    53      s/k/r> k
    54      Enter the number of the file to keep> 1
    55      one.txt: Deleted 1 extra copies
    56      two.txt: Found 3 duplicates - deleting identical copies
    57      two.txt: 3 duplicates remain
    58        1:       564374 bytes, 2016-03-05 16:22:52.118000000, md5sum 7594e7dc9fc28f727c42ee3e0749de81
    59        2:      6048320 bytes, 2016-03-05 16:22:46.185000000, md5sum 1eedaa9fe86fd4b8632e2ac549403b36
    60        3:      1744073 bytes, 2016-03-05 16:22:38.104000000, md5sum 851957f7fb6f0bc4ce76be966d336802
    61      s) Skip and do nothing
    62      k) Keep just one (choose which in next step)
    63      r) Rename all to be different (by changing file.jpg to file-1.jpg)
    64      s/k/r> r
    65      two-1.txt: renamed from: two.txt
    66      two-2.txt: renamed from: two.txt
    67      two-3.txt: renamed from: two.txt
    68  
    69  The result being
    70  
    71      $ rclone lsl drive:dupes
    72        6048320 2016-03-05 16:23:16.798000000 one.txt
    73         564374 2016-03-05 16:22:52.118000000 two-1.txt
    74        6048320 2016-03-05 16:22:46.185000000 two-2.txt
    75        1744073 2016-03-05 16:22:38.104000000 two-3.txt
    76  
    77  Dedupe can be run non interactively using the `--dedupe-mode` flag or by using an extra parameter with the same value
    78  
    79    * `--dedupe-mode interactive` - interactive as above.
    80    * `--dedupe-mode skip` - removes identical files then skips anything left.
    81    * `--dedupe-mode first` - removes identical files then keeps the first one.
    82    * `--dedupe-mode newest` - removes identical files then keeps the newest one.
    83    * `--dedupe-mode oldest` - removes identical files then keeps the oldest one.
    84    * `--dedupe-mode largest` - removes identical files then keeps the largest one.
    85    * `--dedupe-mode smallest` - removes identical files then keeps the smallest one.
    86    * `--dedupe-mode rename` - removes identical files then renames the rest to be different.
    87  
    88  For example to rename all the identically named photos in your Google Photos directory, do
    89  
    90      rclone dedupe --dedupe-mode rename "drive:Google Photos"
    91  
    92  Or
    93  
    94      rclone dedupe rename "drive:Google Photos"
    95  
    96  
    97  ```
    98  rclone dedupe [mode] remote:path [flags]
    99  ```
   100  
   101  ## Options
   102  
   103  ```
   104        --dedupe-mode string   Dedupe mode interactive|skip|first|newest|oldest|largest|smallest|rename. (default "interactive")
   105    -h, --help                 help for dedupe
   106  ```
   107  
   108  See the [global flags page](/flags/) for global options not listed here.
   109  
   110  ## SEE ALSO
   111  
   112  * [rclone](/commands/rclone/)	 - Show help for rclone commands, flags and backends.
   113