diff --git a/SUMMARY.md b/SUMMARY.md index 58702df3f..4bd03570f 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -104,6 +104,7 @@ * [Rate limiting](usingcurl/transfers/rate-limiting.md) * [Request rate limiting](usingcurl/transfers/request-rate.md) * [Compression](usingcurl/transfers/compression.md) + * [Skip download if already done](usingcurl/transfers/skip.md) * [Connections](usingcurl/connections/README.md) * [VLAN](usingcurl/connections/vlan.md) * [Name resolve tricks](usingcurl/connections/name.md) diff --git a/usingcurl/transfers/README.md b/usingcurl/transfers/README.md index 082d3ed33..6d390819e 100644 --- a/usingcurl/transfers/README.md +++ b/usingcurl/transfers/README.md @@ -8,3 +8,4 @@ multiple transfers. * [Rate limiting](rate-limiting.md) * [Request rate limiting](request-rate.md) * [Compression](compression.md) + * [Skip download if already done](skip.md) diff --git a/usingcurl/transfers/skip.md b/usingcurl/transfers/skip.md new file mode 100644 index 000000000..962931738 --- /dev/null +++ b/usingcurl/transfers/skip.md @@ -0,0 +1,24 @@ +# Skip download if already done + +Sometimes when writing a script or similar to do Internet transfers every now +and then you end up in a situation when you rather have curl skip doing the +transfer if indeed the target file is already present locally. Maybe after the +previous invoke, maybe for another reason. + +This kind of extra logic can certainly be added in common shell script logic +before curl is invoked in many cases but not all. For example things get +complicated when you ask curl to use [globbing download +ranges](../../cmdline/urls/globbing.md). + +For example, download one thousand images from a site but skip the ones that +we already have downloaded previously: + + curl --skip-existing -O https://example.com/image[0-999].jpg + +It should be noted that this only checks the *presence* of the local file, it +makes not attempts to verify that it actually has the correct content and it +has no way to do any such checks. + +curl offers other options to do [conditional +transfers](../../http/modify/conditionals.md) based on modified date or +contents.