Is Dropbox an FTP

Why can Dropbox be super fast compared to FTP?


I would like to know why Dropbox is technically much faster than FTP? What kind of technology does it use?

I'm not talking about diff files, I'm talking about transferring new files either way, dropbox is way faster.

I mean a lot faster, maybe 10 times faster than FTP for files I've uploaded. I'll check back for larger files later.






Reply:


There can be several reasons for this.
The FTP protocol is far of efficient.

  1. An FTP transfer requires a minimum of two connections (one for control and one for data), and DropBox may only use a single HTTP connection. It may also open the data connection for an FTP session from the server to your client. If you are using NAT it might fail so your FTP client might try to connect this way and fail if you try the other way around.

  2. There is a lot of back and forth on an FTP connection. To send a file, the client must send at least two commands (one to open the data connection and one to start sending) and each time wait for the server to respond, which adds latency. In addition to these two round trips per file, there are several round trips for the command response on the first connection - one to send the username, one for the password and at least one to set transfer parameters (to make sure it is the server expected binary, not ASCII data). The client can also issue some additional commands to get information about itself from the server. Dropbox will likely only use this one HTTP request, or at most two (one for authentication, one for sending the data).

  3. Depending on which client you are using for FTP transfers (not indicating that it is a good idea to edit your question to include this information), the connection may be disconnected and reconnected after each send manufactured time. It is not unlikely that DropBox will keep a connection open for a long time for the purpose of long-term polling in order to react as quickly as possible to new data that should be downloaded from this client.An HTTP connection to send a file does not have to be authenticated again .

  4. It is not unlikely that the DropBox client will compress data before sending it (to improve speed and save bandwidth) where your FTP client is not. Even for larger files (unless they are precompressed or encrypted), DropBox and similar utilities can be considerably faster than a simple FTP transfer.

For large files, the first three points are no longer relevant compared to the time it takes to actually transfer the data, but point 4 may still be very important. For small files, the extra setup time added by the FTP protocol can potentially be a few times longer than the time it takes to actually send the data.







As mentioned earlier, Dropbox can skip parts of files that haven't changed. But also, Dropbox will skip upload files if it already has a copy on the server side (one that you or someone else has already uploaded).

So if you try to upload a file that is the same as a file that Dropbox already has, the upload will be skipped (and the other connected computers can download it from the Dropbox servers). If you upload a file that is almost identical to another file that has already been uploaded (it is not clear whether the file that has already been uploaded has to be "your" or could have come from a user), only enough parts of the file will be sent. to recreate it on the server along with the file that has already been uploaded.

FTP cannot perform any of these actions (it is a simple protocol for sending and receiving data streams without reference to any other data available on the remote side). Tools like rsync and Unison can "skip blocks the other side already has" but are typically limited to comparing blocks within files on an identical path in the synchronized hierarchy. Dropbox seems to extend this idea to include collections of files (so if you're "uploading" two nearly identical files, it could probably look like only sending one plus enough of a "diff" to recreate the other).


I assume you think files transfer is faster. When you save a file to your Dropbox folder, Dropbox only sends the delta (or diff) of the data to the remote storage server. FTP will (most likely) send the file byte by byte (rather than just sending the changes), which may take much longer to transfer over a network. Similarly, the local clients load when synchronizing from Remote server only downloads the changes.

The LAN synchronization feature can also potentially speed up synchronization and reduce the amount of network traffic required.



Dropbox may be faster if you're sending a large number of files. FTP is as fast as possible when we're talking at speed, but it requires too much "talking" between the server and client computer for each file, making the FTP seem slower. If you are uploading an open source application with thousands of files, it is more convenient to compress all of the files, upload them via FTP, and decompress them on the server.


I suspect they use simple hashing techniques similar to md5 / sha

Whenever you put a file in the local "Dropbox", the Dropbox client calculates the hash of this file and has to send some additional data such as file size and file name to the Dropbox server.

When the Dropbox server finds similar files (you need to manage the index of the hashes and file data on your server) , it simply informs the client that the file was "uploaded" successfully. ;-)

That way, the file is only "uploaded" logically. Since there is no real transfer of file content, this has to be faster than anything else.

I'm not sure what hashing algorithm the Dropbox uses, but I'm 100% sure that its working principle is similar to the one described above.


Although Dropbox uses other services, they have used Amazon AWS (Amazon Web Services) in the past. It sounds like your transmission from source to destination has a very large transmission line. In my experience, Dropbox uses a destination that can accept large amounts of data at a time. Dropbox also distributes the upload to different IP addresses. The site you're sending via FTP likely has a much smaller transmission line and isn't able to distribute uploads as efficiently.

If you run Resource Monitor (resmon) and go to the Network tab, you will find that the various processes are using the network bandwidth.

  • Under Processes with network activity, select the column for
  • Under TCP Connections, select the column for

When I upload a file to Dropbox it uses 4 connections to send 4 different IP addresses.

We use cookies and other tracking technologies to improve your browsing experience on our website, to show you personalized content and targeted ads, to analyze our website traffic, and to understand where our visitors are coming from.

By continuing, you consent to our use of cookies and other tracking technologies and affirm you're at least 16 years old or have consent from a parent or guardian.

You can read details in our Cookie policy and Privacy policy.