Slow upload of multiple small files to ownCloud/nextCloud

Technical Issues WebDAV
1

Hello,
we suffer from very slow upload speed to our local ownCloud storage.
For example, when uploading the folder with 70 of 2 MB files, it takes about 3 MB/s but the upload over Samba or SFTP takes about 60-70 MB/s.

I’m not sure if it is RaiDrive issue, maybe not. The glitch can be directly in ownCloud Server itself but during the problem analysis I noticed the following thing:

ownCloud uses HTTP cookies to store PHP session id which ensures that the same session is used for multiple requests. However, RaiDrive does not use HTTP cookies for WebDAV connections and thus the new session is created on every request. This causes that 4 sessions are created per single file upload and the number of files in /var/lib/php/session is gradually increasing.

I also noticed when I initiate a folder upload to ownCloud storage via RaiDrive, the number of HTTP connection increases with every file. Does RaiDrive create a new HTTP connection for every file? I guess this could cause the additional overhead resulting in slower upload speed when uploading multiple files. Although Apache/ownCloud supports TCP KeepAlive, I noticed that RaiDrive sends “Connection: close” header with every request.

My feature requests:

  • would it be possible to implement HTTP cookies in WebDAV transfers (to use PHP sessions correctly)
  • would it be possible to use single HTTP connection for multiple uploads?

Thank you very much.

EDIT: I tested WinSCP and it has the same slow uploads when uploading a folder with small files. I found this question on StackOverflow: WinSCP creates new session on every WebDAV call - Stack Overflow
It mentions that CarotDAV should support PHP sessions so I tested this client and uploads are superfast with it.

1 Like
2

Hi @Tomas_K,

Thanks a lot for your issue report.
We sent the patch for this issue to your email.
This patch includes the KeepAlive, Cookie issue.

  • KeepAlive: true (keep connection with server)
  • Cookie: true (support cookie for server)
3

Hello,
thank you very much for the response. Your patch provides significant speed improvement on my local testing installation of ownCloud. It also removed the problem with increasing session files. But it still does not reach the full expected speed.

I’m not sure if it helps but there is a WebDAV plugin for Total Commander. It has an option “Use multi-step upload method” which is checked by default. When it is checked the upload speed is same as with your patched version. When unchecked the speed is about twice higher.

4

Hi @Tomas_K,

Thank you so much for your advices and we will take note of the information you provided.
When uploading multiple files, it may be worth using the Quick Copy/Move/Delete for remote storage function.

1 Like
5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.