S3: Only use multipart for big files

[maprambo]

Steps to reproduce

1. Set up Minio with Backblaze B2 as backend
2. Set up Nextcloud to use Minio's S3 storage

Expected behaviour

Nextcloud talks with Minio using S3 protocols and Minio talks with Backblaze B2 using the B2 protocol.

Actual behaviour

Nextcloud always uses a multipart upload, even if only one part is uploaded. Nextcloud didn't behave like this until commit 1d322d3. Because Minio is translating this to a multipart upload to Backblaze, which expects at least 2 parts in a multipart upload, the upload results in an error.

I think Nextcloud should allow for third party S3 providers, and Minio is a big open source project in this field. If it expects multipart uploads for some backends to actually be multi-part, Nextcloud should comply, as long as it doesn't break anything else.

1d322d3 was however required to fix #13845 (see #13882). Is there no other way to prevent that bug?

Server configuration

Operating system: Nextcloud docker image with apache tag

Web server: apache

Database: mariadb

PHP version: 7.3.14

Nextcloud version: 18.0.0.10

Updated from an older Nextcloud/ownCloud or fresh install: fresh install

Where did you install Nextcloud from: docker

Signing status:

Signing status

Login as admin user into your Nextcloud and access 
http://example.com/index.php/settings/integrity/failed 
paste the results here.

Not possible because of an internal server error when trying to access Nextcloud.

List of activated apps:

App list

Enabled:

• accessibility: 1.4.0
• activity: 2.11.0
• cloud_federation_api: 1.1.0
• comments: 1.8.0
• dav: 1.14.0
• federatedfilesharing: 1.8.0
• federation: 1.8.0
• files: 1.13.1
• files_pdfviewer: 1.7.0
• files_rightclick: 0.15.2
• files_sharing: 1.10.1
• files_trashbin: 1.8.0
• files_versions: 1.11.0
• files_videoplayer: 1.7.0
• firstrunwizard: 2.7.0
• logreader: 2.3.0
• lookup_server_connector: 1.6.0
• nextcloud_announcements: 1.7.0
• notifications: 2.6.0
• oauth2: 1.6.0
• password_policy: 1.8.0
• photos: 1.0.0
• privacy: 1.2.0
• provisioning_api: 1.8.0
• recommendations: 0.6.0
• serverinfo: 1.8.0
• settings: 1.0.0
• sharebymail: 1.8.0
• support: 1.1.0
• survey_client: 1.6.0
• systemtags: 1.8.0
• text: 2.0.0
• theming: 1.9.0
• twofactor_backupcodes: 1.7.0
• updatenotification: 1.8.0
• viewer: 1.2.0
• workflowengine: 2.0.0
  Disabled:
• admin_audit
• encryption
• files_external
• user_ldap

Nextcloud configuration:

Config report

{
    "system": {
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "objectstore": {
            "class": "OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "nextcloud-nbg01",
                "autocreate": false,
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "hostname": "nextcloud-s3-b2.minio.int",
                "port": 9000,
                "use_ssl": true,
                "use_path_style": true
            }
        },
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud-test.int"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "18.0.0.10",
        "overwrite.cli.url": "http:\/\/cloud-test.int",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "3306",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true
    }
}

Are you using external storage, if yes which one: S3 as primary storage, that's it

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Firefox 72.0.2

Operating system: macOS Catalina 10.15.2

Logs

Web server error log

Web server error log

10.3.2.4 - - [01/Feb/2020:17:49:33 +0000] "GET /login HTTP/1.1" 500 1178 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:72.0) Gecko/20100101 Firefox/72.0"

Nextcloud log (data/nextcloud.log)

Nextcloud log

{"reqId":"Rx3RSljID8qyN1HrZZsD","level":3,"time":"2020-02-01T17:49:46+00:00","remoteAddr":"10.3.2.4","user":"--","app":"objectstore","method":"GET","url":"/login","message":{"Exception":"Aws\\S3\\Exception\\S3MultipartUploadException","Message":"An exception occurred while completing a multipart upload: Error executing \"CompleteMultipartUpload\" on \"https://nextcloud-s3-b2.minio.int:9000/nextcloud-nbg01/urn%3Aoid%3A93?uploadId=4_z5e1cd8585786e78a6bfc0311_f202ca62147c25f91_d20200201_m174939_c003_v0312001_t0020\"; AWS HTTP error: Client error: `POST https://nextcloud-s3-b2.minio.int:9000/nextcloud-nbg01/urn%3Aoid%3A93?uploadId=4_z5e1cd8585786e78a6bfc0311_f202ca62147c25f91_d20200201_m174939_c003_v0312001_t0020` resulted in a `404 Not Found` response:\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Mes (truncated...)\n NoSuchBucket (client): The specified bucket does not exist - <?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><Key>urn:oid:93</Key><BucketName>nextcloud-nbg01</BucketName><Resource>/nextcloud-nbg01/urn:oid:93</Resource><RequestId>15EF58934252833A</RequestId><HostId>a607f297-65ef-4422-a14c-0737c93ad807</HostId></Error>","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/aws/aws-sdk-php/src/Multipart/AbstractUploadManager.php","line":156,"function":"transformException","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":["*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":203,"function":"Aws\\Multipart\\{closure}","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":156,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":[2,"*** sensitive parameter replaced ***",[{"__class__":"GuzzleHttp\\Promise\\Promise"},null,{"__class__":"Closure"}]]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":98,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":125,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":["*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Coroutine.php","line":65,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Coroutine","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/aws/aws-sdk-php/src/Multipart/AbstractUploadManager.php","line":83,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php","line":95,"function":"upload","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":468,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:93",null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":405,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null,1]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":313,"function":"writeBack","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["/tmp/oc_tmp_zQGOPd-.js","appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"function":"OC\\Files\\ObjectStore\\{closure}","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/icewind/streams/src/CallbackWrapper.php","line":121,"function":"call_user_func","args":[{"__class__":"Closure"}]},{"function":"stream_close","class":"Icewind\\Streams\\CallbackWrapper","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":426,"function":"fclose","args":[null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":381,"function":"file_put_contents","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"," "]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",1580579378]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Availability.php","line":364,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/View.php","line":1162,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/View.php","line":569,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["touch","/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",["touch","create","write"],null]},{"file":"/var/www/html/lib/private/Files/Node/Folder.php","line":184,"function":"touch","class":"OC\\Files\\View","type":"->","args":["/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/SimpleFS/SimpleFolder.php","line":84,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":179,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":105,"function":"cache","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html/core/js","merged-template-prepend.js",{"__class__":"OC\\Files\\SimpleFS\\SimpleFolder"}]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":115,"function":"process","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json","core"]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":71,"function":"cacheAndAppendCombineJsonIfExist","class":"OC\\Template\\JSResourceLocator","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json"]},{"file":"/var/www/html/lib/private/Template/ResourceLocator.php","line":78,"function":"doFind","class":"OC\\Template\\JSResourceLocator","type":"->","args":["js/merged-template-prepend"]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":346,"function":"find","class":"OC\\Template\\ResourceLocator","type":"->","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":174,"function":"findJavascriptFiles","class":"OC\\TemplateLayout","type":"::","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/legacy/template.php","line":184,"function":"__construct","class":"OC\\TemplateLayout","type":"->","args":["error",""]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/template.php","line":333,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":65,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[{"__class__":"OCP\\Files\\NotPermittedException"},500]}],"File":"/var/www/html/3rdparty/aws/aws-sdk-php/src/Multipart/AbstractUploadManager.php","Line":147,"Previous":{"Exception":"Aws\\S3\\Exception\\S3Exception","Message":"Error executing \"CompleteMultipartUpload\" on \"https://nextcloud-s3-b2.minio.int:9000/nextcloud-nbg01/urn%3Aoid%3A93?uploadId=4_z5e1cd8585786e78a6bfc0311_f202ca62147c25f91_d20200201_m174939_c003_v0312001_t0020\"; AWS HTTP error: Client error: `POST https://nnextcloud-s3-b2.minio.int:9000/nextcloud-nbg01/urn%3Aoid%3A93?uploadId=4_z5e1cd8585786e78a6bfc0311_f202ca62147c25f91_d20200201_m174939_c003_v0312001_t0020` resulted in a `404 Not Found` response:\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Mes (truncated...)\n NoSuchBucket (client): The specified bucket does not exist - <?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Message><Key>urn:oid:93</Key><BucketName>nextcloud-nbg01</BucketName><Resource>/nextcloud-nbg01/urn:oid:93</Resource><RequestId>15EF58934252833A</RequestId><HostId>a607f297-65ef-4422-a14c-0737c93ad807</HostId></Error>","Code":0,"Trace":[{"file":"/var/www/html/3rdparty/aws/aws-sdk-php/src/WrappedHttpHandler.php","line":97,"function":"parseError","class":"Aws\\WrappedHttpHandler","type":"->","args":["*** sensitive parameter replaced ***",{"__class__":"GuzzleHttp\\Psr7\\Request"},{"__class__":"Aws\\Command"},[]]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":203,"function":"Aws\\{closure}","class":"Aws\\WrappedHttpHandler","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":174,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":[2,"*** sensitive parameter replaced ***",[{"__class__":"GuzzleHttp\\Promise\\Promise"},{"__class__":"Closure"},{"__class__":"Closure"}]]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/RejectedPromise.php","line":40,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\RejectedPromise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":98,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":125,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":["*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Coroutine.php","line":65,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Coroutine","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/aws/aws-sdk-php/src/Multipart/AbstractUploadManager.php","line":83,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php","line":95,"function":"upload","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":468,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:93",null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":405,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null,1]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":313,"function":"writeBack","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["/tmp/oc_tmp_zQGOPd-.js","appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"function":"OC\\Files\\ObjectStore\\{closure}","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/icewind/streams/src/CallbackWrapper.php","line":121,"function":"call_user_func","args":[{"__class__":"Closure"}]},{"function":"stream_close","class":"Icewind\\Streams\\CallbackWrapper","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":426,"function":"fclose","args":[null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":381,"function":"file_put_contents","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"," "]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",1580579378]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Availability.php","line":364,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/View.php","line":1162,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/View.php","line":569,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["touch","/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",["touch","create","write"],null]},{"file":"/var/www/html/lib/private/Files/Node/Folder.php","line":184,"function":"touch","class":"OC\\Files\\View","type":"->","args":["/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/SimpleFS/SimpleFolder.php","line":84,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":179,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":105,"function":"cache","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html/core/js","merged-template-prepend.js",{"__class__":"OC\\Files\\SimpleFS\\SimpleFolder"}]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":115,"function":"process","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json","core"]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":71,"function":"cacheAndAppendCombineJsonIfExist","class":"OC\\Template\\JSResourceLocator","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json"]},{"file":"/var/www/html/lib/private/Template/ResourceLocator.php","line":78,"function":"doFind","class":"OC\\Template\\JSResourceLocator","type":"->","args":["js/merged-template-prepend"]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":346,"function":"find","class":"OC\\Template\\ResourceLocator","type":"->","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":174,"function":"findJavascriptFiles","class":"OC\\TemplateLayout","type":"::","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/legacy/template.php","line":184,"function":"__construct","class":"OC\\TemplateLayout","type":"->","args":["error",""]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/template.php","line":333,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":65,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[{"__class__":"OCP\\Files\\NotPermittedException"},500]}],"File":"/var/www/html/3rdparty/aws/aws-sdk-php/src/WrappedHttpHandler.php","Line":191,"Previous":{"Exception":"GuzzleHttp\\Exception\\ClientException","Message":"Client error: `POST https://nextcloud-s3-b2.minio.int:9000/nextcloud-nbg01/urn%3Aoid%3A93?uploadId=4_z5e1cd8585786e78a6bfc0311_f202ca62147c25f91_d20200201_m174939_c003_v0312001_t0020` resulted in a `404 Not Found` response:\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>NoSuchBucket</Code><Message>The specified bucket does not exist</Mes (truncated...)\n","Code":404,"Trace":[{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":66,"function":"create","class":"GuzzleHttp\\Exception\\RequestException","type":"::","args":[{"__class__":"GuzzleHttp\\Psr7\\Request"},"*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":203,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":156,"function":"callHandler","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":[1,"*** sensitive parameter replaced ***",[{"__class__":"GuzzleHttp\\Promise\\Promise"},{"__class__":"Closure"},null]]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":47,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Promise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":98,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Handler/CurlMultiHandler.php","line":125,"function":"tick","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"execute","class":"GuzzleHttp\\Handler\\CurlMultiHandler","type":"->","args":["*** sensitive parameter replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Coroutine.php","line":65,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":246,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\Coroutine","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":223,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":267,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":225,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/3rdparty/aws/aws-sdk-php/src/Multipart/AbstractUploadManager.php","line":83,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/S3ObjectTrait.php","line":95,"function":"upload","class":"Aws\\Multipart\\AbstractUploadManager","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":468,"function":"writeObject","class":"OC\\Files\\ObjectStore\\S3","type":"->","args":["urn:oid:93",null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":405,"function":"writeStream","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null,1]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":313,"function":"writeBack","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["/tmp/oc_tmp_zQGOPd-.js","appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"function":"OC\\Files\\ObjectStore\\{closure}","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/html/3rdparty/icewind/streams/src/CallbackWrapper.php","line":121,"function":"call_user_func","args":[{"__class__":"Closure"}]},{"function":"stream_close","class":"Icewind\\Streams\\CallbackWrapper","type":"->","args":[]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":426,"function":"fclose","args":[null]},{"file":"/var/www/html/lib/private/Files/ObjectStore/ObjectStoreStorage.php","line":381,"function":"file_put_contents","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"," "]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\ObjectStore\\ObjectStoreStorage","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",1580579378]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Availability.php","line":364,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php","line":354,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Availability","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",null]},{"file":"/var/www/html/lib/private/Files/View.php","line":1162,"function":"touch","class":"OC\\Files\\Storage\\Wrapper\\Wrapper","type":"->","args":["appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/View.php","line":569,"function":"basicOperation","class":"OC\\Files\\View","type":"->","args":["touch","/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js",["touch","create","write"],null]},{"file":"/var/www/html/lib/private/Files/Node/Folder.php","line":184,"function":"touch","class":"OC\\Files\\View","type":"->","args":["/appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Files/SimpleFS/SimpleFolder.php","line":84,"function":"newFile","class":"OC\\Files\\Node\\Folder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":179,"function":"newFile","class":"OC\\Files\\SimpleFS\\SimpleFolder","type":"->","args":["merged-template-prepend.js"]},{"file":"/var/www/html/lib/private/Template/JSCombiner.php","line":105,"function":"cache","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html/core/js","merged-template-prepend.js",{"__class__":"OC\\Files\\SimpleFS\\SimpleFolder"}]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":115,"function":"process","class":"OC\\Template\\JSCombiner","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json","core"]},{"file":"/var/www/html/lib/private/Template/JSResourceLocator.php","line":71,"function":"cacheAndAppendCombineJsonIfExist","class":"OC\\Template\\JSResourceLocator","type":"->","args":["/var/www/html","core/js/merged-template-prepend.json"]},{"file":"/var/www/html/lib/private/Template/ResourceLocator.php","line":78,"function":"doFind","class":"OC\\Template\\JSResourceLocator","type":"->","args":["js/merged-template-prepend"]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":346,"function":"find","class":"OC\\Template\\ResourceLocator","type":"->","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/TemplateLayout.php","line":174,"function":"findJavascriptFiles","class":"OC\\TemplateLayout","type":"::","args":[["core/js/dist/main","js/merged-template-prepend","search/js/search","core/l10n/de","js/backgroundjobs","files_sharing/l10n/de","files_sharing/js/dist/main","files_pdfviewer/l10n/en","files_pdfviewer/js/previewplugin","files_videoplayer/l10n/en","files_videoplayer/js/main","search/l10n/de","search/js/searchprovider","js/files/fileinfo","js/files/client","core/js/dist/login"]]},{"file":"/var/www/html/lib/private/legacy/template.php","line":184,"function":"__construct","class":"OC\\TemplateLayout","type":"->","args":["error",""]},{"file":"/var/www/html/lib/private/Template/Base.php","line":132,"function":"fetchPage","class":"OC_Template","type":"->","args":[]},{"file":"/var/www/html/lib/private/legacy/template.php","line":333,"function":"printPage","class":"OC\\Template\\Base","type":"->","args":[]},{"file":"/var/www/html/index.php","line":65,"function":"printExceptionErrorPage","class":"OC_Template","type":"::","args":[{"__class__":"OCP\\Files\\NotPermittedException"},500]}],"File":"/var/www/html/3rdparty/guzzlehttp/guzzle/src/Exception/RequestException.php","Line":113}},"CustomMessage":"Could not create object for appdata_ocasm0c5u29c/js/core/merged-template-prepend.js"},"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:72.0) Gecko/20100101 Firefox/72.0","version":"18.0.0.10"}

Browser log

Browser log

Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

No browser log because of server error

Minio log

Minio log

API: CompleteMultipartUpload(bucket=nextcloud-nbg01, object=urn:oid:93/.minio/data)


Time: 17:49:46 UTC 02/01/2020


DeploymentID: b38bcb6b-5e36-4eb4-973b-dc8362b97d3a


RequestID: 15EF5893D897E39E


RemoteHost: 10.3.2.11


Host: nextcloud-s3-b2.minio.int:9000


UserAgent: MinIO (linux; amd64) minio-go/v6.0.45


Error: b2_finish_large_file: 400: large files must have at least 2 parts


       3: cmd/gateway/b2/gateway-b2.go:917:b2.(*b2Objects).CompleteMultipartUpload()


       2: cmd/object-handlers.go:2413:cmd.objectAPIHandlers.CompleteMultipartUploadHandler()


       1: net/http/server.go:2007:http.HandlerFunc.ServeHTTP()

[kesselb]

Thanks for reporting 👍

Closing as duplicate of #288. I'm still waiting for some feedback for the patch.

[k0ste]

@kesselb what exactly patch are you talking about?
We find out that for every object, even if 1KiB in size NextCloud initiate multipart upload
Currently, for 850K obj bucket we have a more than 7k multiparts

$ aws s3api list-multipart-uploads --bucket nextcloud | grep UploadId | wc -l
7250