1
0
Fork 0
mirror of https://github.com/chrislusf/seaweedfs synced 2025-07-26 21:42:48 +02:00
seaweedfs/weed/s3api
Chris Lu 12f50d37fa
test versioning also (#7000)
* test versioning also

* fix some versioning tests

* fall back

* fixes

Never-versioned buckets: No VersionId headers, no Status field
Pre-versioning objects: Regular files, VersionId="null", included in all operations
Post-versioning objects: Stored in .versions directories with real version IDs
Suspended versioning: Proper status handling and null version IDs

* fixes

Bucket Versioning Status Compliance
Fixed: New buckets now return no Status field (AWS S3 compliant)
Before: Always returned "Suspended" 
After: Returns empty VersioningConfiguration for unconfigured buckets 
2. Multi-Object Delete Versioning Support
Fixed: DeleteMultipleObjectsHandler now fully versioning-aware
Before: Always deleted physical files, breaking versioning 
After: Creates delete markers or deletes specific versions properly 
Added: DeleteMarker field in response structure for AWS compatibility
3. Copy Operations Versioning Support
Fixed: CopyObjectHandler and CopyObjectPartHandler now versioning-aware
Before: Only copied regular files, couldn't handle versioned sources 
After: Parses version IDs from copy source, creates versions in destination 
Added: pathToBucketObjectAndVersion() function for version ID parsing
4. Pre-versioning Object Handling
Fixed: getLatestObjectVersion() now has proper fallback logic
Before: Failed when .versions directory didn't exist 
After: Falls back to regular objects for pre-versioning scenarios 
5. Enhanced Object Version Listings
Fixed: listObjectVersions() includes both versioned AND pre-versioning objects
Before: Only showed .versions directories, ignored pre-versioning objects 
After: Shows complete version history with VersionId="null" for pre-versioning 
6. Null Version ID Handling
Fixed: getSpecificObjectVersion() properly handles versionId="null"
Before: Couldn't retrieve pre-versioning objects by version ID 
After: Returns regular object files for "null" version requests 
7. Version ID Response Headers
Fixed: PUT operations only return x-amz-version-id when appropriate
Before: Returned version IDs for non-versioned buckets 
After: Only returns version IDs for explicitly configured versioning 

* more fixes

* fix copying with versioning, multipart upload

* more fixes

* reduce volume size for easier dev test

* fix

* fix version id

* fix versioning

* Update filer_multipart.go

* fix multipart versioned upload

* more fixes

* more fixes

* fix versioning on suspended

* fixes

* fixing test_versioning_obj_suspended_copy

* Update s3api_object_versioning.go

* fix versions

* skipping test_versioning_obj_suspend_versions

* > If the versioning state has never been set on a bucket, it has no versioning state; a GetBucketVersioning request does not return a versioning state value.

* fix tests, avoid duplicated bucket creation, skip tests

* only run s3tests_boto3/functional/test_s3.py

* fix checking filer_pb.ErrNotFound

* Update weed/s3api/s3api_object_versioning.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_object_handlers_copy.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update weed/s3api/s3api_bucket_config.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update test/s3/versioning/s3_versioning_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-19 21:43:34 -07:00
..
cors convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
policy fix 2024-10-03 09:03:17 -07:00
policy_engine convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
s3_constants Fix get object lock configuration handler (#6996) 2025-07-18 02:19:50 -07:00
s3bucket adjust fix 2025-02-01 14:11:57 -08:00
s3err Test object lock and retention (#6997) 2025-07-18 22:25:58 -07:00
AmazonS3.xsd add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
auth_credentials.go convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
auth_credentials_subscribe.go Fix get object lock configuration handler (#6996) 2025-07-18 02:19:50 -07:00
auth_credentials_test.go improve iam error handling (#6446) 2025-01-16 08:23:19 -08:00
auth_signature_v2.go rewrite, simplify, avoid unused functions (#6989) 2025-07-15 10:11:49 -07:00
auth_signature_v4.go rewrite, simplify, avoid unused functions (#6989) 2025-07-15 10:11:49 -07:00
auto_signature_v4_test.go rewrite, simplify, avoid unused functions (#6989) 2025-07-15 10:11:49 -07:00
bucket_metadata.go added context to filer_client method calls (#6808) 2025-05-22 09:46:49 -07:00
bucket_metadata_test.go [s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 2023-09-25 08:34:12 -07:00
chunked_reader_v4.go rewrite, simplify, avoid unused functions (#6989) 2025-07-15 10:11:49 -07:00
chunked_reader_v4_test.go implement s3 streaming-unsigned-payload-trailer (#6539) 2025-02-12 12:29:13 -08:00
custom_types.go weed s3: generate time format acceptable by aws cli s3 2019-04-01 16:13:29 -07:00
filer_multipart.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
filer_multipart_test.go [s3] fix s3 test_multipart_resend_first_finishes_last (#5471) 2024-04-06 10:56:39 -07:00
filer_util.go S3: add object versioning (#6945) 2025-07-09 01:51:45 -07:00
filer_util_tags.go added context to filer_client method calls (#6808) 2025-05-22 09:46:49 -07:00
object_lock_utils.go refactor (#6999) 2025-07-19 00:49:56 -07:00
README.txt add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
s3api_acl_helper.go "golang.org/x/exp/slices" => "slices" and go fmt 2024-12-19 19:25:06 -08:00
s3api_acl_helper_test.go [s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 2023-09-25 08:34:12 -07:00
s3api_acp.go [s3acl] Step1: move s3account.AccountManager into to iam.S3ApiConfiguration (#4859) 2023-09-25 08:34:12 -07:00
s3api_auth.go add s3 signature tests and prepare implementation of STREAMING-UNSIGNED-PAYLOAD-TRAILER (#6525) 2025-02-07 10:54:31 -08:00
s3api_bucket_config.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_bucket_cors_handlers.go adding cors support (#6987) 2025-07-15 00:23:54 -07:00
s3api_bucket_handlers.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_bucket_handlers_object_lock_config.go refactor (#6999) 2025-07-19 00:49:56 -07:00
s3api_bucket_handlers_test.go add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
s3api_bucket_skip_handlers.go adding cors support (#6987) 2025-07-15 00:23:54 -07:00
s3api_circuit_breaker.go convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
s3api_circuit_breaker_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
s3api_governance_permissions_test.go Add policy engine (#6970) 2025-07-13 16:21:36 -07:00
s3api_handlers.go Set allowed origins in config (#5109) 2023-12-20 16:21:11 -08:00
s3api_object_handlers.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_object_handlers_copy.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_object_handlers_copy_test.go rename files 2024-04-29 05:33:56 -07:00
s3api_object_handlers_delete.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_object_handlers_legal_hold.go refactor (#6999) 2025-07-19 00:49:56 -07:00
s3api_object_handlers_list.go convert error fromating to %w everywhere (#6995) 2025-07-16 23:39:27 -07:00
s3api_object_handlers_list_test.go add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
s3api_object_handlers_multipart.go Test object lock and retention (#6997) 2025-07-18 22:25:58 -07:00
s3api_object_handlers_postpolicy.go fix(s3api): make post api upload object default return 204 (#5648) 2024-06-04 06:18:47 -07:00
s3api_object_handlers_put.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_object_handlers_retention.go refactor (#6999) 2025-07-19 00:49:56 -07:00
s3api_object_handlers_skip.go implement PubObjectRetention and WORM (#6969) 2025-07-12 21:58:55 -07:00
s3api_object_handlers_tagging.go rename files 2024-04-29 05:33:56 -07:00
s3api_object_handlers_test.go add back "/" prefix if it is missing in object 2022-10-29 17:54:30 -07:00
s3api_object_lock_fix_test.go Fix get object lock configuration handler (#6996) 2025-07-18 02:19:50 -07:00
s3api_object_lock_headers_test.go Test object lock and retention (#6997) 2025-07-18 22:25:58 -07:00
s3api_object_retention.go refactor (#6999) 2025-07-19 00:49:56 -07:00
s3api_object_retention_test.go refactor (#6999) 2025-07-19 00:49:56 -07:00
s3api_object_versioning.go test versioning also (#7000) 2025-07-19 21:43:34 -07:00
s3api_policy.go [s3] Put bucket lifecycle configuration (#5510) 2024-04-27 07:39:22 -07:00
s3api_put_object_helper.go Fix chunked data reading if iam not enabled (#6898) 2025-06-19 22:58:10 -07:00
s3api_put_object_helper_test.go Add credential storage (#6938) 2025-07-02 18:03:17 -07:00
s3api_server.go Fix get object lock configuration handler (#6996) 2025-07-18 02:19:50 -07:00
s3api_server_grpc.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
s3api_status_handlers.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
s3api_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00
s3api_xsd_generated.go add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
s3api_xsd_generated_helper.go add s3test for sql (#5718) 2024-07-04 11:00:41 -07:00
stats.go change version directory 2025-06-03 22:46:10 -07:00
tags.go sort tags by key (#4047) 2022-12-11 22:49:57 -08:00
tags_test.go move to https://github.com/seaweedfs/seaweedfs 2022-07-29 00:17:28 -07:00

see https://blog.aqwari.net/xml-schema-go/

1. go get aqwari.net/xml/cmd/xsdgen
2. Add EncodingType element for ListBucketResult in AmazonS3.xsd
3. xsdgen -o s3api_xsd_generated.go -pkg s3api AmazonS3.xsd
4. Remove empty Grantee struct in s3api_xsd_generated.go
5. Remove xmlns: sed s'/http:\/\/s3.amazonaws.com\/doc\/2006-03-01\/\ //' s3api_xsd_generated.go