Turn unwarranted ABORT cases into FAIL for scs-0117, scs-0123 #1063
Conversation
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
| It will abort with an exception if no service of type object-storage is present. As of now, we deem | ||
| this behavior adequate. | ||
| """ | ||
| if 'object-store' not in services_lookup: |
There was a problem hiding this comment.
Do I misunderstand something here or should this be s3 here instead? s3 is listed in the Mandatory IaaS APIs, whereas object-store is listed optional.
There was a problem hiding this comment.
I know what you mean, but this test requires the catalog entry to derive the S3 host. We can improve this test if and when a test subject arrives that satisfies our standard without offering object-store.
There was a problem hiding this comment.
One alternative would be to have the test succeed, and I'm open to that.
There was a problem hiding this comment.
As this seems to be related to the issue #1003, let's discuss this in the next SIG Std/Cert.
There was a problem hiding this comment.
The thing is that we also have the testcase scs-0123-service-object-store listed as mandatory, which would then be wrong as well, and I have a hunch that this was a conscious decision to make testing possible. We would have to check the supplement.
There was a problem hiding this comment.
The supplement doesn't help here. The origin can be found here:
https://github.com/SovereignCloudStack/standards/blame/b61054b7ca8a7de6bd627351964bf63b28642feb/Tests/iaas/mandatory-services/mandatory-iaas-services.py#L27
There was a problem hiding this comment.
Apart from that, the script used to have this capability:
If the deployment uses s3 only and does not have the object store endpoint specified in the service catalog, the "
--s3-endpoint" flag may be used to specify the s3 endpoint. In that case the "--s3-access" and "--s3-access-secret" flags must also be set, to give all necessery credentials to the test suite
This capability has since been removed (I think) because it's hard to automate. However, it can be added in again should the need arise.
That doesn't affect the matter regarding object-store.
There was a problem hiding this comment.
Sorry, I have to retract the last sentence:
if s3_credentials:
mandatory_services.remove("object-store")this was in the original code and has since been removed 👀
|
Case "swift/s3" is also related to #1003. |
|
@depressiveRobot actually, I think this is more a case of this: #1004 |
The cases actually warrant FAIL:
Reporting those as ABORT doesn't do the severity of the failures justice.