Create ray cluster with ssa

[akram]

Issue link

What changes have been made

Adding apply function.

Verification steps

Checks

• I've made sure the tests are passing.
• Testing Strategy
  • Unit tests
  • Manual tests
  • Testing is not required for this change

[codecov]

Codecov Report

Attention: Patch coverage is 95.12195% with 2 lines in your changes missing coverage. Please review.

Project coverage is 92.40%. Comparing base (b5c13dc) to head (b21b177).

Files with missing lines	Patch %	Lines
src/codeflare_sdk/ray/cluster/cluster.py	95.12%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #778      +/-   ##
==========================================
+ Coverage   90.43%   92.40%   +1.97%     
==========================================
  Files          23       23              
  Lines        1359     1396      +37     
==========================================
+ Hits         1229     1290      +61     
+ Misses        130      106      -24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from bobbins228. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Bobbins228]

Can we add user documentation for this PR?
https://github.com/project-codeflare/codeflare-sdk/blob/main/docs/sphinx/user-docs/ray-cluster-interaction.rst

[Bobbins228]

Began testing this again on both Kind and OpenShift.
It seems that Appwrapper is not working with the new changes when creating a cluster via apply().

TypeError: Discoverer.get() takes 1 positional argument but 2 were given

[akram]

Began testing this again on both Kind and OpenShift. It seems that Appwrapper is not working with the new changes when creating a cluster via apply().

TypeError: Discoverer.get() takes 1 positional argument but 2 were given

I have encapsulated the exception and was not able to reproduce the exact issue here.
Instead I was seeing an issue related to appwrapper not being enabled.

[Bobbins228]

Tested this again and Appwrapper is working for initial apply but I ran into a couple of issues when trying to update the AppWrapper and apply my changes.

I kept running into this error with regards to the field manager even with force=True:

A conflict occurred with the RayCluster resource.
Only one RayCluster with the same name is allowed. Please delete or rename the existing RayCluster before creating a new one with the desired name.
Response: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Apply failed with 1 conflict: conflict with \\"codeflare-operator\\" using workload.codeflare.dev/v1beta2: .spec.components","reason":"Conflict","details":{"causes":[{"reason":"FieldManagerConflict","message":"conflict with \\"codeflare-operator\\" using workload.codeflare.dev/v1beta2","field":".spec.components"}]},"code":409}\n'

[akram]

Tested this again and Appwrapper is working for initial apply but I ran into a couple of issues when trying to update the AppWrapper and apply my changes.

I kept running into this error with regards to the field manager even with force=True:

A conflict occurred with the RayCluster resource.
Only one RayCluster with the same name is allowed. Please delete or rename the existing RayCluster before creating a new one with the desired name.
Response: b'{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Apply failed with 1 conflict: conflict with \\"codeflare-operator\\" using workload.codeflare.dev/v1beta2: .spec.components","reason":"Conflict","details":{"causes":[{"reason":"FieldManagerConflict","message":"conflict with \\"codeflare-operator\\" using workload.codeflare.dev/v1beta2","field":".spec.components"}]},"code":409}\n'

nice catch. I reproduced it. There is 2 issues actually. For appwrappers, I was not propagating the force=True .
But, on top of that, changing the spec.components[0].template is also blocked by webhooks.
I tried to update the mutating webhook the same way I did for raycluster oauth proxy field, but that's not working.
The validating webhook is currently enforcing the raycluster.template to be immutable.

Trying to edit an appwrapper manually with oc edit leads to the same error.

(codeflare-sdk-py3.9) akram@wadez-m1 codeflare-sdk % oc edit appwrappers.workload.codeflare.dev jobtest
error: appwrappers.workload.codeflare.dev "jobtest" could not be patched: admission webhook "vappwrapper.kb.io" denied the request: spec.components[0].template.raw: Forbidden: attempt to change immutable field

[dgrove-oss]

AppWrappers enforce that spec.components[*].template is immutable after creation, because updates may invalidate the PodSet info that they've extracted and given to Kueue.

I'm missing the bigger context of what you are trying to do, but you will not be able to mutate a RayCluster wrapped in an AppWrapper in place.

[akram]

AppWrappers enforce that spec.components[*].template is immutable after creation, because updates may invalidate the PodSet info that they've extracted and given to Kueue.

I'm missing the bigger context of what you are trying to do, but you will not be able to mutate a RayCluster wrapped in an AppWrapper in place.

hi @dgrove-oss ,
the goal of this PR is to introduce an apply() function that would avoid cases where users have to do down+up to update rayclusers and appwrappers. That's an idempotent alternative to the up() function that can also makes resource creation more straitghtforward.

[KPostOffice]

Overall lgtm, just one nit

[akram]

@KPostOffice and @Bobbins228 do you think we can merge it?
right now, error states are already handled in _kube_api_error_handling and explicit error messages are provided.