Configure egress access during golden initialization

[Eitan Yarmush (EItanya)]

Adds ActorTemplate.goldenEgressPolicy to configure network access while preparing the golden snapshot. The golden actor and its policy are created in one transaction, so the policy exists before initialization starts.

The policy object contains only rules today and leaves room for additional fields. Rules use the existing egress validation and defaults. Omitting the field creates no policy. When set, it must contain at least one rule. Ordinary actors, including those restored from the golden snapshot, start without an egress policy. Callers configure their runtime access separately with CreateActorEgressPolicy.

Related to #1324. Motivation and API rationale: #2358.

• Tests pass — the full control API, functional, storage, validation, and defaulting suites pass with -race. All repository verifier scripts pass. Validated after rebasing onto upstream 59c5b2a54. Full make verify fails in the known CA-file reload tests in ateapi, atelet, and CSI, previously reproduced on unchanged upstream.
• Appropriate changes to documentation are included in the PR

[Eitan Yarmush (EItanya)]

I opted to create another message here for simplicity, but we could opt to instead require the whole EgressPolicy object with metadata and require the name to be default etc.

Julian Gutierrez Oschmann (@juli4n) and Tim Hockin (@thockin) for API context

[Tim Hockin (thockin)]

Clarify for me - does this mean that every actor created from such a template has the same policy unless subsequently changed? Maybe that should be opt-in? E.g. create actor --template=<T> --with-template-policy ?

Does this also mean that a subsequent call to set policy will require a read-modify-write? That may be surprising.

[Eitan Yarmush (EItanya)]

Clarify for me - does this mean that every actor created from such a template has the same policy unless subsequently changed? Maybe that should be opt-in? E.g. create actor --template=<T> --with-template-policy ?

Yes it would. However, I think that if you are creating from an ActorTemplate, the default policy is actually something you always want, otherwise your actor probably won't work correct. Minus the policy would probably warrant a separate ActorTemplate. The other option could be to accept a policy when calling CreateActor. Part of the problem is that this field is actually doing 2 important, but separate things.

1. Setting Policy for the golden snapshot, allowing the actor template to use the network to prepare the golden.
2. Also adding that same policy as the default policy for the actor itself once created.

Now that I've spelled that out for you, I actually think these should be separate. The field I created here should really be something like golden_egress_policy, and then also accept a policy when calling CreateActor. (Doesn't have to be exactly that, but something in that spirit.)

Does that make sense?

Does this also mean that a subsequent call to set policy will require a read-modify-write? That may be surprising.

Yes, and this is a very good point. I think that can mostly be solved with documentation, but it's definitely an interaction worth calling out. This is also a result of not having multiple policy layers, and therefore needing one policy source of truth for an actor.

[Lior Lieberman (LiorLieberman)]

I am worried on just "goldenEgressPolicy". There are going to be many usecases that you need to just fetch github repos/s3/gcs for some assets. Not necessrially part of "golden" - how do we get those to work.

maybe its an "init actor" support and a policy for that? A very common use case for that is "I want an actor with no network, but give it all this github repos available in local folders (e.g for RL)

FWIW init actor can be useful for many other things.

[Eitan Yarmush (EItanya)]

I am worried on just "goldenEgressPolicy". There are going to be many usecases that you need to just fetch github repos/s3/gcs for some assets. Not necessrially part of "golden" - how do we get those to work.

If it's not part of the golden then it's already supported. When an actor comes up from golden it will already have its own identity. I specifically narrowed the scope of this PR to just handle the golden identity.

Adding policy for pre readyz should already work.

maybe its an "init actor" support and a policy for that? A very common use case for that is "I want an actor with no network, but give it all this github repos available in local folders (e.g for RL)

That would be exactly what this would enable. The actor identity would not be allowed to make network calls, only during golden preparation. Mounting other data can/should be done with volumes otherwise.

FWIW init actor can be useful for many other things.