[Docs] Add Migrate from Sidecar to Ambient documentation

[fjglira]

Description

This is still work in progress. Creating the PR to check the format and start to get some feedback on the documentation
Adding specific documentation to migrate from sidecar to ambient to help users guide on this kind of migrations

Fixes #17000

Reviewers

• Ambient
• Docs
• Installation
• Networking
• Performance and Scalability
• Extensions and Telemetry
• Security
• Test and Release
• User Experience
• Developer Infrastructure
• Localization/Translation

[krinkinmu]

Can you clarify a bit what happens during transitional state when some pods have been migrated, but some have not?

To be specific, let's say we have L7 policies that affect backend selection and/or rewrite the request (i.e., HTTPRoute that matches the request and route to different destination and/or modify request headers). We don't want those L7 policies to be applied multiple times (e.g., by sidecar and by waypoint).

Do I understand correctly, that to avoid this issue, we basically need to make sure that sidecar, after applying L7 policies never sends a request to waypoint to avoid applying any L7 policies the second time (IOW, service selected by the sidecar as a destination for the request is treated as a backend service and no L7 policies will be applied to it again and modify the request again or change the destination of the request)?

And if my understanding is correct, can you clarify who is responsible for ensuring that? Does ISTIO_META_ENABLE_HBONE guarantee that requests will never be routed to waypoints or does the user have to make sure to adjust the policies in the right way?

And finally, how would that interact with the L4 policy suggest elsewhere in this guide that makes sure that ambient-enabled pods don't accept requests from non-waypoints via an L4 authorization policy?

[fjglira]

Hey, sure. Thanks for raising these questions @krinkinmu! I'll update the docs to fill these gaps, as these are definitely points that other users will find during their migration.

• What happens during that transitional state when some pods are in sidecar mode and others are in ambient? We don't want L7 policies applied multiple times.

Once ISTIO_META_ENABLE_HBONE is enabled in the sidecars, the sidecar will be able to communicate with the ambient mesh using HBONE. Because it talks now HBONE, it knows how to tunnel that traffic directly to the destination pod's ztunnel. This effectively bypasses the destination's waypoint for L7 processing, ensuring policies aren't applied twice. The sidecar treats the ambient destination as a standard backend once the L7 logic is resolved.

• How would that interact with the L4 policy suggested elsewhere in this guide that ensures ambient-enabled pods don't accept requests from non-waypoints?

You found an error in my documentation. This is a critical point in a step-by-step migration. If you apply a strict L4 AuthorizationPolicy that requires all traffic to come from a waypoint, it will reject the traffic coming from pods still in Sidecar mode (since they bypass the waypoint).

I’ll update the guide to clarify that during this state, your L4 policies must be inclusive to accept both traffic. I think the best approach is to define policies that allow traffic from either the Waypoint or sidecars. We can transition to waypoint only enforcement once the migration for that specific service path is complete. Also I'll add a note to make user it's clear for users

[krinkinmu]

Thanks for clarifying things. I have a followup question here.

In sidecar mode, roughly, client-side sidecar does endpoint selection and load-balancing and the server-side sidecar does authorization policy enforcement.

With that in mind, does it mean that during the transient state, if client is still using sidecar and the server is already in ambient mode, the L7 authorization policies don't get enforced?

If my understanding is correct, I can see how it might be ok for a short period of time for somebody, but in general it seems somewhat problematic.

[mikemorris]

Agreed @krinkinmu, I would expect the more desirable migration path might be "making the sidecar dumb" i.e. stop applying L7 policies at sidecar egress and start routing traffic from sidecar clients to waypoints which will apply L7 policies instead (this also frontloads potential breaking changes where client-side policies not supported in ambient are dropped before a full cut-over).

[fjglira]

Ok ok, sounds good to me. I'll do today some updates

[fjglira]

I added a whole new section at the beginning of the file called ### Phase 1: Make the client sidecar passive, the idea is to explain to users how to move those L7 policies to waypoints making as you mentione the "sidecar dumb", I rephrased to passive xD

[krinkinmu]

I'm a bit confused, we agreed earlier that sidecar never sends to the waypoint, right?

If you remove the policies from the sidcar to waypoint, doesn't it create the same problem again - during the transient state the L7 policies may not be enforced. Did I misunderstood something?

[krinkinmu]

Actually, forget about that. My understanding is still that authorization policies in sidecar mode are enforced by the server-side sidecars (at least the authorization part of the policies), so if sidecars never send to the waypoint, it does not seem to matter how the policies are changed - we still kind of have the situation when the authoirzation policies are not enforced if the client is sidecar.

[krinkinmu]

FWIW, @fjglira we are quite interested in migration to ambient ourselves, if you think it would be helpful to discuss it over a higher bandwidth channel we can set up a vide call or start a chat in slack and I'd be happy to help.

[fjglira]

We chat more about this on slack. Let's connect and find the best path together :)

[voipp]

may be add some notes on probable replacement rules

[voipp]

what about migrating Kind: Gateway apiVersion: networking.istio.io/v1

[fjglira]

Hi @mikemorris @krinkinmu @keithmattix @sridhargaddam I added an update with changes from the suggestion. I'm still going over the Netlify version to see that everything looks good

[fjglira]

After today's discussion around the best approach for this topic, I'll do some modifications to the current documentation, and tomorrow I'll submit an update. The main idea is to focus first on the L4-only migration use case, and add warning about gaps on migration with L7 policies. At the same time we will work into fill those gaps