Add service wrong pod selection benchmark for Hotel Reservation - #757

The Real-World Failure Story

This problem is inspired by a Kubernetes Service selector misconfiguration where a Service selector is too broad or labels are accidentally shared across unrelated workloads.

In this kind of failure, the Service still has endpoints, but the endpoint list is polluted with pods that do not actually serve the expected traffic. This is harder to diagnose than a simple empty-endpoint failure because the pods may all be Running, the Service may look healthy at first glance, and the endpoint list is not empty.
Reference:
https://medium.com/emburse/intermittent-connection-refused-errors-after-new-service-deployment-in-kubernetes-22dd5feae1ad
However, some requests can still be routed to unintended pods, which may reject connections, time out, or return invalid responses.

How We Simulate the Failure on SREGym

This PR adds a new SREGym benchmark problem:

service_wrong_pod_selection_hotel_reservation

The problem uses the Hotel Reservation application and targets the frontend Service.

Under normal behavior, the frontend Service selects frontend pods using:

io.kompose.service: frontend

The injected fault changes the Service selection behavior by:

1. Adding sregym.io/frontend-route=true to the intended frontend deployment.
2. Adding sregym.io/frontend-route=true to the unintended search deployment.
3. Replacing the frontend Service selector with:

sregym.io/frontend-route: "true"

After the fault is injected, the frontend Service selects both the real frontend pod and the unintended search pod.

frontend Service
├── frontend pod  correct endpoint
└── search pod    wrong endpoint

The frontend Service routes traffic to targetPort 5000, while the search container listens on port 8082. Therefore, traffic routed to the search pod through the frontend Service can fail.

The Problem Runtime Behavior

After fault injection:

• The Hotel Reservation application deploys successfully.
• Application pods can remain Running and Ready.
• The frontend Service still has endpoints.
• The frontend EndpointSlice is non-empty but contains both frontend and search pods.
• Requests through frontend:5000 may fail because some traffic can be routed to the search pod, which does not serve frontend traffic on port 5000.
• The failure is caused by Service endpoint pollution, not an empty endpoint set or pod crash.

The deterministic failure signal is EndpointSlice pollution. Curl failures are useful supporting evidence, but the mitigation oracle does not depend on the exact curl failure rate.

Useful debugging commands:

kubectl get svc frontend -n hotel-reservation -o yaml

kubectl get endpointslices -n hotel-reservation \
  -l kubernetes.io/service-name=frontend \
  -o yaml

kubectl get pods -n hotel-reservation --show-labels

kubectl describe svc frontend -n hotel-reservation

Expected broken-state evidence:

frontend Service selector:
{"sregym.io/frontend-route":"true"}

frontend EndpointSlice contains:
pod=frontend-...
pod=search-...

The Agent Behavior

A successful agent should diagnose that the frontend Service is selecting unintended pods because its selector was broadened to:

sregym.io/frontend-route: "true"

The agent should identify that both the intended frontend pod and the unintended search pod match this selector.

The root cause is the frontend Service selector misconfiguration. It is not a pod crash, not an empty endpoint set, and not a failure inside the search service itself.

Expected diagnosis:

The frontend Service selector is too broad. It selects both frontend pods and the search pod, causing the frontend Service endpoints to be polluted. The search pod does not serve frontend traffic on targetPort 5000, so traffic through the frontend Service can fail.

Expected mitigation is to restore the frontend Service selector to its original value:

io.kompose.service: frontend

The mitigation should result in the frontend EndpointSlice containing only frontend pods.

The mitigation oracle checks that:

• The frontend Service selector is exactly restored to {"io.kompose.service": "frontend"}.
• Frontend endpoints are non-empty.
• All selected endpoint pods are actual frontend pods.
• No unintended search endpoint remains.

In manual validation, the diagnosis submission passed the LLM-as-a-judge evaluation with 100/100, and the mitigation oracle passed after restoring the frontend Service selector and removing the polluted endpoint.

Validation

Local compile check passed:

PYTHONPATH=. uv run python -m py_compile \
  sregym/conductor/problems/service_wrong_pod_selection_hotel_reservation.py \
  sregym/conductor/oracles/wrong_pod_selection_mitigation.py \
  sregym/conductor/problems/registry.py

Registry check passed:

True

End-to-end SREGym validation passed:

• The problem started successfully.
• Fault injection succeeded.
• The frontend Service selector changed to {"sregym.io/frontend-route":"true"}.
• The frontend EndpointSlice contained both frontend and search pods.
• Curl to frontend:5000 showed connection failures as supporting runtime evidence.
• Diagnosis passed with 100/100.
• Mitigation restored the frontend Service selector.
• The frontend EndpointSlice contained only frontend pods.
• The mitigation oracle passed.

Files Changed

Problem List.md
sregym/conductor/problems/registry.py
sregym/conductor/tasklist.yml.example
sregym/conductor/problems/service_wrong_pod_selection_hotel_reservation.py
sregym/conductor/oracles/wrong_pod_selection_mitigation.py