PB-051: Fabric REST client resilience

State: Proposed. Not approved for implementation. Priority: P1 (mandatory: required). Owner when approved: Master product (unassigned).

1. PR title and ID#

Retry safe Fabric calls on server errors and refresh an expired token once (PB-051)

2. Business problem#

Fabric's API returns occasional 5xx errors and tokens expire during long sessions. Today either fails the user's action outright, and a long-running operation gives up after 300 seconds.

3. Current verified behavior#

sidecar/fabric_desktop/fabric_api.py FabricClient._request retries up to 5 times on 429 using Retry-After, otherwise raises for any status 400 or above. _wait polls a 202 operation until poll_timeout (300 s by default). Tests in sidecar/tests/test_fabric_api.py cover 429 and LRO (passed in this assessment).

4. Desired behavior#

GET and other idempotent calls retry on 500, 502, 503 and 504 with capped exponential backoff.
POST calls that create items are not retried on 5xx unless the call is safe to repeat (the item is looked up first); this prevents duplicate items.
A 401 asks the token provider for a fresh token once and repeats the call.
The LRO timeout can be raised by callers that know an operation is slow.

5. In scope and out of scope#

In scope: _request, _wait, their tests. Out of scope: OneLake DFS calls (onelake.py), the updater.

6. Architecture and files#

fabric_api.py, sidecar/tests/test_fabric_api.py.

7. Functional requirements#

As section 4. Retries respect Retry-After when present and stop after a bounded total time.

8. Nonfunctional requirements#

Total added wait per call is bounded (for example 60 s). Errors keep their plain-language messages.

9. Dependencies#

None.

10. Security and compatibility#

AGENTS.md: flag unsafe retries. The PR must list which methods retry and why each is safe. Token refresh never logs the token (redaction already covers errors).

11. Acceptance criteria#

Tests: GET retried on 503 then succeeds; create POST not retried on 500; 401 refreshes once and succeeds; second 401 fails with a sign-in message.
Existing tests pass.

12. Unit and integration tests#

Four tests above with a fake session.

13. Manual verification#

Covered by the tenant smoke test (PB-002) on a real tenant.

14. Documentation#

docs/troubleshooting.md: what the app retries.

15. Rollback#

Revert; no data change.

16. Definition of Done#

Merged, tests green, FEAT-FAB-01 limitation updated after review.