Bug (Langfuse): _sanitize_usage_data unexpectedly flattens OpenAI-style usage format

[LastRemote]

Describe the bug
In langfuse-haystack>=3.0.0 we now use _sanitize_usage_data for better feeding into Langfuse specification. However, this unexpectedly flattens the usage data format in OpenAI style, e.g.:

{
    "completion_tokens": 1080,
    "completion_tokens_details": {
        "accepted_prediction_tokens": 0,
        "audio_tokens": 0,
        "reasoning_tokens": 0,
        "rejected_prediction_tokens": 0
    },
    "prompt_tokens": 4126,
    "prompt_tokens_details": {
        "audio_tokens": 0,
        "cached_tokens": 4096
    },
    "total_tokens": 5206
}

After the flattening, it seems like the data is no longer recognized by Langfuse, and it falls back to the default behavior, showing 0->0 (14508) instead.

Langfuse summarizes all usage types that include the string input as input usage types, similarly output as output usage types. If no total usage type is ingested, Langfuse sums up all usage type units to a total.

Digging deeper inside Langfuse handling, it expects a yaml structure for OpenAI Usage as follows:

OpenAICompletionUsageSchema:
  docs: OpenAI Usage schema from (Chat-)Completion APIs
  properties:
    prompt_tokens: integer
    completion_tokens: integer
    total_tokens: integer
    prompt_tokens_details: optional<map<string, optional<integer>>>
    completion_tokens_details: optional<map<string, optional<integer>>>

To Reproduce
Run an OpenAI model in a pipeline and observe the trace.

Suggestion on Fixing
The easiest solution should be skip sanitizing if we found OpenAI-style usage pattern.

[LastRemote]

This is my temporary workaround which also translates Gemini format:

def normalize_usage_for_langfuse(usage: Optional[Dict[str, Any]]) -> Dict[str, Any] | None:
    """
    Normalize and sanitize usage data for Langfuse.

    This function converts provider-specific usage formats to Langfuse-compatible keys,
    then applies sanitization to flatten nested dicts and keep only numeric values.

    Langfuse natively supports:
    - OpenAI format: prompt_tokens, completion_tokens, total_tokens
    - Anthropic format: input_tokens, output_tokens
    - Generic format: input, output

    Gemini format (promptTokenCount, candidatesTokenCount) needs normalization.

    :param usage: The usage object from various LLM providers. If None, returns None.
    :returns: A Langfuse-compatible usage dict with only numeric values, or None.
    """
    if usage is None:
        return None

    # Gemini format: {"promptTokenCount": int, "candidatesTokenCount": int, "totalTokenCount": int}
    # Langfuse doesn't recognize these keys, so we normalize them
    if "promptTokenCount" in usage or "candidatesTokenCount" in usage:
        normalized: Dict[str, Any] = {}
        if "promptTokenCount" in usage:
            normalized["input_tokens"] = usage["promptTokenCount"]
        if "candidatesTokenCount" in usage:
            normalized["output_tokens"] = usage["candidatesTokenCount"]
        if "totalTokenCount" in usage:
            normalized["total_tokens"] = usage["totalTokenCount"]
        # Copy any other fields that might be present
        for key, value in usage.items():
            if key not in ("promptTokenCount", "candidatesTokenCount", "totalTokenCount"):
                normalized[key] = value
        return _sanitize_usage_data(normalized)

    # Skip sanitizing OpenAI Usage data (https://github.com/deepset-ai/haystack-core-integrations/issues/2587)
    if "prompt_tokens" in usage and "completion_tokens" in usage and "total_tokens" in usage:
        return usage

    # Anthropic and generic formats are natively supported by Langfuse
    return _sanitize_usage_data(usage)