Skip to content

vllm.config.aux_output

Configuration for execution auxiliary outputs.

Classes:

AuxOutputConfig

Configuration for auxiliary-output delivery.

Methods:

  • compute_hash –

    Hash AuxOutput settings that alter the model forward graph.

Attributes:

Source code in vllm/config/aux_output.py 
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
@config
class AuxOutputConfig:
    """Configuration for auxiliary-output delivery."""

    enable_return_routed_experts: bool = False
    """Capture and return routed-experts auxiliary outputs."""

    max_bytes: int | None = Field(default=None, gt=0)
    """LRU capacity, or ``None`` to derive it from the KV cache capacity."""

    @property
    def enabled(self) -> bool:
        """Whether any execution auxiliary output is enabled."""
        return self.enable_return_routed_experts

    def compute_hash(self) -> str:
        """Hash AuxOutput settings that alter the model forward graph."""
        from vllm.config.utils import hash_factors

        return hash_factors(
            {"enable_return_routed_experts": self.enable_return_routed_experts}
        )

enable_return_routed_experts = False class-attribute instance-attribute

Capture and return routed-experts auxiliary outputs.

enabled property

Whether any execution auxiliary output is enabled.

max_bytes = Field(default=None, gt=0) class-attribute instance-attribute

LRU capacity, or None to derive it from the KV cache capacity.

compute_hash()

Hash AuxOutput settings that alter the model forward graph.

Source code in vllm/config/aux_output.py 
25
26
27
28
29
30
31
def compute_hash(self) -> str:
    """Hash AuxOutput settings that alter the model forward graph."""
    from vllm.config.utils import hash_factors

    return hash_factors(
        {"enable_return_routed_experts": self.enable_return_routed_experts}
    )