​

Related Pages

Related topics: Configuration

The following files were used as context for generating this wiki page:

• src/starfish/telemetry/init.py
• src/starfish/telemetry/posthog_client.py
• src/starfish/data_factory/factory_.py
• src/starfish/data_factory/utils/data_class.py
• src/starfish/common/env_loader.py
• src/starfish/llm/prompt/prompt_template.py

​

Telemetry

Telemetry within Starfish is designed to collect minimal and anonymous data to help improve the library. It provides insights into the usage and performance of different features, aiding in identifying areas for optimization and bug fixing. Participation is optional, and users can opt-out via an environment variable.

The telemetry system uses posthog to send events. It collects data related to job execution, platform, and configuration to provide insights into the library’s usage and performance. src/starfish/telemetry/posthog_client.py, src/starfish/data_factory/factory_.py

​

Telemetry Configuration

The telemetry system is configured using environment variables and a configuration file stored in the application data directory. src/starfish/telemetry/posthog_client.py

​

Configuration Parameters

The AnalyticsConfig dataclass holds the configuration parameters for the telemetry service. src/starfish/telemetry/posthog_client.py

​

Opting Out

Users can disable telemetry by setting the TELEMETRY_ENABLED environment variable to false. README.md

# Example of disabling telemetry via environment variable
TELEMETRY_ENABLED=false

Sources: README.md, src/starfish/common/env_loader.py

​

Telemetry Data Collection

The telemetry system collects data related to data factory jobs and sends it to the analytics service. src/starfish/data_factory/factory_.py

​

Telemetry Events

Telemetry events are represented by the Event dataclass, which includes the event name, data, and a unique client ID. src/starfish/telemetry/posthog_client.py

@dataclass
class Event:
    """Represents a telemetry event."""

    name: str
    data: Dict
    client_id: str = field(default_factory=lambda: str(TelemetryConfig.generate_client_id()))

Sources: src/starfish/telemetry/posthog_client.py

​

Data Factory Telemetry

The DataFactory class in src/starfish/data_factory/factory_.py sends telemetry events at the end of a job. This includes information about the job configuration, execution environment, and outcome. src/starfish/data_factory/factory_.py

The TelemetryData dataclass is used to structure the data sent with the telemetry event. src/starfish/data_factory/utils/data_class.py

@dataclass
class TelemetryData:
    """Class representing telemetry data for data processing jobs.

    This class captures various metrics and statistics about the execution of data processing jobs,
    including job configuration, execution time, and outcome summaries.
    """

Sources: src/starfish/data_factory/utils/data_class.py

​

Telemetry Data Attributes

​

Sending Telemetry

The _send_telemetry_event method in the DataFactory class is responsible for sending the telemetry data to the analytics service. src/starfish/data_factory/factory_.py

def _send_telemetry_event(self):
    """Send telemetry data for the completed job.

    Collects and sends job metrics and error information to the analytics service.
    """
    telemetry_data = TelemetryData(
        job_id=self.config.master_job_id,
        target_reached=False,
        run_mode=self.config.run_mode,
        run_time_platform=get_platform_name(),
        num_inputs=self.input_data_queue.qsize(),
        library_version=__version__,  # Using the version here
        config={
            "batch_size": self.config.batch_size,
            "target_count": self.config.target_count,
            "dead_queue_threshold": self.config.dead_queue_threshold,
            "max_concurrency": self.config.max_concurrency,
            "task_runner_timeout": self.config.task_runner_timeout,
            "job_run_stop_threshold": self.config.job_run_stop_threshold,
        },
        error_summary={
            "err": str(self.err),
        },
    )
    if self.job_manager:
        telemetry_data.count_summary = {
            "completed": self.job_manager.completed_count,
            "failed": self.job_manager.failed_count,
            "filtered": self.job_manager.filtered_count,
        }
    self._analytics.capture_event(Event(name="data_factory_job_complete", data=asdict(telemetry_data)))

Sources: src/starfish/data_factory/factory_.py

​

Analytics Service

The AnalyticsService class handles the communication with the analytics backend (Posthog). src/starfish/telemetry/posthog_client.py

​

Service Setup

The _setup_client method configures the Posthog client. It checks if telemetry is active and initializes the client with the API key and endpoint. If telemetry is disabled, it uses a NoOpPosthog client. src/starfish/telemetry/posthog_client.py

​

Capturing Events

The capture_event method sends an event to the analytics service. It ensures that the event data includes the client ID. src/starfish/telemetry/posthog_client.py

def capture_event(self, event: Event) -> None:
    """Capture an event with the analytics service."""
    if not self.settings.active:
        return

    try:
        self.client.capture(event.client_id, event.name, event.data)
        if self.settings.verbose:
            print(f"[telemetry] Captured event: {event.name} with data: {event.data}")
    except Exception as e:
        print(f"Failed to send telemetry event: {e}")

Sources: src/starfish/telemetry/posthog_client.py

​

Client ID Generation

The TelemetryConfig class is responsible for generating and retrieving a unique client identifier. The identifier is stored in a file in the application data directory. src/starfish/telemetry/posthog_client.py

@staticmethod
def generate_client_id() -> UUID:
    """Generate or retrieve a unique client identifier."""
    config_path = path.join(path.expanduser(APP_DATA_DIR), ".starfish_config")
    makedirs(path.dirname(config_path), exist_ok=True)

    if path.exists(config_path):
        with open(config_path) as f:
            return UUID(f.read().strip())

    new_id = uuid4()
    with open(config_path, "w") as f:
        f.write(str(new_id))
    return new_id

Sources: src/starfish/telemetry/posthog_client.py

​

Conclusion

The telemetry system in Starfish provides valuable insights into the library’s usage and performance, aiding in continuous improvement. It is designed to be minimally invasive and respects user privacy by allowing users to opt-out. The data collected helps the developers understand how the library is being used and identify areas for optimization.