AI evaluation

Package {{ package }} · generated {{ generated }}

{{ total }}events compared
{% for name, count in outcomes.items() %}
{{ count }}{{ name }} ({{ (count / total * 100) | round | int }}%)
{% endfor %}

Precision and recall per label

Precision: of the events the AI gave a label, the share where it was true. Recall: of the events that truly had a label, the share the AI named exactly.

{% for row in metrics %} {% endfor %}
LabelTrue eventsAI answersPrecisionRecall
{{ row.label }}{{ row.events }}{{ row.predicted }} {{ "%.2f" | format(row.precision) if row.precision is not none and row.precision == row.precision else "–" }} {{ "%.2f" | format(row.recall) if row.recall is not none and row.recall == row.recall else "–" }}

Confidence thresholds for {{ (target_precision * 100) | round | int }}% precision

Answers at or above a label's threshold can be trusted; review the rest. ⚠ marks labels with fewer than ten answers, whose thresholds are unreliable.

{% for row in thresholds %} {% endfor %}
AI labelAnswersPrecisionThresholdKept
{{ row.label }}{{ row.answers }}{{ "%.2f" | format(row.precision) }} {{ "%.2f" | format(row.threshold) if row.threshold is not none and row.threshold == row.threshold else "never" }} {{ row.kept }} {% if row.fewAnswers %}⚠ few answers{% endif %}

Events the AI did not get exactly right ({{ failures | length }})

{% for failure in failures %} {% set event = failure.event %}

{% if event.outcome == "wrong" %}✕{% else %}◐{% endif %} {{ event.eventID }} · true: {{ event.truth }} · AI: {{ event.ai }} · {{ event.outcome }} · AI confidence {{ "%.2f" | format(event.confidence) }}

{% for image in failure.images %}
{{ image.name }} {% for box in image.boxes %}
{{ box.scientificName or box.observationType }}{% if box.classificationProbability %} {{ (box.classificationProbability | float * 100) | round | int }}%{% endif %}
{% endfor %}
{{ image.name }} · {{ image.timestamp }}
{% endfor %}
{% else %}

✓ The AI got every event exactly right.

{% endfor %}