Skip to main content
Toolathlon-Verified · Website task ID 141 · Canonical task ab-testing View the task source at d57361c0

​
Required Tools

MCP Servers
google-cloud
filesystem
Local Tools
claim_done
python_execute
handle_overlong_tool_outputs
manage_context
history

​
Instruction

The A/B test for our new homepage has concluded, and the raw clickstream data has been stored in the ab_testing dataset in BigQuery. Analyze the data and fill the record.csv, determine which version (‘A’ or ‘B’) has the highest overall conversion rate, i.e., the overall conversion rate is defined as the arithmetic mean of the per-scenario conversion rates. If version B outperforms, immediately create a new Cloud Storage bucket whose name is prefixed with promo-assets-for-b for the full promotion, and you do not need to write any log entry in this process. If version A wins or the results are a tie, no bucket creation is required, but a log entry with the message {'status': 'AB_Test_Concluded', 'winner': 'A', 'action': 'No_Change'} must be written to the existing log bucket whose name is prefixed with abtesting_logging. During the entire process, there is no need to consider any log entries in the log bucket that existed before the task was executed.

​
Initial State

​
Local Workspace

workspace/ └── record.csv

​
Runtime Setup

This task initializes application or service state during preprocessing. Review the pinned setup source.

​
Legacy Trajectories

These replays were produced on the original Toolathlon release. They are retained for historical inspection and are not Toolathlon-Verified results.