Understanding Benchmarking
In business and technology, a benchmark is a standard against which you can measure performance. Think of it as a fixed reference point. Software developers use benchmarks to test an application's speed, efficiency, and resource usage compared to industry
standards or previous versions. This process is vital for identifying bottlenecks, optimising performance, and gaining a competitive edge. For artificial intelligence, benchmarks are standardised tests—collections of problems with known answers—used to score and compare different models. Traditionally, these benchmarks are external standards you aim to meet. The idea of creating your own, however, flips this concept on its head.
The Power of a 'Rough Answer'
A 'rough answer' is an initial, unpolished estimation or solution based on limited information. It's the back-of-the-envelope calculation, the first draft, the intuitive guess. This approach is famously captured in 'Fermi problems', named after physicist Enrico Fermi, who was skilled at making remarkably accurate estimations with very little data. A Fermi problem—like estimating the number of piano tuners in a city—forces you to break down a complex question into smaller, more manageable parts and make reasonable assumptions. This process provides a good base estimate when you have little or no data to start with. In business, this is equivalent to a Rough Order of Magnitude (ROM) estimate, used to check a project's feasibility before all details are known. The value isn't in getting the exact right answer, but in creating a logical starting point.
From Answer to Benchmark
So how does a rough guess transform into a valuable benchmark? It happens through an iterative process. Iterative design is a methodology based on cycles of prototyping, testing, analysing, and refining. You start with your initial 'rough answer'—be it a simple spreadsheet, a basic workflow, or a human-generated response to a query you want an AI to handle. This first attempt becomes your Version 1.0 benchmark. Every subsequent development of the 'tool'—whether it's a new piece of software, a refined business process, or a smarter AI model—is measured against this initial standard. Did the new version make the process faster? More accurate? More intuitive? The goal is to continuously improve upon your own baseline in a focused way. This method turns an abstract goal into a series of concrete, measurable steps.
The Mindset in Action
This principle is especially powerful in the world of AI. When developing specialised AI models, relying solely on generic, public benchmarks can be misleading. Those tests might not reflect the unique tasks you need the AI to perform. A more effective approach is to create your own task-specific evaluation. For example, you could write your own 'good' responses to 100 customer queries. This human-generated dataset becomes the benchmark. You then fine-tune the AI model with the goal of exceeding the quality and consistency of your own answers. This same logic applies to personal productivity. Instead of endlessly searching for the 'perfect' task management app, you could start with a simple notebook. Your manual system becomes the benchmark to beat. Any app you test must prove it is significantly better than your rough, homegrown solution.
Escaping the Perfection Trap
The biggest obstacle to progress is often the 'perfection trap'—the belief that we must start with a flawless solution. This mindset leads to procrastination and prevents innovation. We spend more time planning and searching for ideal tools than doing the actual work. Embracing the 'rough answer' approach is a powerful antidote. It reframes the goal from achieving immediate perfection to making incremental progress. It gives you permission to start with something 'good enough,' which is often the only way to get started at all. By establishing your own baseline, you create momentum and a clear path for improvement, moving from a state of paralysis to one of purposeful action and continuous learning.
