Dashboards

Overview

Method category: Evaluative product experiment

A Dashboard gives you a visual display of your most important product metrics so that you can track changes over time and spot anomalies quickly and easily.

In GLIDR, this may be a one-time Experiment that you set up, or it can be an ongoing task. To set up the Dashboard, first you must decide exactly which metrics are the most important ones that you want to track. After you figure this out, enter this information into the Plan phase of an Experiment and decide which target metrics you want to hit for each of these items. In the Run phase, connect your metrics or your whole Dashboard as a piece of Evidence and track them over the time period you selected. Finally, in Analyze, reflect back on what you learned from the Dashboard and if you hit your targets, then iterate on the metrics you are tracking if needed.

In Brief

"A dashboard is a simple view of product metrics that displays information about the general health and viability of the product." While more detailed metrics analyze specific feature-level experiments, dashboards indicate when complex factors affect the product, such as competitor behaviors, seasonality, or multiple conflicting experiments.

Helps Answer

• What is going on in the business right now?
• How is the situation changing over time?
• Do we have any major blind spots?
• Does everyone in the company have access to the right metrics in order to track progress to overarching goals?
• Are we making the right decision right now?
• What are our current priorities?

Tags

• Visual
• Metrics
• Tracking
• Operations
• KPI

Description

In situations with great uncertainty, dashboards provide a clear picture of current status rather than focusing on future planning. They function as visual "information radiators" showing key operational metrics:

• How much of a product is built, or a goal already achieved?
• What is the state of current channel testing in marketing?
• What is our market share?

Dashboards help visualize business inter-relationships. For example, understanding that a $10k investment represents a proportion of last year's $250k revenue provides important context.

"Dashboards are inherently motivating. They presuppose an open and data-driven culture." Choosing key organizational metrics generates focus and transparency that can motivate employees and partners.

This technique applies to:

• The company as a whole
• Specific departments
• Key roles (i.e., the VP of Marketing's dashboard)
• Individual contributors

Implementation ranges from manual spreadsheets to custom monitoring systems providing real-time company views.

Time Commitment

Deciding what belongs on each dashboard requires significant thought investment (1-5 days). Implementation varies widely:

• Manual approach: approximately one hour weekly for a junior employee
• Automated systems: potentially significant upfront technical costs with no recurring expense
• Off-the-shelf SaaS solutions: modest cost for small companies capturing most benefits

How To

1. Get all key decision makers in one room, ideally physically (even if it's one startup founder).
2. Decide what needs to be on the dashboard(s). What are the key metrics and drivers of the business as a whole?
3. Design a process and/or a visualization of those key metrics.
4. Include visual cues to help interpret quantitative data, i.e., two standard deviations.
5. Consider how much you need to integrate "change over time" or cohort analysis in the dashboard, so that it's actionable.
6. Publish them in a visually accessible place for everyone they affect.

Interpreting Results

Dashboard colors, shapes (traffic light), and status icons help you quickly interpret the reported data. The size of each dashboard component should also reflect the importance of the particular data point.

Potential Biases

Graphs and visualizations can easily be misleading:

1. Using scales on an axis that doesn't start from zero will make the immediate trend swings seem much bigger and therefore draw attention away from the fact that the absolute value is quite high.
2. Labels and naming on axes are often overlooked or unclear.
3. Data can intentionally or inadvertently be left out, making it possible to draw conclusions that do not reflect the full picture of the situation.
4. Sources should be fully documented, clear, and agreed upon by all parties.
5. Using cumulative graphs rather than breaking down data by time period.
6. Ignoring conventions, i.e., pie charts that don't add up to 100 percent.

Field Tips

• "A good dashboard communicates everything you need to know even when looking at it from across the room." — @LaunchTomorrow