Method category: Evaluative product experiment
How to Use This in GLIDR
A Dashboard gives you a visual display of your most important product metrics so that you can track changes over time and spot anomalies quickly and easily.
In GLIDR, this may be a one-time Experiment that you set up, or it can be an ongoing task. To set up the Dashboard, first you must decide exactly which metrics are the most important ones that you want to track. After you figure this out, enter this information into the Plan phase of an Experiment and decide which target metrics you want to hit for each of these items. In the Run phase, connect your metrics or your whole Dashboard as a piece of Evidence and track them over the time period you selected. Finally, in Analyze, reflect back on what you learned from the Dashboard and if you hit your targets, then iterate on the metrics you are tracking if needed.
Learn more about each of those aspects of GLIDR:
Article excerpted from The Real Startup Book
A dashboard is a simple view of product metrics that displays information about the general health and viability of the product. While more detailed metrics are generally used to analyze the results of a specific feature-level experiment, a dashboard can indicate when complex factors are affecting the product. For example, competitor behaviors, seasonality, or multiple conflicting experiments are all potential factors.
- What is going on in the business right now?
- How is the situation changing over time?
- Do we have any major blind spots?
- Does everyone in the company have access to the right metrics in order to track progress to overarching goals?
- Are we making the right decision right now?
- What are our current priorities?
In situations where there is great uncertainty, planning for the future has less value than having a clear picture of the current status. Dashboards give you a visual "information radiator," which shows the exact current status on key metrics affecting operations:
- How much of a product is built, or a goal already achieved?
- What is the state of current channel testing in marketing?
- What is our market share?
Dashboards help visualize inter-relationships among parts of a business. For example, a $10k investment in a channel may seem like a lot of money, unless you knew that last year's revenue was $250k.
Dashboards are inherently motivating. They presuppose an open and data-driven culture. For many employees and partners, this level of trust and transparency motivates them to do their best work. By going through the effort of choosing one or a handful of key metrics for the whole organization, you generate a lot of focus. Dashboards help maintain this focus operationally, if everyone continually checks a dashboard that contains those key metrics driving the business.
This technique can be used for:
- The company as a whole
- Specific departments
- Key roles (i.e., the VP of Marketing's dashboard)
- Individual contributors
In terms of how it works, it can be anywhere from "manually using a spreadsheet" to a custom-built monitoring system that integrates a number of the business systems so that you have a "real-time view" of the company.
Approaches that may or may not be helpful:
- AARRR: pirate metrics can help
This method tends to require a significant investment of thought to decide what needs to be on each dashboard (1-5 days). The implementation of the dashboard itself can vary widely. If done manually, it could cost one hour a week of a junior employee's time. If automated, there would be no recurring cost, but instead a potentially significant up-front technical implementation cost. The actual cost would vary widely based on exactly which systems and data need to be visualized. There are also off-the-shelf SaaS solutions that can provide a sufficient subset of the data required in order to reap most of the benefits in a small company, without bearing a significant cost.
- Get all of the key decision makers in one room, ideally physically (even if it's one startup founder).
- Decide what needs to be on the dashboard(s). What are the key metrics and drivers of the business as a whole?
- Design a process and/or a visualization of those key metrics.
- Include visual cues to help interpret quantitative data, i.e., two standard deviations.
- Consider how much you need to integrate "change over time" or cohort analysis in the dashboard, so that it's actionable.
- Publish them in a visually accessible place for everyone they affect.
Possible resources include:
- Google Sheets: manually keep track of key metrics in a spreadsheet
- Cyfe.com: basic integration with a lot of standard startup tools
- Geckoboard.com: geared towards being a TV interface in an open office space
- GuidingMetrics.com: builds dashboards for small businesses
- BareMetrics.com: subscription analytics and insights for SaaS or other subscription businesses
- Mixpanel.com: decent free option for product-level analytics
- Tableau: enterprise-level data visualization tool
Dashboard colors, shapes (traffic light), and status icons help you quickly interpret the reported data. The size of each dashboard component should also reflect the importance of the particular data point.
Graphs and visualizations can easily be misleading.
- Using scales on an axis that doesn't start from zero will make the immediate trend swings seem much bigger and therefore draw attention away from the fact that the absolute value is quite high. For example: https://blog.heapanalytics.com/wp-content/uploads/2014/04/misleading1_yaxis.png
- Labels and naming on axes are often overlooked or unclear.
- Data can intentionally or inadvertently be left out, making it possible to draw conclusions that do not reflect the full picture of the situation.
- Sources should be fully documented, clear, and agreed upon by all parties.
- Using cumulative graphs rather than breaking down data by time period, e.g., https://qz.com/122921/the-chart-tim-cook-doesnt-want-you-to-see/
- Ignoring conventions, i.e., pie charts that don't add up to 100 percent.
- "A good dashboard communicates everything you need to know even when looking at it from across the room." @LaunchTomorrow
- Got a tip? Add a tweetable quote by emailing us: firstname.lastname@example.org
- Geckoboard: Pirate Metrics (AARRR) dashboard example
- VentureBeat: 6 dashboards I use daily—and why every startup CEO should as well
- Got a case study? Add a link by emailing us: email@example.com
- Statistics How To: Misleading Graphs: Real Life Examples
- Edward Tufte: Blog
- Got a reference? Add a link by emailing us: firstname.lastname@example.org