Which analytics products (Flurry, Mixpanel, GA) do you use? Where do they fall short?

March 18th, 2013

I'm looking to get a better feel for which analytics products people are using, and how they're being used. I'm particularly interested in feedback from mobile devs.

1. Which analytics products do you use? Do you use multiple?

2. If you use multiple products, do you use any kind of dashboard/aggregator for visualization? (Examples: Ducksboard, Geckoboard.)

3. What are you most concerned about when viewing/parsing analytics data? Are you tracking by version launch? By marketing campaign? By ad buy?

4. Where do the products you use fall short? I know that question's pretty broad, but the more insight I can get into the shortcomings of the analytics products you're currently using, the better.

Thanks for your help here. Much appreciated.

- Brendan

Paul O'Brien MediaTech Ventures CEO

March 18th, 2013

I'm probably going to go a direction that many entrepreneurs slam as I usually get this but a complete understanding of and time spent setting up Google Analytics makes it second to none.

I'm referring not to the basics, the tracking codes, setting up goals, etc. but the more sophisticated use of Events, filters, multi-channel funnels, experiments, social analytics, cost analysis, and segmentation (to name a few) to really measure every aspect of your business, optimize your conversion funnels, leverage and optimize SEM and SEO as one channel, and understand the direct/indirect implications of social media. There is really only one gap in Google Analytics and that's the ability to repeatedly track a specific individual (which, frankly when you're startup is at scale, isn't very valuable as it should be done by a Marketing Automation platform so that you can automatically do something with that individual tracking).

How does it work well for mobile? Track EVERYTHING with one set up of Google Analytics. Use filters, segmentation, and Goals and Events specific to mobile vs. web to give each resource in your organization reporting specific to their needs. That last point is key, you should be setting up Dashboards for EVERYONE in your company from the CEO to the customer service rep. Why? Each person impacts the business in ways that can and should be measured, evaluated, and optimized. The CEO may want to see top line metrics while the customer service rep should be looking at the performance of FAQs, help desk, your blog, and social media as well as their impact on retention, word of mouth (yes, you can measure that with Google Analytics - using segmentation), and conversion rates.

This is sophisticated stuff though as you really have to think strategically about how to use campaign tracking, segmentation, filters, and click stream analysis to measure your business entirely AS WELL AS looking only to your mobile products or web site(s) as specific data sets.

What's sophisticated? For example, GA is the only platform integrated directly with Adwords AND A/B testing so that you can do all of that in one place.

Do I use multiple? Yes. But GA is ALWAYS one of the multiple and you have to set the expectations in your organization that platform A will never report exactly the same as platform B. Don't try to solve for that discrepancy, instead use it to identify gaps, problems, and opportunities and focus only one on platform as the definitive data set (I prefer GA in this regard because it is so comprehensive).

Think that's all of your questions except this one, "What are you most concerned about when viewing/parsing analytics data? Are you tracking by version launch? By marketing campaign? By ad buy?" so, in GA...

Are you tracking by version launch? yes
By marketing campaign? yes
By ad buy? yes
By responsibility in my company? yes
By traffic source? yes
By traffic influence? yes
By retention rate? yes
By bounce rate? yes
By platform? yes
By A/B version? yes
By keyword? yes
By location? yes
By point of engagement? yes
By exit? yes
By viral measures? yes
By individual? NO but by aggregating like individuals? Yes

What about a Dashboard platform? I like Geckoboard but I've used it moreso for Company KPIs that are publicly displayed so they are always top of mind throughout the organization. Using GAs dashboards directly enables meetings to be much more valuable as each person in the room should have the data in front of them so we can deep dive, investigate, troubleshoot, and optimize in real time.

Bo Han tinkerer

March 19th, 2013

The following is from personal experience at my current company:

Here's my take on couple different tools out there:

Google Analytic-

Pros:

-Free (for starting out), can't really beat that

-Provides ~80% of needs

-Great for tracking traffic sources given its integration with adwords

Cons:

-Prohibited by the terms of service to sent personally identifying information (http://stackoverflow.com/questions/4030780/is-it-possible-to-use-google-analytics-to-track-single-user-account )

-Hits over 10 million on free accounts might not be processed (http://support.google.com/analytics/bin/answer.py?hl=en&answer=1070983)

-$150k annual flat fee for premium account not exactly start up friendly (http://www.google.com/analytics/premium/features.html)

-limited integration with financial transactional data.

-Complex UI with a learning curve

-Time delay, for certain cases

Mixpanel-

Pros:

-Designed to track unique users (i.e. you can track Jane Smith throughout her sessions)

-Great support from their staff, also a startup

-Great for retention and engagement

-Raw data export feature

-Realtime

Cons:

-Not so great for traffic sources, not integration with ads or CMS or financial transactions

-While the UI is simple, it can be frustrating at certain use cases

DIY Option (Ad Hoc):

Pros:

-Integration with internal data

-Can be used to provide data to users

-Can customize the reporting dashboard

Cons:

-Major engineering/IT undertaking

-Can get outdated

-Very much depends on your execution, not recommend until you have substantial traffic

1. Which analytics products do you use? Do you use multiple?

I use Mixpanel, Google Analytic, Qlikview (http://www.qlikview.com) with a custom-build data warehouse.

2. If you use multiple products, do you use any kind of dashboard/aggregator for visualization? (Examples: Ducksboard, Geckoboard.)

Very much depends on the usage case. Something like Geckoboard works great as a dashboard/conversation piece in lobby or to help Dev-Ops check that website is functioning normally. However, it's not so great for pivoting or drill downs.

Making sure the data can be exported to tools for in-depth analysis like Excel or R is always important.

3. What are you most concerned about when viewing/parsing analytics data? Are you tracking by version launch? By marketing campaign? By ad buy?

Depending on the business need/use case; .ie. a buying ads will call for a different dashboard than debugging retention problems.

It's very important to have consistency, to compare week vs. week numbers.

4. Where do the products you use fall short? I know that question's pretty broad, but the more insight I can get into the shortcomings of the analytics products you're currently using, the better.

Maintenance, maintenance, maintenance... It takes a lot of recurring effort to keep whatever analytic suite you pick current. This is organizational problem not a technical one that tends to be ignored.

Jerome Dangu CTO & Co-Founder at ClarityAd

March 18th, 2013

Hi Brendan

My goal is to:

- Understand the friction points in the engagement process, act on them and measure the results

- Measure the value of my traffic sources (lifetime value of users per traffic source)

I recently wrote an article about how I use KissMetrics for lifecycle emails that you might find useful (a bit technical...!).

www.j-ro.me/kissmetrics-for-lifecycle-emails.html

Jerome

Charles Brewer Visible Innovation: Product & UX

March 18th, 2013

Paul,

Where is the best place to get up to speed on the advanced GA practices you outline? Google's tutorials? I agree that GA offers the most comprehensive set of reporting tools.

Anonymous

March 18th, 2013

Looking forward to these answers, as we're just getting started with our analytics plans... and the options can be overwhelming.

Had a great demo today from a Dev/Entrepreneur friend of customer.io. He couldn't speak highly enough of the, and spent time showing us how he sends customer-based events to the customer.io API, which saves a lot of time in email-based lifecycle and behavior actions. Looks like customer.io does what Jerome's blog post does but with less work. It's focused on allowing a customer-development person to create metrics via GUI to drive emails, while keeping that person out of code.

Friend also mentioned they use segment.io to write events in their logic once, and let segment.io push the data to GA, customer.io, kissmetrics, and other analytics consumers.

Paul O'Brien MediaTech Ventures CEO

March 18th, 2013

Charles, that's the catch. Unfortunately, GA documentation and guides are awful and generally terribly out of date anyway because the platform is indeed evolving so quickly. Google has made some serious investments there in the last 6 months and so frankly, it's a bit like learning SEO - you can pretty easily learn and do the basics but to really understand and implement it properly and effectively is rather difficult. Sorry, I realize that's not the answer you're seeking... Google Analytics has become so sophisticated that using it properly is a bit like using Marketo or Salesforce... the real reason to figure it out is ultimately it's free, completely scalable, and so tightly integrated with and invested in by other Google technology that nothing else can keep up.

Tell you what, send me a message with a brief on your business, the website/mobile app, and a few of the key things you want to know and I'll see if I can't point you in the right direction to get you going.

Brendan Duffy Product Manager

March 18th, 2013

Jerome,

Thanks.

Here's a follow-on. You want to:

- Understand the friction points in the engagement process, act on them and measure the results
- Measure the value of my traffic sources (lifetime value of users per traffic source)

Would it be helpful for you to be able to see where other companies/apps in similar spaces might have friction points, and from where they are drawing high-value traffic?

It might work like this: You agree to make your data accessible, and it gets anonymized (that is, company name is not shown, certain attributes remain attached--you choose which). Then you can view averages across your market space/category. It's like a trade off: you make your data available, you get to see other companies' (unattributed) data. Obviously, there's a tradeoff here. Would it be worth it your eyes? Would you allow your data to be accessed?

Thanks again.

Paul,

Thanks a ton for the insight. Correct me if I'm wrong, but I from your answer, I get the following:

(i) You can do pretty much anything you want to do with Google Analytics

(ii) Understanding GA well enough to do everything you want comes with a significant learning curve

Anything I'm missing here? Can you monitor push campaigns w/ GA? (I looked through the documentation briefly, and wasn't able to find anything for push tracking.)

--> Any other FD members have any input? I'd love to hear from Mixpanel/Flurry users.

Thanks,

Brendan

Toren Ajk Digital Marketer

March 19th, 2013

When I'm looking at analytics platforms it is purely from a marketing perspective, in particular on the mobile side these days. In the big picture what I want is to be able to measure a campaign's ROI based on the LTV of the users generated. Basically marry cost data to revenue data and view it in multiple instances.

In a more granular sense that means being able to easily track/compare/optimize based on the various levers I have control over, e.g. creative, landing page, mobile carrier, time of day, geo-location, site/category, etc. The ability to slice thinly, e.g. what is the LTV of a downloand using Ad1 on Network 1, Network 2, Network 3, going to Landing Page 1, Landing Page 2, on Verizon, in Manhattan Beach between 1-3pm on Tuesdays.

The ability to capture data and run analysis like that is in my eyes the holy grail in marketing analytics.

Raymond Colletti

March 19th, 2013

Hi Brendan,

I work at Datadog which is specifically relevant to your second question (I don't want to advertise via this forum but we're pretty awesome): http://www.datadoghq.com/integrations/

Feel free to message me any thoughts or questions you may have!

Cheers,

Ray

Paul O'Brien MediaTech Ventures CEO

March 20th, 2013

Brendan

You can do pretty much anything you want to do with Google Analytics - Yes

Understanding GA well enough to do everything you want comes with a significant learning curve - I wouldn't say the HOW is steep, someone who knows could talk you through it all in a couple hours. They WHAT and WHY are what matters (in any analytics platform): thinking strategically about the business and setting up your platform to provide the intelligence that's meaningful and actionable and not just what you think you want.

Push campaigns: Google's iPhone and Android analytics platforms are new and in beta; I'm not even sure everyone has access yet. If/when you do, it uses Goal and Event tracking in a manner similar to a website and though I haven't set it up yet, I can't conceive of any reason the Event tracking couldn't be scripted into pushes to track them.

Question stats