Glad I’m not alone in this. It’s rare that an interface makes me feel as dumb and frustrated as GA4
One of the more baffling choices they made was to have “today” and “yesterday” as options on the user reporting tab, even though it can take up to 48 hours to process the data and my dashboard always shows zero users for “today”. Took a while to figure out what was going on there…
Meanwhile the real-time reporting works reasonably well but is limited to a 30 minute window. Some real head-scratching choices were made here.
Exactly. I'm currently building in-house analytics (thanks Clickhouse) due to GA4 as well.
The only way I'm able to use GA4 in any way is with their search box + autocomplete, you can sometimes get a decent query result, then you click the three dots next to the tiny line chart, export as CSV and you get the data.
And for events, I think they want you to pay for BigQuery, so you only get some kind of aggregate count, which is useless. It'a too confusing to figure out if I'm wrong about that.
We (request metrics, author) are also using clickhouse. But we go beyond analytics to integrate performance, security, api monitoring, and errors under a single interface. We think of it as “client side observability”.
Agree on all points.
It's extra frustrating because GA exposes some metrics that are hard to get elsewhere like traffic sources, organic search, accurate user metrics, live geo location, etc. The only reason I keep it activated (alongside a more flexible alternative) is because it's free.
Engineer's Job is to carry out product's vision as faithfully as possible. Yes, can give suggestions and dispute things but engineers can't drive 100% of the product's function and if they can, what's point of having a separate function of product managers.
There has never ever been a better time to jump ship from GA. All of your historical GA data will NOT be viewable in your GA4 property. The biggest moat they had was millions of sites with multi year page view data for easy trend comparisons. The moat is now broken and they don't care. GA4 and the forced migration is being handled so bad the only explanation is they would rather nobody use GA anymore.
Yep, I just setup Umami (https://umami.is/) yesterday and added it to some properties alongside GA to see how it goes. It's a very simple interface with everything I really need for web analytics so I'm enjoying it so far. I self-hosted so if it sticks around the only thing I might look at is having a replica running for it (already put a high frequency backup in place).
People who don't think the forced change to Google Analytics 4 is a big deal may want to consider talking to actual business users of the current Google Analytics (because Google clearly isn't). GA4 is clearly aimed at data scientists. Even as someone who owns data teams it is a frustration to use. Try it for yourself, count the steps necessary to determine many page view counts a certain page is receiving in current "Universal Analytics" vs. the new GA4. That is if you can even figure out how to do it in GA4.
And it's all a shame because Google already acquired a great data science/viz tool, Looker, and that's a perfectly fine place to do data analytics/science rather than forcing it in Analytics.
Because it stores data on a user's computer, isn't using localStorage to track sessions still considered to be a "cookie" for ePrivacy Directive purposes? To that end, I don't think it's not "no cookie banner" compliant for the law as it exists today.
(Note: It _may_ be compliant for future updates of the directive coming in a couple of years, but iirc that isn't out yet.)
just to point it out, laws like GDPR do not mention "cookies" anywhere. It's regulating how you may or may not retrieve and store ANY data that makes an individual identifyable.
The whole "cookie" topic is a dark pattern mostly pushed by ad providers as a friendly word to bypass having to say "may we track everything you do?".
So if it tracks data points that can be used to clearly identify a visitor, or marks the visitor in a way that can be used to personally identify them later, it will need the users consent, regardless of where and how this is stored.
The parent was talking about ePrivacy Directive which is a regulation for companies registered and operating in EU. That is where the cookie banner requirement comes from.
The ePrivacy Directive does mention cookies in its text, but it says "cookies or similar devices", so it clearly includes localStorage, which the 2002 text predates.
And there was NEVER a requirement for a "cookie banner" in its text. Consent under it can be given in any other way that are not as intrusive, but it was the industry who chose the stupid banner and invasive analytics. Also notice that under both GDPR and ePrivacy Directive, consent is not required for cookies/etc with a legitimate purpose. Examples of those have been given under the ePrivacy Directive from almost the beginning, but more explicitly in the "Opinion 04/2012 on Cookie Consent Exemption" which is from 2012 and predates cookie banners.
[1] Here's an example of asking for consent to store a cookie that doesn't involve a banner: https://www.williamgrant.com . It's in the age-check modal.
Indeed. The IAB has, imo, purposely made the “consent” modal as confusing and annoying as possible to train users to just accept. All those different purposes… it is all very confusing even for an engineer like me who’s job it is to work with (due to legal department’s conviction that unless we have such a modal we are breaking the law)
What "proposal"? I'm talking about the checkbox in the bottom. It is asking for consent to store PII in a cookie, and it doesn't need a banner.
I was answering to a claim that cookie banners are required by the ePrivacy Directive. There is no requirement for banner anywhere there or in GDPR. They were invented by the advertisement industry, and almost all the time are found to be non-compliant.
> The whole "cookie" topic is a dark pattern mostly pushed by ad providers as a friendly word to bypass having to say "may we track everything you do?".
Go to https://gdpr.eu/ and tell me what their cookie banner says.
Just from a UX perspective, the quality of Google products seems to have tanked in recent years. Android, Sheets and Maps seem like they're still best-in-class. Maybe Gmail is still ok too (I don't know, I don't use it). But almost everything else has started to feel completely unusable to me. Even Google Fi is starting to suck.
Analytics is literally the cornerstone of Google's business model. Did Google just get too complacent at the top?
Agreed. Maps is the only product remaining where Google is a leader. There are much better alternatives to GMail and Google Search, but not to maps. I think online maps are only going to become stronger and more important in people's lives, as everything there is real and not machine generated. Apple will never be able to compete until they allow users to add places that are missing.
I'm referring to the UX, not the UI. I have Google Fi that I used to use as a backup hotspot while traveling. But then they decided I travel too much and arbitrarily cut off my data one day, so I started buying a global eSIM from Airalo and using it alongside Google Fi for calls/SMS. Then the calls and SMSes started having trouble. I reached out to Fi support, and they didn't seem to comprehend the issue. It took several days to get a response and the response was less intelligent than talking to GPT3.
"I tried the most used thing for A. Had an issue with it, so we built our own."
Maybe I'm just cynical, but I read this as the outcome of "What's the best way to market this thing we built in an already saturated market with a lot of well made free and open source solutions?"
I recently gave google analytics 4 a shot. I actually had the thought that I had more access to information, and my analytics were better in 1998. Its a disaster.
Nah a lot of it is weird UI decisions. Like to view what browser versions your users are using, in UA you just go to the browser page and click the browser and it lists the versions and number of users. In GA4... good fucking luck, you need to make some weird custom report where you need to select the right parts of your data and tell it whether they are "metrics" or "dimensions" or "rows" or "columns" or "values" or "filters" or "segment comparisons". It's a nightmare. And it's like that for basically everything other than the premade UIs, which feel like they are oversimplified and targeted more towards apps than websites. Generally it feels like I'm using a product made by people who hate the Internet and wish all their website users would go away, but hey it's free...
Most GDPR experts will tell you that traditional log files aren't compliant. They contain personally identifiable information (IP address), and you didn't get consent from the user before collecting it, nor are they required to provide the service the user requested.
CNIL disagrees and says you can need them to secure your service. Legitimate interest (art. 6(1)f) is applicable (recital 49). You can also be required by law to keep them for some time, so legal requirement is applicable (art. 6(1)c). Other uses may require consent.
Exactly. It's not just about what you collect and store, but also about what you do with the information.
You can freely collect data as a legal requirement, or for "legitimate interest" purposes such as fraud prevention. But you can't use the data you just collected for analytics without proper consent.
Contractual basis too, is often the easiest way to collect and store PII. Eg if you have a contract with someone you can often store a lot of their data to fulfil the contract.
There is meant to be a sense of proportionality, but as many things with privacy laws it's subject to interpretation and intentionally left vague.
Storing the log files (or IP addresses in general) is not a problem IF you're using them only with a legitimate interest basis.
For instance, you can use this stored IP address to help identify whether your user has had their account breached, and prompt for extra verification before letting them log in. You can also do a full browser fingerprint for this purpose, this is all covered under legitimate basis.
However, once you use any of this data to market to the user then you are in breach of the GDPR as you did not have a consent basis for it. The storage was never a problem, it's the use of it that becomes a problem.
Depends on the product, payments products generally use fingerprinting and present extra prompts if you're using an unknown device – that is kind of one of the main problems of the GDPR though, there are nuances and it's usually not white and black what can be done without specialised legal counsel (and sometimes, even then...)
Sounds like there could be an opportunity here for a GDPR noncompliant analytics product. Personally, my customers are in the United States and I don't want ambiguity in my analytics because of Lawyers who reside outside of my jurisdiction.
Technically correct, but arguable... There are lots of UK and EU-based companies that blatantly breach the GDPR and get away with it as the regulatory bodies don't have the resources to chase after every breach at home, let alone abroad.
Unless you are a huge company or have a significant amount of customers in the UK/EU it's probably okay to ignore the GDPR.
Can someone explain what the rationale is behind this move by google? Theoretically people working at google are fairly smart. But they are forcing a switch to a product that is an order of magnitude worse.
I wouldn’t be surprised if they lose over 50% of GA users with this. Why? Why is it good for google? I don’t get it.
GA4 has several advantages over UA to account for the evolving privacy landscape. For one, it supports user modeling in ways that are GDPR compliant, and you simply cannot count these users in UA. Apple and the leading browsers have made changes with cookies that have necessitated product changes, but not using fingerprinting as suggested elsewhere.
GA4 also reports on combined app+web data whereas UA did not.
Another distinction is that GA4 has better support for muti-touch attribution, e.g. all of the user ad touchpoints, for ad traffic, whereas UA was primarily last click, or first click if you look at the user acquisition reports.
I am also frustrated with Google Analytics but one thing it does very well (that I can't seem to find elsewhere) is counting unique unregistered visitors. If I had to guess this is done with some advanced fingerprinting - but I'd be curious if anyone has found an alternative for these metrics.
One thing to note is that many Google Analytics alternatives suffer from data accuracy issues.
They record and count analytics data differently from how Google Analytics does.
Therefore, before fully transitioning to Google Analytics and being surprised by a drop in visitors, it's advisable to use any analytics alternative alongside it first.
> One thing to note is that many Google Analytics alternatives suffer from data accuracy issues.
They record and count analytics data differently from how Google Analytics does.
During my years of experience using Google Analytics, I had very limited faith in Google Analytics accuracy. For example, bots routinely poisoned the referrer logs.
So holding them as a benchmark of accuracy seems off the mark for me.
I have the strong suspicion, that nobody has really great bot detection to this day.
Even with Google Analytics, I mostly used the data for rough trend analysis over longer periods, rather than react to every little 5% blip in the data.
For example, it was useful for observing the shift to mobile at differing pace for different types of websites. And that allowed for prioritizing website redesign projects during the 2010s.
And I’m also pretty puzzled at the criticism in your article of Matomo for being too close to how to Google Analytics works, while simultaneously holding Google Analytics as the gold standard to be compared against. That seems rather inconsistent to me.
I work with a lot of non-profit Drupal clients and we've switched them from GA to Plausible Analytics because of GA4. It's not free, but it's cheap enough that they're willing to pay in order to not deal with all this bullshit.
Plausible has a really simple interface, and we're able to embed the dashboard within the Drupal admin interface too so they don't even need to log in to a separate service to see their stats.
The GA4 interface along with Ads and some of the Cloud interfaces are so bizarrely complex it's comical.
I wonder what went wrong, but i'm so happy to not trapped in that hellscape anymore.
I remember it could take my team a week to even find the page something was on, or figure out the grotesquely convoluted methodology or naming scheme they had come up with for some simple task from 100 versions of their docs.
Yea the market is rammed full of privacy based analytics tools. I guess the market is pre-validated but it is probably a lot of work to break out in that market.
Not everyone can build or host their solution. And most hosted solutions outright prohibit processing personal data, throwing users and customers under the bus.
Hence, a long time ago we built web analytics, the Wide Angle Analytics. We actually support processing Personal Data when requested.
When building it, we spent more time on compliance than on technology.
I just enabled Cloudflare web analytics on my site rather than try to figure out Google's new thing. All I care about is tracking visit trends and referrers, and that's actually way easier in Cloudflare's UI so I'm happy.
One nice thing about it is that there are some pages on my site that I never bothered to add Google Analytics to, but Cloudflare adds their tracker automatically to every page. It couldn't be easier, which is great as analytics is not something I want to spend any time on.
My experience with cloudflare is that it counts every bot as a visitor, so Facebook pinging your site 5000 times in a day shows as 5000 United States visitors - not so useful to analyze. Is it different nowadays?
Not sure how it compares, I set up self hosted matomo (originally named piwik) and it's been running without issue since 2013 or so. https://github.com/matomo-org/matomo. Only useful if what you're running is on php. Probably would've picked a different stack today.
Nothing really wrong with PHP or Matomo, which is why I mention it as something I would recommend. But, it is a requirement for hosting Matomo.
I would pick a different stack today mostly due to wanting to try other stuff out, and not so much because something was broken about it. Website still loads faster than 99% of the internet today, and I'm happy with it.
My point was that Matomo is PHP, but your website doesn't have to be on the same stack.
Normally the analytics platform should be hosted on a separate server for performance and security reasons.
For example, the demo dashboard of uxwizz.com runs on PHP/MySQL on a $5/mo VPS and it's faster than most analytics platforms, even those running on big cloud infrastructures, because for the low-traffic use-case the simpler the solution and technology, the better.
I assumed that if you are self hosting, that you don't bother using a separate hosting provider for just analytics. I should probably not have suggested anything other than "matomo is nice". The technical requirements are fairly obvious, and my extra information was just easy to misinterpret. I'm sure people can figure it out. Have a nice day
I hope one day when Google forces us to migrate to Google Next Analytics Product™ my org will finally get fed up and switch to something stable. I’m philosophically opposed to analytics in the first place and would really like to never have to think about it again.
I currently use the maxmind webservices for client-side resolution in a commercial application. It's reliable but expensive enough that I'm looking for alternatives. I'm currently experimenting CloudFront's location request headers for a massive savings.
Where maxmind is $0.0003 per query.
And my CloudFront concept is $0.0000015 per query.
Well, if you want to limit your lookups to country level you can use our free IP to Country database [0]. Feel free to ask what is the catch. It's an easy answer, there is none.
The database is updated daily, there is no quality compromise, it is a database so unlimited lookups and it is licensed under CC-BY-SA 4.0 so it permits commercial use with attribution. It is a subset of our paid city level data. If you want city level data, we have the free tier on the API permits 50k lookups per month.
By the way, I already pinged you on Twitter about the database. :/
There really isn't much to learn from this article apart from that feeling about "we did it in-house instead and it does 60% of what the dumb original solution would do. Also we need to maintain it all ourselves now.":
> Not invented here (NIH) is the tendency to avoid using or buying products, research, standards, or knowledge from external origins. It is usually adopted by social, corporate, or institutional cultures. Research illustrates a strong bias against ideas from the outside
Is the problem that Google's existing customer base doesn't want the restrictions of GDPR, so Google is tying itself into knots trying to achieve the impossible?
Or is it simply that the concept of "an American company processing data on visitors to a European website" is completely illegal now, and the courts haven't gotten around to spelling it out that bluntly yet?
Change is hard I guess. A 15 + year old product, based on pageviews, that known no bounds in regards to tracking, getting replaced by a new hit-based one that is not perfect, but at least takes GDPR into account. It actually hooks up to BigQuery easily and that can get hooked up to Looker Studio. And it's free. BQ is just a few bucks a month). You can even personalize the main menu. Don't need e-commerce? Just delete it from the menu! Need custom stuff? Build it easily in 'Explore' and place it in the main menu. But, yes you do have to learn the new interface. (edit; typo)
One of the more baffling choices they made was to have “today” and “yesterday” as options on the user reporting tab, even though it can take up to 48 hours to process the data and my dashboard always shows zero users for “today”. Took a while to figure out what was going on there…
Meanwhile the real-time reporting works reasonably well but is limited to a 30 minute window. Some real head-scratching choices were made here.