Creator of Grafana here, would love to fix this. Grafana like most software is not immune to bloat over time (constant battle). Saying no to features requested by many users, esp ones implemented by the community contributed is one of the hardest things. Sometimes we get it wrong and add unnecessary features or add features that make everything else harder (a pet peeve of mine, and something always try to minimize).
I am curious
* Where are there 5 layers of nested scrollbars?
* Where does Grafana override right click?
> alerting in which 80% of the alerts are 5xx errors or timeouts from grafana itself
That is strange, will have the alert team look into that. Not something I have seen in many issues reports. But I recall some timeout issues with more than 100K alert rules.
> The final nail in the coffin was waking up and all of a sudden my metrics are gone when my app hasn't changed at all. I'm tired of debugging it.
Grafana does not store any metrics, did you mean something else here?
> half broken features that is nearly impossible to use.
Any specifics would help and we can hopefully fix it.
That would be great. A few months ago in our (biophysics-ish) lab, I set up Grafana and InfluxDB to display output from some of our equipment during experiments as a bit of a gimmick, since I had just figured out how to extract real-time data from the machines and had used Grafana for monitoring my own servers. But rather than remaining a gimmick, while at times we've run into cases that feel a bit outside of Grafana's intended use, it has actually become our preferred way of watching the experiments, and far more useful than the manufacturer's software. With buttons and an ability to include some basic control interfaces, Grafana could well replace our everyday need for the manufacturer's software entirely.
you should check out kst-plot from kde it is great for real time plotting and watching of machines and processes. I use it all the time to make plots and zoom in on the interesting parts. It is so fast and easy.
great. Ignition from inductive automation is good, but old players like Allen Bradley / Rockwell Factory Talk, Citect, Wonderware, need a kick in the pants to get with the times.
can you report an issue on github for this and how to replicate? We have no reports of this scroll jumping and that sounds super annoying and something we want to fix asap.
Just wanted to say thanks for an awesome piece of software. It's one of those "I can't believe it's doing it and doing it this well, and it's free" kind of projects. For me, Grafana together with ElasticSearch and InfluxDB+Telegraf is pure monitoring/analysis/alerting bliss. Stay true to your roots and don't get tempted to turn into Kibana ;)
Upvote: https://github.com/grafana/grafana/issues/18041 and add any additional comments and context there. In addition to making the light/dark themes more accessible, I think we should add an additional high contrast one.
I'm also colorblind and a heavy Grafana user but I haven't had much difficulty reading dashboards. What are you having trouble with? The most trouble I've had is colors of multiple lines on a chart being too similar but being able to use the mouse to select a line and see what the line is and its value helps there.
The default palette is definitely a challenge, since it has pretty low contrast and Grafana's defaults use a lot of thin lines.
I especially have trouble matching colors in a legend with the drawn objects in the graph. The best help has been the hover panel, but it's unsorted by default and the highlight contrast on the panel for hovering over a line is pretty dim.
First off, Grafana is amazing! So keep up the good work.
Question: Currently alerts are fairly limited both in applicable panels and how they export/share relevant data to the chosen notification channel, as well as more advanced conditions. Is there any work going on with improving alerts?
Also to mention frequently asked feature: alerts using queries and templated variables [1][2]. Creating even simple generic dashboards with alerts is not possible, unless variables are hardcoded in hidden queries.
Agreed. If Grafana alerts supported templated variables I would migrate a decent chunk of my monitoring stack. But without templated variables the alert system feels half baked and I almost prefer it not even be a feature.
How about adding support for configuring Grafana with files?
This would allow us to deploy immutable Grafana instances using infrastructure-as-code tools like Terraform and Docker. Specifically, I want a way to eliminate post-install steps such as adding users and organizations and setting user passwords.
Wouldn't a better way to configure users and passwords be to use an external auth provider such as OAuth or LDAP? If you don't already have a auth provider, you could deploy Keycloak as your user DB.
OAuth/LDAP are great for large organizations. Small organizations don't need such complexity.
Even if you use OAuth/LDAP for accounts and passwords, Grafana still requires manual steps to create the organization, install plugins, and set the admin password. I wrote a Python script to do this on Docker container startup. That script is a workaround for Grafana server's poor ops design. It's extra code that we must maintain.
At work we have to compile everything from source because compliance, and Grafana is one of the harder ones to get right. There used to be good documentation for it a few versions back but I can't seem to find it lately. Is there any good documentation on how to get Grafana compiled from source?
I've got a complicated proxy setup to contend with and have to build via Jenkins on some older Linux boxes, so it's just trial and error. You have a build script you'd be willing to share?
I have a mongodb database with several collections. All collections share a set of fields: "createdAt", "channelId", "status", etc.
How easy would it be to dump all this data into Grafana, and then be able to slice and dice these collections by any combination of the shared fields.
Basically, I'm tired of writing mongo queries whenever I want to inspect my data. I want to point and click around a beautiful UI, with data visualizations and easy filtering/aggregation. Is this easy to do with Grafana?
If you have mongodb enterprise you can use Mongo bi connector and connect it with any bi tool. I recommend metabase since it is open source and very good for slice and dice and reports.
The mongodb bi connector is a postgres with foreign data wrapper to mongodb. So even if you don't have a mongodb subscription, you may be able to pull this off using the OS version https://github.com/EnterpriseDB/mongo_fdw
have you seen any implementations of internal tracing like APM being used with Grafana? I think lack of a good open source alternative to New Relic or Datadog APM kinda killed OSS monitoring for my company.
The Jaeger integration is a good step in the right direction though, kudos!
AFAIK, You can have internal trace markers, not just between apps. I've used that (for Jaegger) in one of our Windows Application to see the call-graph (it's a level editor for our game), and it was tracking the loading of various resources across multiple threads, then it was visualized with jaeger and even zipkin.
when you post release announcements that possibly get posted snd forwarded to other places, please include a short description about what grafana is.
and maybe also add one to your home page. (make it link to the features)
when i see any release anouncement about a product i am unfamiliar with, the first question i have is what kind of product it is, so i can decide if i want to check it out
Hi Torkel, this issue has been open since 2016[1]. There is incredible demand for it yet complete silence from Grafana on committing to the effort. Can I get official word on this please?
Have there been no efforts to add this with a PR? I mean Grafana is open source and if the project leads wants to focus on other things that's his choice.
It has no support for rrdcached flush, AVERAGE CF is hardcoded , and I cannot seem to get it to not connect lines through gaps where there were no measurements.
Hi Torkel,
Just finished watching the GrafanaCon 7.0 Release talk. There are some great Usability improvements and new functionality which will be definitely useful for my upcoming project.
Is it now possible to create a horizontal lined based on a single value (either a fixed or a computed number)?
For example when I'm tracking the power grid frequency, I would like to add two lines at 50.05 and 49.95 Hz, as well as one at 50 Hz (with different color).
Others have voiced their desire to display a horizontal line at the 95th percentile in a bandwidth usage graph.
Thanks for adding more support for logs in Grafana.
I'm not using AWS CloudWatch Logs because of the financial risk of a runaway server running up a huge AWS bill. I had difficulty getting log display and log search working well in Grafana 6.6.2 and InfluxDB. Do you plan to improve log support for other databases?
We have all sorts of data flowing into Prometheus and charted in Grafana. Grafana lets me alert based on thresholds and that works fine for many things.
EXAMPLE 1:
Imagine a service. Load avg of 2 is unacceptable and means things will be slowing down. However, a heavy job runs every night and pushes load averages between 1.5 and 2.5 during a 30 minute window.. I wanted to set a load avg alert threshold @ 2.0 but can't since it would fire false alarms too often. So i end up setting it at 3, and end up getting a late alert when things get really bad.
Instead, an anomaly detection would look for a number of standard deviations away from a mean, and ideally account for seasonality (i.e. compare the data with same data last few weeks). Now, things will behave better. If loads hit 2 during the day, it'll fire. But during that 30 minute window at night, loads would need to exceed 3 before it fires.
EXAMPLE 2:
Let's say I'm tracking 30 metrics for a service. I understand the service well enough to set thresholds in 5 metrics. However, I'm sure a failure won't happen alone and the other 25 metrics may show some oddities and give me an early warning of "something wrong happening" .. Here, some kind of loose "anomaly alerts" could help as an early warning. If a certain metric throws too many false alerts, we find a better way to monitor it.
EXAMPLE 3:
Lets say you have a service that auto scales. There's no limit to how many requests the service handles. So I can't really thresholds on requests/5min. Here, some kind of "anomaly alert" would be useful to let the team know that something is going on that's causing a high level of requests. Again, accounting for seasonality because things tend to change on weekends.
At the end of the day, an anomaly alert is not some "magic solution" but just and extra tool in the toolkit to use when it makes sense. Datadog and AWS both have built in easy to use anomaly alerting. Would be nice for grafana/prometheus to have something similar.
Are there plans for letting power-users style their own custom panels using html, css and js using just the web interface? Meaning without having to develop a plugin.
Do you plan to make debug logging useful? I had difficulty troubleshooting Grafana & InfluxDB auth problems and found that debug logging did not provide any extra detail.
Does the new plugin architecture allow me to provision plugins from a config file on startup? Like installing the Kubernetes App without `grafana-cli plugins install kubernetes-app`
A few things have been added that may help:
1. only visible data is queried and loaded
2. If you have multiple panels that use the same data, you can now just reuse the results (or a subset of the results) in a new panel. This will cut down on additional queries that may not be necessary
Haha, my team is going the opposite direction: we're leaving prometheus + grafana to datadog, and we're loving it. having every AWS monitoring default dashboard work right out of the bat is a bliss..
We just moved everything from datadog to Grafana, Prometheus and Loki, mainly due to cost. Honestly though, I feel like we get more out of the Grafana stack. Its powerful but simple enough to keep it pleasant to work with.
How about don't? Datadog offers everything that grafana does plus alerting, as well as more and better integrations (databases, AWS, middleware, etc...).
The cost aspect is one thing. Vendor lock-in another. Probably need to have a team that can lead the transition towards Grafana, educate and speak for it. Grafana has an enterprise plugin for datadog that potentially could ease the migration
Grafana Labs | Frontend Engineer | Remote Europe or EST timezone
Want to work on an open-source data visualization & observability platform? We are using Typescript, React, RxJS, Redux & Emotion among other things. Have some really cool projects coming up and looking to expand our team of frontend engineers! Grafana Labs is a remote first company.
Today is a momentous day for the Grafana project and for raintank, the company behind Grafana. We’re rebranding as Grafana Labs, and announcing something we’ve been working on for the better part of a year: GrafanaCloud. We are also merging our website sites into https://grafana.com We are also launching a community site for everything metrics & monitoring related on https://community.grafana.com
I am curious
* Where are there 5 layers of nested scrollbars? * Where does Grafana override right click?
> alerting in which 80% of the alerts are 5xx errors or timeouts from grafana itself
That is strange, will have the alert team look into that. Not something I have seen in many issues reports. But I recall some timeout issues with more than 100K alert rules.
> The final nail in the coffin was waking up and all of a sudden my metrics are gone when my app hasn't changed at all. I'm tired of debugging it.
Grafana does not store any metrics, did you mean something else here?
> half broken features that is nearly impossible to use.
Any specifics would help and we can hopefully fix it.