Last active
October 6, 2025 13:16
-
-
Save kylebrandt/eb307b49961e3003c8c93b32dac0313d to your computer and use it in GitHub Desktop.
Revisions
-
kylebrandt revised this gist
Oct 6, 2025 . 1 changed file with 2 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,5 +1,7 @@ # Dataplane FAQ Note: much of this is moved to https://github.com/grafana/dataplane/pull/56 ## How does dataplane relate to Dataframes? Dataplane adds a data typing layer on top of data frames. By analogy Dataplane Types are to dataframes as typescript is to javascript. -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -97,3 +97,7 @@ Rather, if we want to help the user, auto-determine the type should be done with - Influx - Snowflake - Victoria metrics ## Are there examples of frames for dataplane types? Yes, see https://github.com/grafana/dataplane/tree/main/examples/data for numeric and time series examples. Each json file is an array of frame(s) as they are encoded in JSON. -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 13 additions and 12 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -81,18 +81,19 @@ Rather, if we want to help the user, auto-determine the type should be done with ## What data types work with what today? - Many visualizations work natively with the labeled types (`multi` and `wide`). (TODO: more discovery around what works with what viz). The Long types will naturally work in the table visualization. But presently `long` will not work with the timeseries viz until a user adds the `prepare time series` transform (from Grafana Transformations). - Grafana Managed Alerts and Recording rules via Server Side Expressions (SSE) support `multi`, `wide`, and `long` format formats for both `numeric` and `timeseries` kinds. Serve side expressions converts the responses to the corresponding kind. `numeric` can be directly alerted on, `timeseries` will need a SSE reduce operation (which creates a numeric result) to be alerted on. ## What are some datasources that already send dataplane data today for some of their responses? - Prometheus (and amamazon / azure variants) - Loki - Azure Monitor and Azure Data Explorer - Bigquery - Clickhouse - Cloudlflare - Databricks - New Relic - Oracle, Postgres, MySQL - Influx - Snowflake - Victoria metrics -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 17 additions and 3 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -72,13 +72,27 @@ This often appears as "format as" in the UI. This data source query information ### That is extra work for the user, isn't guessing better? Perhaps, but not at the feature/consumer level. The data source knows more about the data that comes from the system behind the data source. So this more ends up confusing the user. Rather, if we want to help the user, auto-determine the type should be done with in the DS in a way that fits with the constraints of the datasource. ## How do I know data type (kind and format) to use for my Datasource queries? ## What data types work with what today? - Many visualizations work natively with the labeled types (`multi` and `wide`). (TODO: more discovery around what works with what viz). The Long types will naturally work in the table visualization. But presently `long` will not work with the timeseries viz until a user adds the `prepare time series` transform (from Grafana Transformations). ## What are some datasources that already send dataplane data today for some of their responses? - Prometheus (and amamazon / azure variants) - Loki - Azure Monitor and Azure Data Explorer - Bigquery - Clickhouse - Cloudlflare - Databricks - New Relic - Oracle, Postgres, MySQL - Influx - Snowflake - Victoria metrics -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 11 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -60,12 +60,22 @@ Besides interoperability: ## Is dataplane defined in something like `plugin.json`? No, and this would be a very helpful evolution. Right now everything is runtime, or in code. If had a schema of query types, and the data types they returned. We could generate compatibility matrices of data sources and features. ## What if my data source is schemaless and doesn't have kinds or types? The normal pattern is to have a drop down in DS query UI to assert the query type. This often appears as "format as" in the UI. This data source query information can then be used my the data source to produce a dataplane compatible type for the response. ### That is extra work for the user, isn't guessing better? Perhaps, but not at the feature level, as the data source knows more about the data that comes from the system behind the data source. So this more ends up confusing the user. Rather, if we want to help the user, auto-determine the type should be done with in the DS in a way that fits with the constraints of the datasource. ## How do I know data type (kind and format) to use for my Datasource queries? ## What data types work with what today? -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 15 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -43,7 +43,7 @@ Consumers of data have to infer the type from looking at the data returned. This - Users get confused about how to write queries to work with different things - Error messages can become seemingly unrelated to what users are doing - Different features guess differently (e.g. alerting vs visualizations), hard for users and devs to know what to send - "Guessing code" (on the consumer side) gets more convoluted over time as more exceptions are added for various data sources. ## What if there is no type close to what my data source returns @@ -58,3 +58,17 @@ Besides interoperability: - We can suggest what actions can be taken with the data if the type is reliably known. So for example, we can suggest creating alert rules or certain visualizations in dashboards that work well that type. - We can suggest transformations that get you from the current type to another type support additional actions ## Is dataplane defined in something like `plugin.json`? No, and this would be a very helpful evolution. Right now everything is runtime, or in code. If had a schema of query types, and the data types they returned. We could generate compatibility matrices of data sources and features. ## What if my data source is schemaless and doesn't have kinds or types? ## How do I know data type (kind and format) to use for my Datasource queries? ## What data types work with what today? ## Which datasources already send dataplane data today? -
kylebrandt revised this gist
Oct 3, 2025 . 1 changed file with 60 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,60 @@ # Dataplane FAQ ## How does dataplane relate to Dataframes? Dataplane adds a data typing layer on top of data frames. By analogy Dataplane Types are to dataframes as typescript is to javascript. Specifically, a property is added to the dataframe to indicate its type with is made up of a kind and a format, for example "Timeseries Wide". There are two properties attached to each data frame for this, `frame.meta.type` and `frame.meta.typeversion`. ## What is the Dataplane "Contract"? The contract is a written set of rules for each dataplane data type. It says how frames should be formed by producers of data (datasources, transformations) and how consumers like dashboard, alerting, and apps can expect that data. In short it describes the rules for valid and invalid schemas for each type. ## What is the point? The main point is self-interoperability within Grafana between datasources and features like Dashboards and alerting. For devs: Data source authors know what type of frames to output, and authors of features know what to expect for their input. This makes the platform scalable and development more efficient and less frustrating due to incompatibilities. For users: They should see a side effect of things being more reliable and working as expected. So compatibility becomes about supporting data types and not specific features and data sources. For example, if datasource produces type "A", and alerting accepts type "A" and certain visualizations. That source should work with alerting and those visualizations. ## Why the Frack (or silly swear word of your choice) is it so complicated? Two main reasons, open tent and history. ### Open Tent The Open tent nature of our software means that there are different formats of data, in order to support those more natively, there are set of formats where generally one of them should be close to what a datasource produces, so the data source can relay the information with minimal distortion. ### Historical Retrofitting Dataframes existed before the dataplane types did. So the Dataplane formats were designed to be close to the coventions for data that had are evolved while trying remove places where things were inconsistent, and make decision one way or the other. ## How do things work when there are frames with no dataplane type? Consumers of data have to infer the type from looking at the data returned. This has a few problems: - Users get confused about how to write queries to work with different things - Error messages can become seemingly unrelated to what users are doing - Different features guess differently (e.g. alerting vs visualizations), hard for users and devs to know what to send - "Guessing code" gets more convoluted over time as more exceptions are added ## What if there is no type close to what my data source returns You can still return dataframes and work with consumers to support them. The point of dataplane types is for well established types or proposing types to be established. You can set the `frame.meta.type` to whatever string you want, and the version to whatever `x.x`. So dataplane is designed to allow data types to grow into maturity in a way not limit new innovation. ## What opportunities does dataplane provide? Besides interoperability: - We can suggest what actions can be taken with the data if the type is reliably known. So for example, we can suggest creating alert rules or certain visualizations in dashboards that work well that type. - We can suggest transformations that get you from the current type to another type support additional actions -
kylebrandt revised this gist
Oct 2, 2025 . 2 changed files with 2 additions and 1705 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -107,3 +107,5 @@ sum(rate(grafana_sse_sql_command_input_count[$__rate_interval])) by (status,atte ## What does the conversion look like? <img width="1385" height="1223" alt="image" src="https://gist.github.com/user-attachments/assets/92e5c227-1045-4725-8259-0f91962c6ac0" /> See https://gist.github.com/kylebrandt/ff5fc4061d85fe08fd24f5cee6333414 for dashboard.json of above (uses mock datasource). This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,1705 +0,0 @@ -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 1705 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,1705 @@ { "__inputs": [ { "name": "DS_MOCK", "label": "Mock", "description": "", "type": "datasource", "pluginId": "grafana-mock-datasource", "pluginName": "Mock" }, { "name": "DS_EXPRESSION", "label": "Expression", "description": "", "type": "datasource", "pluginId": "__expr__" } ], "__elements": {}, "__requires": [ { "type": "datasource", "id": "__expr__", "version": "1.0.0" }, { "type": "grafana", "id": "grafana", "name": "Grafana", "version": "12.3.0-pre" }, { "type": "datasource", "id": "grafana-mock-datasource", "name": "Mock", "version": "0.1.5" }, { "type": "panel", "id": "table", "name": "Table", "version": "" }, { "type": "panel", "id": "timeseries", "name": "Time series", "version": "" } ], "annotations": { "list": [ { "builtIn": 1, "datasource": { "type": "grafana", "uid": "-- Grafana --" }, "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "type": "dashboard" } ] }, "editable": true, "fiscalYearStartMonth": 0, "graphTooltip": 0, "links": [], "panels": [ { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 }, "id": 2, "panels": [], "title": "Time Series Conversion (Wide)", "type": "row" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "description": "Host A should have different numbers, seems to be bug in table display?", "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 9, "w": 11, "x": 0, "y": 1 }, "id": 5, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-wide", "meta": { "type": "timeseries-wide" }, "name": "New Frame", "source": "count" } ], "hide": false, "queryType": "frames_builder", "refId": "time_series_wide" } ], "title": "Time Series Wide (Before Conversion)", "type": "table" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 9, "w": 13, "x": 11, "y": 1 }, "id": 1, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-wide", "meta": { "type": "timeseries-wide" }, "name": "New Frame", "source": "count" } ], "hide": true, "queryType": "frames_builder", "refId": "time_series_wide" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\nFROM time_series_wide", "hide": false, "refId": "B", "type": "sql" } ], "title": "Time Series Wide Conversion To Full Long", "type": "table" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 10 }, "id": 4, "panels": [], "title": "Time Series Conversion (Multi)", "type": "row" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 11, "x": 0, "y": 11 }, "id": 6, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "Frame 1", "source": "count" }, { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "Frame 2", "source": "count" } ], "hide": false, "queryType": "frames_builder", "refId": "time_series_wide" } ], "title": "Time Series Multi (Before Conversion)", "type": "table" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 13, "x": 11, "y": 11 }, "id": 3, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "New Frame", "source": "count" }, { "count": 50, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "New Frame", "source": "count" } ], "hide": true, "queryType": "frames_builder", "refId": "time_series_wide" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\nFROM time_series_wide", "hide": false, "refId": "B", "type": "sql" } ], "title": "Time Series Multi Conversion To Full Long", "type": "table" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 18 }, "id": 7, "panels": [], "title": "Numeric Conversion (Wide)", "type": "row" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 6, "w": 11, "x": 0, "y": 19 }, "id": 9, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "3.14" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "93.14" } ], "frameInputType": "numeric-wide", "meta": { "type": "numeric-wide" }, "name": "New Frame", "source": "count" } ], "hide": false, "queryType": "frames_builder", "refId": "numeric_wide" } ], "title": "Numeric Wide (Before Conversion)", "type": "table" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 6, "w": 13, "x": 11, "y": 19 }, "id": 8, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=a,iface=x", "name": "cpu", "type": "nullable-float64", "value": "3.14" }, { "disabled": false, "labels": "host=b,iface=y", "name": "cpu", "type": "nullable-float64", "value": "93.14" } ], "frameInputType": "numeric-wide", "meta": { "type": "numeric-wide" }, "name": "New Frame", "source": "count" } ], "hide": true, "queryType": "frames_builder", "refId": "numeric_wide" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\nFROM numeric_wide", "hide": false, "refId": "B", "type": "sql" } ], "title": "Numeric Wide Conversion To Numeric Full Long", "type": "table" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 25 }, "id": 10, "panels": [], "title": "Numeric Conversion (Multi)", "type": "row" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 6, "w": 7, "x": 0, "y": 26 }, "id": 12, "options": { "cellHeight": "sm", "frameIndex": 0, "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=a", "name": "cpu", "type": "nullable-float64", "value": "3.14" } ], "frameInputType": "numeric-many", "meta": { "type": "numeric-multi" }, "name": "Frame 1", "source": "count" }, { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=b", "name": "cpu", "type": "nullable-float64", "value": "93.14" } ], "frameInputType": "numeric-many", "name": "Frame 2", "source": "count" } ], "hide": false, "queryType": "frames_builder", "refId": "numeric_multi" } ], "title": "Numeric Multi (No Conversion)", "type": "table" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "thresholds" }, "custom": { "align": "auto", "cellOptions": { "type": "auto" }, "footer": { "reducers": [] }, "inspect": false, "wrapHeaderText": false }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 6, "w": 9, "x": 7, "y": 26 }, "id": 11, "options": { "cellHeight": "sm", "showHeader": true }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=a", "name": "cpu", "type": "nullable-float64", "value": "3.14" } ], "frameInputType": "numeric-many", "meta": { "type": "numeric-multi" }, "name": "New Frame", "source": "count" }, { "count": 1, "disabled": false, "fields": [ { "disabled": false, "labels": "host=b", "name": "cpu", "type": "nullable-float64", "value": "93.14" } ], "frameInputType": "numeric-many", "name": "New Frame", "source": "count" } ], "hide": true, "queryType": "frames_builder", "refId": "numeric_multi" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\nFROM numeric_multi", "hide": false, "refId": "B", "type": "sql" } ], "title": "Numeric Multi Conversion To Numeric Full Long", "type": "table" }, { "collapsed": false, "gridPos": { "h": 1, "w": 24, "x": 0, "y": 32 }, "id": 14, "panels": [], "title": "Conversion Errors", "type": "row" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 33 }, "id": 13, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "a=b", "name": "value", "type": "nullable-float64", "value": "1,2,3,2" } ], "frameInputType": "timeseries", "name": "New Frame", "source": "count" } ], "queryType": "frames_builder", "refId": "A" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\n FROM A\n LIMIT 10", "hide": false, "refId": "B", "type": "sql" } ], "title": "No Type & Response has Labels", "type": "timeseries" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 33 }, "id": 15, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" } ], "frameInputType": "timeseries-multi", "meta": { "type": "histogram" }, "name": "New Frame", "source": "count" }, { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=b", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "New Frame", "source": "count" } ], "generatorSource": "frontend", "queryType": "raw_frame", "rawFrameContent": "[\n {\n \"schema\": {\n \"meta\": {\n \"type\": \"\",\n \"typeVersion\": [0, 1],\n \"custom\": {\n \"exampleInfo\": {\n \"summary\": \"TimeseriesMulti with 2 items. There is 1 item name (slothCount) and 1 dimension (city). The timestamps are not aligned between the two series.\",\n \"itemCount\": 2,\n \"collectionVersion\": 1,\n \"valid\": true,\n \"noData\": false\n }\n }\n },\n \"fields\": [\n {\n \"name\": \"t\",\n \"type\": \"time\",\n \"typeInfo\": {\n \"frame\": \"time.Time\"\n }\n },\n {\n \"type\": \"number\",\n \"name\": \"slothCount\",\n \"typeInfo\": {\n \"frame\": \"float64\"\n },\n \"labels\": {\n \"city\": \"LGA\"\n }\n }\n ]\n },\n \"data\": {\n \"values\": [\n [1664901845976, 1664902845976],\n [3, 5]\n ]\n }\n },\n {\n \"schema\": {\n \"meta\": {\n \"type\": \"timeseries-multi\",\n \"typeVersion\": [0, 1],\n \"custom\": {\n \"exampleInfo\": {\n \"valid\": true,\n \"noData\": false\n }\n }\n },\n \"fields\": [\n {\n \"name\": \"t\",\n \"type\": \"time\",\n \"typeInfo\": {\n \"frame\": \"time.Time\"\n }\n },\n {\n \"type\": \"number\",\n \"name\": \"slothCount\",\n \"typeInfo\": {\n \"frame\": \"float64\"\n },\n \"labels\": {\n \"city\": \"MIA\"\n }\n }\n ]\n },\n \"data\": {\n \"values\": [\n [1664901855976, 1664902455976, 1664902855976],\n [6, 7, 9]\n ]\n }\n }\n]", "rawFrameSource": "inline", "refId": "A", "resultState": "success", "scenario": "no-data" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\n FROM A\n LIMIT 10", "hide": false, "refId": "B", "type": "sql" } ], "title": "Multiple Frames No Type", "type": "timeseries" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 41 }, "id": 16, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "a=b", "name": "value", "type": "nullable-float64", "value": "1,2,3,2" } ], "frameInputType": "timeseries", "meta": { "type": "slothArt" }, "name": "New Frame", "source": "count" } ], "queryType": "frames_builder", "refId": "A" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\n FROM A\n LIMIT 10", "hide": false, "refId": "B", "type": "sql" } ], "title": "Unsupported Type & Response has Labels", "type": "timeseries" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 8, "x": 8, "y": 41 }, "id": 17, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "string", "value": "a,b,c,d" }, { "disabled": false, "labels": "a=b", "name": "value", "type": "nullable-float64", "value": "1,2,3,2" } ], "frameInputType": "timeseries", "meta": { "type": "timeseries-wide" }, "name": "New Frame", "source": "count" } ], "queryType": "frames_builder", "refId": "A" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\n FROM A\n LIMIT 10", "hide": false, "refId": "B", "type": "sql" } ], "title": "Type Time but no Time Field", "type": "timeseries" }, { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "fieldConfig": { "defaults": { "color": { "mode": "palette-classic" }, "custom": { "axisBorderShow": false, "axisCenteredZero": false, "axisColorMode": "text", "axisLabel": "", "axisPlacement": "auto", "barAlignment": 0, "barWidthFactor": 0.6, "drawStyle": "line", "fillOpacity": 0, "gradientMode": "none", "hideFrom": { "legend": false, "tooltip": false, "viz": false }, "insertNulls": false, "lineInterpolation": "linear", "lineWidth": 1, "pointSize": 5, "scaleDistribution": { "type": "linear" }, "showPoints": "auto", "showValues": false, "spanNulls": false, "stacking": { "group": "A", "mode": "none" }, "thresholdsStyle": { "mode": "off" } }, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": 0 }, { "color": "red", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 8, "w": 8, "x": 0, "y": 49 }, "id": 18, "options": { "legend": { "calcs": [], "displayMode": "list", "placement": "bottom", "showLegend": true }, "tooltip": { "hideZeros": false, "mode": "single", "sort": "none" } }, "pluginVersion": "12.3.0-pre", "targets": [ { "datasource": { "type": "grafana-mock-datasource", "uid": "${DS_MOCK}" }, "frames": [ { "count": 60, "disabled": false, "fields": [ { "disabled": false, "name": "timestamp", "type": "timestamp-auto" }, { "disabled": false, "labels": "host=a", "name": "cpu", "type": "nullable-float64", "value": "1,4,2,3" } ], "frameInputType": "timeseries-multi", "meta": { "type": "timeseries-multi" }, "name": "New Frame", "source": "count" }, { "count": 60, "disabled": false, "fields": [ { "disabled": false, "labels": "host=b", "name": "cpu", "type": "nullable-float64", "value": "6,8,5,9" } ], "frameInputType": "timeseries-multi", "meta": { "type": "numeric-multi" }, "name": "New Frame", "source": "count" } ], "queryType": "frames_builder", "refId": "A" }, { "datasource": { "type": "__expr__", "uid": "${DS_EXPRESSION}" }, "expression": "SELECT *\n FROM A\n LIMIT 10", "hide": false, "refId": "B", "type": "sql" } ], "title": "Type Time but no Time Field", "type": "timeseries" } ], "schemaVersion": 42, "tags": [], "templating": { "list": [] }, "time": { "from": "now-1m", "to": "now" }, "timepicker": {}, "timezone": "utc", "title": "SQL Expression \"Metrics\" Conversion", "uid": "d56e24c4-ffab-4604-a6db-94782a5bae88", "version": 10, "weekStart": "", "id": null } -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 13 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -94,4 +94,16 @@ You should get an error like: This is not currently supported, SQL expressions will treat the first response as the type of all the frames, and you will like get an odd error that the additional type doesn't have what the first type needs. Or it will work and the results will be odd. TODO: Should add an error in the case, or discard anything past the first type. ## Do instrument any Datasource specific metrics in relation to SQL expression support? Yes: ```promql sum(rate(grafana_sse_sql_command_input_count[$__rate_interval])) by (status,attempted_conversion,datasource_type,input_frame_type) ``` ## What does the conversion look like? <img width="1385" height="1223" alt="image" src="https://gist.github.com/user-attachments/assets/92e5c227-1045-4725-8259-0f91962c6ac0" /> -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 50 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -25,7 +25,7 @@ In the case of `full-long` they are nullable string fields. When the value on a The numeric kind is for when each item in the response as a value. The numeric kind in format full long format is identified by the data frame type `numeric-full-long`. ### Numeric Full Long Examples #### Single Metric Example @@ -91,6 +91,55 @@ The timeseries kind shares the same properties as the numeric kind for the full The main difference is that there must be a time column the is is sorted by time in ascending order (old to new). When this type is produced by converting from the `multi` or `wide` formats, the name of the field (column) is kept. ### Timeseries Full Long Examples #### Example 1 <table> <tr> <td><strong>time</strong></td> <td><strong>__value__</strong></td> <td><strong>__metric_name__</strong></td> <td><strong>host</strong></td> <td><strong>iface</strong></td> </tr> <tr> <td>2025-10-02 15:55:41.000</td> <td>1</td> <td>cpu</td> <td>a</td> <td>x</td> </tr> <tr> <td>2025-10-02 15:55:41.000</td> <td>6</td> <td>cpu</td> <td>b</td> <td>y</td> </tr> <tr> <td>2025-10-02 15:55:42.000</td> <td>4</td> <td>cpu</td> <td>a</td> <td>x</td> </tr> <tr> <td>2025-10-02 15:55:42.000</td> <td>8</td> <td>cpu</td> <td>b</td> <td>y</td> </tr> <tr> <td>2025-10-02 15:55:43.000</td> <td>2</td> <td>cpu</td> <td>a</td> <td>x</td> </tr> </table> ## How Full Long differs from the Long format The long `long` format is partially flatted, where as the `full-long` format is _fully_ flatted. @@ -115,7 +164,6 @@ In `long`, the numeric type for the value is persevered. For example a metric th - The field.Name from `wide`/`multi` gets flatted into the `__metric_name__` field. - If the `timeseries` kind, the first field of type `time` gets flatted and keeps the name ## Use in SQL expressions When SQL expressions receives the data in the kinds of `timeseries` or `numeric` in the `wide` or `multi` formats, it converts it to the corresponding full long format. This is because there is no notion of labels in SQL, so the data must be flatted into tabular format. -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 25 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -15,7 +15,7 @@ For tabular data responses SQL expressions should work out of the box, so the on ## Labeled Metric Time Data (Timeseries or Numeric) For labeled metric data, SQL Expressions detects it based on the `Frame.Meta.Type` property of the data frame. This property holds the the dataplane type. The supported types are: @@ -71,3 +71,27 @@ In a dashboard, and for each type for response type your query offers: - Add Expression -> Type SQL - The default should query should be `SELECT * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) # FAQ / Other ## What happens my DS sends labeled data without a frame type and Select from it in a SQL expressions? You should get an error like: ```text [sse.sql.input_conversion] failed to convert the results of query [A] (Datasource Type: [grafana-mock-datasource]) into a SQL/Tabular format for sql expression [B]: can not convert because the response is missing the data type (frame.meta.type) and has labels in the response that can not be mapped to a table ``` ## What happens if DS returns with multiple frames without a frame type? You should get an error like: ```text [sse.sql.input_conversion] failed to convert the results of query [A] (Datasource Type: [grafana-mock-datasource]) into a SQL/Tabular format for sql expression [B]: can not convert because the response is missing the data type (frame.meta.type) and has more than one dataframe that can not be automatically mapped to a single table ``` ## What happens if DS returns with multiple frames without different types? This is not currently supported, SQL expressions will treat the first response as the type of all the frames, and you will like get an odd error that the additional type doesn't have what the first type needs. Or it will work and the results will be odd. TODO: Should add an error in the case, or discard anything past the first type. -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 6 additions and 2 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -89,7 +89,7 @@ The numeric kind is for when each item in the response as a value. The numeric k The timeseries kind shares the same properties as the numeric kind for the full long format in terms of reserved columns and the behavior of dimensions. The timeseries kind in format full long format is identified by the data frame type `timeseries_full-long`. The main difference is that there must be a time column the is is sorted by time in ascending order (old to new). When this type is produced by converting from the `multi` or `wide` formats, the name of the field (column) is kept. ## How Full Long differs from the Long format @@ -99,7 +99,9 @@ In the `full-long` formats, the metric name is treated as another dimension. Thi In the `long` and each unique metric name gets a field (column). The field's Name property is the metric name. Like `long` the dimensions that would be labels in the `wide` and `multi` formats get flatted. But in `long` the metric name is _not treated as a dimension_ and does not get flatted, whereas it does in `full-long` This format also uses nullable strings for dimensions. This means that if the value is `null` for the row, where converting to labels, that label should be considered not present. This allows `full-long` to avoid creating label sets that didn't exist during a round trip conversion from `wide` or `multi`. In the case of the `timeseries` kind, with roundtrip conversion (`multi` or `wide` to `long`, and then back) `long` can create false datapoints. Since `long` does not flatten the metric name portion, there was no way to represent timestamped data points that existed for one set of series sharing a name but not the other set of series. Since `full-long` is fully flattened into `__value__` and `__metric_name__`, those missing points don't get falsely created like to with full long with doing roundtrip conversion. In `long`, the numeric type for the value is persevered. For example a metric that is a unit64 will stay that way. Because `full-long` flattens to a a single column (`__value__`), all numeric values become nullable float64 values. In most cases, upstream we effectively convert to something like a float64 (or JS float), so this likely doesn't matter much in current practice. @@ -111,6 +113,8 @@ In `long`, the numeric type for the value is persevered. For example a metric th - Each label key present becomes a field (column) with the name of the label key - All the numeric value columns are combined and flatted into one `__value__` field. Whatever the numeric type is, it will be converted to a nullable `*float64` field. - The field.Name from `wide`/`multi` gets flatted into the `__metric_name__` field. - If the `timeseries` kind, the first field of type `time` gets flatted and keeps the name ## Use in SQL expressions -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 8 additions and 5 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -6,6 +6,8 @@ So the use case for this format is when you need to be able to convert between l ## Properties shared by Full Long Formats They are single frame formats. There are three reserved fields (columns): - `__metric_name__`: The name of the metric @@ -17,11 +19,11 @@ There are three reserved fields (columns): Like `long` any dimensions (that would be labels in in `wide` or `multi`) become their own field. The dimension fields name corresponds to key of the dimensions, and the values values are in the rows of that field. In the case of `full-long` they are nullable string fields. When the value on a row is null, the label is considered absent. This helps with more correct conversion to and from the `wide` and `multi` formats because of all of the items do not have the same set of label keys. ## Numeric Full Long The numeric kind is for when each item in the response as a value. The numeric kind in format full long format is identified by the data frame type `numeric-full-long`. ### Examples @@ -85,7 +87,9 @@ The numeric kind is for when each item in the response as a value. The numeric k ## Timeseries Full Long The timeseries kind shares the same properties as the numeric kind for the full long format in terms of reserved columns and the behavior of dimensions. The timeseries kind in format full long format is identified by the data frame type `timeseries_full-long`. The main difference is that there must be a time column the is is sorted by time in ascending order (old to new). ## How Full Long differs from the Long format @@ -101,14 +105,13 @@ In `long`, the numeric type for the value is persevered. For example a metric th ## Conversion between `wide` and `multi` formats ### From `wide`/`multi` to `full-long` - Fundamentally we are flattening a column-oriented structure with labels into row-oriented structure without labels. - Each label key present becomes a field (column) with the name of the label key - All the numeric value columns are combined and flatted into one `__value__` field. Whatever the numeric type is, it will be converted to a nullable `*float64` field. - The field.Name from `wide`/`multi` gets flatted into the `__metric_name__` field. ## Use in SQL expressions When SQL expressions receives the data in the kinds of `timeseries` or `numeric` in the `wide` or `multi` formats, it converts it to the corresponding full long format. This is because there is no notion of labels in SQL, so the data must be flatted into tabular format. -
kylebrandt revised this gist
Oct 2, 2025 . 1 changed file with 10 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -21,6 +21,8 @@ In the case of `full_long` they are nullable string fields. When the value on a ## Numeric Full Long The numeric kind is for when each item in the response as a value. The numeric kind in format full long format is identified by the data frame type `numeric_full_long`. ### Examples #### Single Metric Example @@ -81,7 +83,11 @@ In the case of `full_long` they are nullable string fields. When the value on a </tr> </table> ## Timeseries Full Long The timeseries kind shares the same properties as the numeric kind for the full long format in terms of reserved columns and the behavior of dimensions. ## How Full Long differs from the Long format The long `long` format is partially flatted, where as the `full-long` format is _fully_ flatted. @@ -93,14 +99,14 @@ This format also uses nullable strings for dimensions. This means that if the va In `long`, the numeric type for the value is persevered. For example a metric that is a unit64 will stay that way. Because `full-long` flattens to a a single column (`__value__`), all numeric values become nullable float64 values. In most cases, upstream we effectively convert to something like a float64 (or JS float), so this likely doesn't matter much in current practice. ## Conversion between `wide` and `multi` formats ### From `wide`/`multi` to `full_long` - Fundamentally we are flattening a column-oriented structure with labels into row-oriented structure without labels. - Each label key present becomes a field (column) with the name of the label key - All the numeric value columns are combined and flatted into one `__value__` field. Whatever the numeric type is, it will be converted to a nullable `*float64` field. - The field.Name from `wide`/`multi` gets flatted into the `__metric_name__` field. ## Use in SQL expressions -
kylebrandt revised this gist
Oct 1, 2025 . 1 changed file with 15 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,4 +1,3 @@ # Full Long Formats Two new data types have been introduced with SQL expressions. `timeseries-full-long` and `numeric-full-long`. This is a row-oriented tabular format similar to `long` (labels are not used), but structured in a way that allows more lossless conversion of information between the labeled (`wide` and `multi`) formats. @@ -10,7 +9,7 @@ So the use case for this format is when you need to be able to convert between l There are three reserved fields (columns): - `__metric_name__`: The name of the metric - `__value__`: A nullable `*float64` numeric field, where this is the value that that represents combination of the metric name and dimensions for the row. - `__display_name__`: (Optional) When converting from other formats, if the `DisplayNameFromDS` property is set, it is flatted into this field - Note: Additional reserved names may be added in the same naming style. This would be to flatten other column metadata such as a data links or units. @@ -84,14 +83,26 @@ In the case of `full_long` they are nullable string fields. When the value on a ## How it differs from the Long format The long `long` format is partially flatted, where as the `full-long` format is _fully_ flatted. In the `full-long` formats, the metric name is treated as another dimension. This creates two reserved fields for this purpose in full long: `__value__` and `__metric_name__`. In the `long` and each unique metric name gets a field (column). The field's Name property is the metric name. Like `long` the dimensions that would be labels in the `wide` and `multi` formats get flatted. But in `long` the metric name is _not treated as a dimension_ and does not get flatted, whereas it does in `full-long` This format also uses nullable strings for dimensions. This means that if the value is `null` for the row, where converting to labels, that label should be considered not present. In `long`, the numeric type for the value is persevered. For example a metric that is a unit64 will stay that way. Because `full-long` flattens to a a single column (`__value__`), all numeric values become nullable float64 values. In most cases, upstream we effectively convert to something like a float64 (or JS float), so this likely doesn't matter much in current practice. ## Conversion to and from `wide` and `multi` formats ### From `wide`/`multi` to `full_long` - Fundamentally we are flattening a column-oriented structure with labels into row-oriented structure without labels. - Each label key present becomes a field (column) with the name of the label key - All the numeric value columns are combined into one `__value__` field. Whatever the numeric type is, it will be converted to a nullable `*float64` field. - ## Use in SQL expressions When SQL expressions receives the data in the kinds of `timeseries` or `numeric` in the `wide` or `multi` formats, it converts it to the corresponding full long format. This is because there is no notion of labels in SQL, so the data must be flatted into tabular format. -
kylebrandt renamed this gist
Oct 1, 2025 . 1 changed file with 0 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -58,7 +58,6 @@ In the case of `full_long` they are nullable string fields. When the value on a </tr> <tr> <td>cpu_load</td> <td>0.82</td> <td>a</td> <td>us-east-1</td> -
kylebrandt revised this gist
Oct 1, 2025 . 1 changed file with 12 additions and 7 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,19 +1,24 @@ # Full Long Formats Two new data types have been introduced with SQL expressions. `timeseries-full-long` and `numeric-full-long`. This is a row-oriented tabular format similar to `long` (labels are not used), but structured in a way that allows more lossless conversion of information between the labeled (`wide` and `multi`) formats. So the use case for this format is when you need to be able to convert between labeled and tabular formats with more preservation of the information. Whereas, when converting back and forth with `Long` can create artifacts. ## Properties shared by Full Long Formats There are three reserved fields (columns): - `__metric_name__`: The name of the metric - `__value__`: A nullable float64 numeric field, where this is the value that that represents combination of the metric name and dimensions for the row. - `__display_name__`: (Optional) When converting from other formats, if the `DisplayNameFromDS` property is set, it is flatted into this field - Note: Additional reserved names may be added in the same naming style. This would be to flatten other column metadata such as a data links or units. ### Dimension Columns Like `long` any dimensions (that would be labels in in `wide` or `multi`) become their own field. The dimension fields name corresponds to key of the dimensions, and the values values are in the rows of that field. In the case of `full_long` they are nullable string fields. When the value on a row is null, the label is considered absent. This helps with more correct conversion to and from the `wide` and `multi` formats because of all of the items do not have the same set of label keys. ## Numeric Full Long @@ -53,6 +58,7 @@ Like `long` any dimensions (that would be labels in in `wide` or `multi`) become </tr> <tr> <td>cpu_load</td> <td>0.82</td> <td>a</td> <td>us-east-1</td> @@ -90,4 +96,3 @@ In `long`, the numeric type for the value is persevered. For example a metric th ## Use in SQL expressions When SQL expressions receives the data in the kinds of `timeseries` or `numeric` in the `wide` or `multi` formats, it converts it to the corresponding full long format. This is because there is no notion of labels in SQL, so the data must be flatted into tabular format. -
kylebrandt revised this gist
Sep 30, 2025 . 1 changed file with 15 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -47,6 +47,20 @@ So if your DS has metric data that matches one of those data types, it should wo ] ``` In go code: ```go import ( "github.com/grafana/grafana-plugin-sdk-go/data" ) func main() { frame := data.NewFrame("") // ... add data and fields the create "NumericWide" type. frame.Meta = &data.FrameMeta{Type: data.FrameTypeNumericWide} } ``` When SQL expressions receives labeled metric data, it will convert (flatten) the data into the `full-long` format for the corresponding kind (`timeseries` or `numeric`). This happens once the data source query is selected by RefID from a SQL expression. This is because SQL has no notion of labels. ## Manual Testing @@ -56,4 +70,4 @@ In a dashboard, and for each type for response type your query offers: - Add your data source query - Add Expression -> Type SQL - The default should query should be `SELECT * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) -
kylebrandt revised this gist
Sep 30, 2025 . 2 changed files with 68 additions and 69 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -0,0 +1,59 @@ # What does a data source need to do to support SQL Expressions? It must be a backend data source since SQL expressions is server side. SQL Expressions supports two general categories of responses: - **Tabular data**: A single dataframe, with no labels on any of the fields (columns). In short, a dataframe that can be directly mapped to a SQL table. - **Labeled Metric Time Data (Timeseries or Numeric)**: Data that meets the [Dataplane](https://grafana.com/developers/dataplane) spec and has the type `Frame.Meta.Type` property set. Therefore, data source support is really per query type or response type within a data source. So it is often not all or nothing in terms of data source support. ## Tabular Data For tabular data responses SQL expressions should work out of the box, so the only thing to do is test out a basic select query. ## Labeled Metric Time Data (Timeseries or Numeric) For labeled metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` property of the data frame. This property holds the the dataplane type. The supported types are: - `timeseries-multi` - `timeseries-wide` - `numeric-multi` - `numeric-wide` (Note: the `timeseries-long` and `timeseries-multi` fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` (serialized into json as `schema.meta.type`) is set to that data type, for example: ```jsonc [ { "schema": { "meta": { "type": "numeric-wide", // SQL expressions need this property set for labeled metric data "typeVersion": [0, 1], // optional for SQL Expressions (so can be default 0.0) // TypeVersion > 0.0 should make other SSE operations more deterministic, // but if not a new DS, safest path is to do that as a separate task. // ... } }, "fields": [ // ... ] } ] ``` When SQL expressions receives labeled metric data, it will convert (flatten) the data into the `full-long` format for the corresponding kind (`timeseries` or `numeric`). This happens once the data source query is selected by RefID from a SQL expression. This is because SQL has no notion of labels. ## Manual Testing In a dashboard, and for each type for response type your query offers: - Add your data source query - Add Expression -> Type SQL - The default should query should be `SELECT * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -1,85 +1,25 @@ # Full Long Formats Two new data types have been introduces with SQL expressions. `timeseries-full-long` and `numeric-full-long`. This is a row-oriented tabular format similar to long (labels are not used), but structured in a way that allows more lossless conversion of information between the labeled (`wide` and `multi`) formats. So the use case for this format is when you need to be able to convert between labeled and tabular formats with more preservation of the information when converting back and forth. Whereas `Long` can create artifacts when converting. ## Properties shared by Full Long Formats There are three reserved fields (columns): - `__metric_name__`: The name of the metric - `__value__`: A nullable float64 numeric column, where this is the value that that represents combination of the metric name and dimensions for the row. - `__display_name__`: (Optional) When converting from other formats, if the `DisplayNameFromDS` property is set, it is flatted into this column - Note: Additional reserved names may be added in the same naming style. This would be to flatten other column metadata such as a data links or units. ### Dimension Columns Like `long` any dimensions (that would be labels in in `wide` or `multi`) become their own field (column) for . They become nullable string fields where the name of column correspond to a label/dimension key, and values are in the rows of that field. The the case of `full_long` they are nullable string fields. When the value on a row is null, the label is considered absent. ## Numeric Full Long ### Examples #### Single Metric Example <table> <tr> @@ -102,7 +42,7 @@ Single Metric Example: </tr> </table> #### Multi-Metric Example <table> <tr> @@ -137,7 +77,7 @@ Multi-Metric Example: </tr> </table> ## How it differs from the Long format In the long format each dimension gets a field (column) and each unique metric name gets a field as well. The Field's Name property is the metric name. Therefore the dimensions that would be labels in the `wide` and `multi` formats get flatted, but the metric name is not treated as a dimension and does not get flatted. @@ -147,7 +87,7 @@ This format also uses nullable strings for dimensions. This means that if the va In `long`, the numeric type for the value is persevered. For example a metric that is a unit64 will stay that way. Because `full-long` flattens to a a single column (`__value__`), all numeric values become nullable float64 values. In most cases, upstream we effectively convert to something like a float64 (or JS float), so this likely doesn't matter much in current practice. ## Use in SQL expressions When SQL expressions receives the data in the kinds of `timeseries` or `numeric` in the `wide` or `multi` formats, it converts it to the corresponding full long format. This is because there is no notion of labels in SQL, so the data must be flatted into tabular format. -
kylebrandt revised this gist
Sep 30, 2025 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -48,7 +48,7 @@ So if your DS has metric data that matches one of those data types, it should wo ] ``` When SQL expressions receives labeled metric data, it will convert (flatten) the data into the `full-long` format for the corresponding kind (`timeseries` or `numeric`). This happens once the data source query is selected by RefID from a SQL expression. This is because SQL has no notion of labels. ### Manual Testing -
kylebrandt revised this gist
Sep 30, 2025 . 1 changed file with 23 additions and 9 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,18 +5,29 @@ It must be a backend data source since SQL expressions is server side. SQL Expressions supports two general categories of responses: - **Tabular data**: A single dataframe, with no labels on any of the fields (columns). In short, a dataframe that can be directly mapped to a SQL table. - **Labeled Metric Time Data (Timeseries or Numeric)**: Data that meets the [Dataplane](https://grafana.com/developers/dataplane) spec and has the type `Frame.Meta.Type` property set. Therefore, data source support is really per query type or response type within a data source. So it is often not all or nothing in terms of data source support. ### Tabular Data For tabular data responses SQL expressions should work out of the box, so the only thing to do is test out a basic select query. ### Labeled Metric Time Data (Timeseries or Numeric) For labeled metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` property of the data frame. This property holds the the dataplane type. The supported types are: - `timeseries-multi` - `timeseries-wide` - `numeric-multi` - `numeric-wide` (Note: the `timeseries-long` and `timeseries-multi` fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` (serialized into json as `schema.meta.type`) is set to that data type, for example: ```jsonc [ @@ -37,13 +48,16 @@ For labeled metric data, SQL Expressions detects metrics based on the `Frame.Met ] ``` When SQL expressions receives labeled metric data, it will convert (flatten) the data into the `full-long` format for the corresponding kind (`timeseries` or `numeric`). This happens once the data source query is selected by RefID from a SQL expression. ### Manual Testing In a dashboard, and for each type for response type your query offers: - Add your data source query - Add Expression -> Type SQL - The default should query should be `SELECT * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) ## Full Long Formats -
kylebrandt revised this gist
Sep 30, 2025 . 1 changed file with 4 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -10,8 +10,12 @@ SQL Expressions supports two general categories of responses: Therefore, data source support is really per query type or response type within a data source. So it is often not all or nothing in terms of support. ### Tabular Data For tabular data responses SQL expressions should work out of the box, so the only thing to do is test out a basic select query. ### Metric Data For labeled metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` dataplane type only (not the TypeVersion.) The supported types are `timeseries-multi`, `timeseries-wide`, `numeric-multi`, or `numeric-wide`. (Long types fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` is set to that data type, for example: ```jsonc -
kylebrandt revised this gist
Sep 30, 2025 . 1 changed file with 11 additions and 10 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,23 +5,24 @@ It must be a backend data source since SQL expressions is server side. SQL Expressions supports two general categories of responses: - Tabular data: A single dataframe, with no labels on any of the fields (columns). In short, a dataframe that can be directly mapped to a SQL table. - [Dataplane](https://grafana.com/developers/dataplane) Labeled Metric Data (time series or numeric data kinds). Therefore, data source support is really per query type or response type within a data source. So it is often not all or nothing in terms of support. For tabular data responses SQL expressions should work out of the box, so the only thing to do is test out a basic select query. For labeled metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` dataplane type only (not the TypeVersion.) The supported types are `timeseries-multi`, `timeseries-wide`, `numeric-multi`, or `numeric-wide`. (Long types fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` is set to that data type, for example: ```jsonc [ { "schema": { "meta": { "type": "numeric-wide", // SQL expressions need this property set for labeled metric data "typeVersion": [0, 1], // optional for SQL Expressions (so can be default 0.0) // TypeVersion > 0.0 should make other SSE operations more deterministic, // but if not a new DS, safest path is to do that as a separate task. // ... } }, @@ -35,9 +36,9 @@ For metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` ### Manual Testing In a dashboard, and for each type for response type your query offers: - Add your data source query - Add Expression -> Type SQL - The default should query should be `SELECT * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) ## Full Long Formats -
kylebrandt revised this gist
Sep 29, 2025 . No changes.There are no files selected for viewing
-
kylebrandt revised this gist
Sep 29, 2025 . 1 changed file with 19 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -12,7 +12,25 @@ Therefore, Data source support is really per query type or response type. So it For tabular data it should just work, so the only thing to do is test out a basic select query. For metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` dataplane type only (not the TypeVersion.) That can be `timeseries-multi`, `timeseries-wide`, `numeric-multi`, or `numeric-wide`. (Long types should fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` is set to that data type, for example: ```jsonc [ { "schema": { "meta": { "type": "numeric-wide", // Need this set for metric data "typeVersion": [0, 1], // optional for SQL Expressions // TypeVersion should make other SSE operations more deterministic, but if not a new DS, safest path is to do it in separate PR or flag // ... } }, "fields": [ // ... ] } ] ``` ### Manual Testing -
kylebrandt revised this gist
Sep 29, 2025 . 1 changed file with 4 additions and 4 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -5,14 +5,14 @@ It must be a backend data source since SQL expressions is server side. SQL Expressions supports two general categories of responses: - Tabular data (A single Frame, with no labels on any of the columns). In short, a Dataframe that can be directly mapped to a SQL table. - [Dataplane](https://grafana.com/developers/dataplane) Labeled Metric Data (time series or numeric data kinds). Therefore, Data source support is really per query type or response type. So it is often not all or nothing in terms of support. For tabular data it should just work, so the only thing to do is test out a basic select query. For metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` dataplane type only (not the TypeVersion.) That can be `timeseries-multi`, `timeseries-wide`, `numeric-multi`, or `numeric-wide`. (Long types should fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` is set to that data type. ### Manual Testing -
kylebrandt revised this gist
Sep 29, 2025 . 1 changed file with 1 addition and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -18,7 +18,7 @@ For metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` In a dashboard, and for each type for response type your query offers: - Add your DS query - Add Expression, Type SQL - The default should be `select * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) -
kylebrandt revised this gist
Sep 29, 2025 . 1 changed file with 10 additions and 0 deletions.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -8,10 +8,20 @@ SQL Expressions supports two general categories of responses: - Tabular data (A single Frame, with no labels on any of the columns) - Dataplane Metric Data (For labaled time series or numeric data) https://grafana.com/developers/dataplane Therefore, support is really per query type or response type in DSes, and is often not all or nothing. For tabular data it should just work, so the only thing to do is test out a basic select query. For metric data, SQL Expressions detects metrics based on the `Frame.Meta.Type` dataplane type only (not the version.) That can be `timeseries-multi`, `timeseries-wide`, `numeric-multi`, or `numeric-wide`. (Long types should fall into the tabular category). So if your DS has metric data that matches one of those data types, it should work as long as `Frame.Meta.Type` is set to that data type. ### Manual Testing In a dashboard, and for each type for response type your query offers: - Add your DS query - Add Expression SQL - The default should be `select * from A LIMIT 10` (assuming your query is RefID `A`). If this works on a few variations of that query type, it should be compatible with SQL expressions. - If it doesn't work, see section above, if you are sending metadata and the datatypes mentioned above (or tabular data, and it isn't working, there may be an issue with SQL expressions) ## Full Long Formats Two new data types have been introduces with SQL expressions. `timeseries-full-long` and `numeric-full-long`. This is a row-oriented tabular format similar to long (labels are not used), but structured in a way that allows more lossless conversion of information between the labeled (`wide` and `multi`) formats. So the use case for this format is when you need to be able to convert between labeled and tabular formats with more preservation of the information when converting back and forth. Whereas `Long` can create artifacts when converting. -
kylebrandt revised this gist
Sep 29, 2025 . 1 changed file with 63 additions and 1 deletion.There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersOriginal file line number Diff line number Diff line change @@ -24,10 +24,72 @@ There are three reserved fields (columns): - `__display_name__`: (Optional) When converting from other formats, if the `DisplayNameFromDS` property is set, it is flatted into this column - Note: Additional reserved names may be added in the same naming style. This would be to flatten other column metadata such as a data links or units. #### Dimension Columns Like `long` any dimensions (that would be labels in in `wide` or `multi`) become their own field (column) for . They become nullable string fields where the name of column correspond to a label/dimension key, and values are in the rows of that field. The the case of `full_long` they are nullable string fields. When the value on a row is null, the label is considered absent. ### Numeric Full Long #### Example Single Metric Example: <table> <tr> <td><strong>__metric_name__</strong></td> <td><strong>__value__</strong></td> <td><strong>host</strong></td> <td><strong>region</strong></td> </tr> <tr> <td>cpu_load</td> <td>0.82</td> <td>a</td> <td>us-east-1</td> </tr> <tr> <td>cpu_load</td> <td>0.61</td> <td>b</td> <td>us-east-1</td> </tr> </table> Multi-Metric Example: <table> <tr> <td><strong>__metric_name__</strong></td> <td><strong>__value__</strong></td> <td><strong>host</strong></td> <td><strong>region</strong></td> </tr> <tr> <td>cpu_load</td> <td>0.82</td> <td>a</td> <td>us-east-1</td> </tr> <tr> <td>cpu_load</td> <td>0.61</td> <td>b</td> <td><em>null</em></td> </tr> <tr> <td>disk_free_bytes</td> <td>1.23e+12</td> <td>a</td> <td>us-east-1</td> </tr> <tr> <td>disk_free_bytes</td> <td>9.80e+11</td> <td>b</td> <td>us-east-1</td> </tr> </table> ### How it differs from the Long format In the long format each dimension gets a field (column) and each unique metric name gets a field as well. The Field's Name property is the metric name. Therefore the dimensions that would be labels in the `wide` and `multi` formats get flatted, but the metric name is not treated as a dimension and does not get flatted.
NewerOlder