@datalust/seqcli-osx-arm64 2026.1.2638-dev-02638

package/Skills/seq-search-and-query/SKILL.md

@@ -0,0 +1,360 @@
+---
+name: seq-search-and-query
+description: Search and query logs and spans in Seq. Use when interacting with Seq.
+license: Apache-2.0
+metadata:
+ author: Datalust and Contributors
+---
+
+Seq is a storage service for log and trace telemetry. Search Seq to retrieve matching log events and spans. Query Seq to
+compute tabular, aggregate results from log events and spans.
+
+Seq's query language is **not SQL** — assuming SQL semantics will produce errors. Do NOT rely on prior knowledge of 
+Seq's query language. **Always** use the grammar, rules, and examples in this document as a basis for Seq queries and
+searches.
+
+## Starting a Diagnostic Session
+
+Being "confidently wrong" is the most common and worst failure mode when working with diagnostic data. EVERY diagnostic
+session MUST begin with the following steps:
+
+1. Check for relevant signals. Many event filtering problems have already been solved and the resulting filters saved as efficiently indexed signals.
+2. Retrieve a sample of relevant events. Use signals identified in (1) where appropriate, event search and query supports them.
+3. Proactively confirm the schema of the search results using appropriate tools.
+4. Inspect a subset of relevant events in full.
+5. Determine how the correctness of any conclusions can be verified using real diagnostic data.
+
+DO NOT skip steps just because early results suggest a quicker path to a solution: this is the path to being confidently wrong!
+
+## Event Data Model
+
+All events stored in Seq use the same data model. Spans are distinguished from log events by the presence of the
+`@Start` property. The following built-in properties are supported.
+
+| Built in property name | Type | Description |
+|---|---|---|
+| `@Arrived` | `number` | An integer indicating the order in which the event arrived at the Seq server relative to other events in the same batch. |
+| `@Data` | `object` | A compact internal representation of the event as a single structured object. High runtime cost, should be avoided when possible. |
+| `@Elapsed` | `number?` | The elapsed duration of a span, expressed in 100 nanosecond ticks. In the same domain as Seq's duration literals such as `1s`, `23ms`, or `3d`. Only present on spans, not present on log events. |
+| `@EventType` | `number` | A numeric hash of the message template that was used to generate the event. The message template itself is in the `@MessageTemplate` property. |
+| `@Exception` | `string?` | The exception associated with the event if any, as a string. This normally incorporates the exception type, message, and stack trace. |
+| `@Id` | `string` | The event's unique id in Seq. |
+| `@Level` | `string` | The severity of a log event, or completion status of a span. Values are source-dependent, so for example `'Error'`, `'ERROR'`, and `'err'` would all be typical values. |
+| `@Message` | `string` | Human-readable text associated with the event. Often the result of substituting `@Properties` values into `@MessageTemplate`. For spans, this property carries the span name. |
+| `@MessageTemplate` | `string` | A message template, following the `messagetemplates.org` syntax. Message templates collectively identify events generated from the same line of logging/tracing code. |
+| `@ParentId` | `string?` | The `@SpanId` of the parent of a given span, if any. The parent span will always belong to the same trace, that is, share a `@TraceId` value. Only present on spans, not log events. |
+| `@Properties` | `object?` | An object containing the user-defined properties of a log event or span. Properties with names that are valid C-style identifiers can be accessed implicitly, so `RequestPath` is syntactically equivalent to `@Properties['RequestPath']`. Properties generally conform to naming conventions used throughout the Seq server - sometimes simple PascalCase names, and at other times using the OpenTelemetry semantic conventions. See also `@Resource` and `@Scope`. |
+| `@Resource` | `object?` | For an OpenTelemetry log event or span, the properties associated with the OpenTelemetry resource. These may follow the OTel semantic conventions, but may also be domain-specific or user-defined. |
+| `@Scope` | `object?` | For an OpenTelemetry log event or span, the properties associated with the OpenTelemetry scope. These may match definitions in the OTel semantic conventions, but may also be domain-specific or user-defined. |
+| `@SpanId` | `string?` | The W3C span id that uniquely identifies a span within a trace. Log events recorded during the span carry the same `@SpanId` value as the span itself. |
+| `@SpanKind` | `string?` | The OpenTelemetry span kind. Only present on spans, not log events. |
+| `@Start` | `number?` | The time at which the span started. `@Timestamp` minus `@Start` is the `@Elapsed` time of the span. In the same units as Seq's duration literal syntax. |
+| `@Timestamp` | `number` | The time at which an event was recorded (completion time, for spans). Carried on all log events and spans. In the same units as Seq's duration literal syntax. |
+| `@TraceId` | `string?` | The W3C trace id that uniquely identifies a trace. All spans and log events within a trace carry the same trace id value. |
+
+> Tip: if searching or querying with `seqcli`, pass `--native` to show results in round-trippable, agent-friendly syntax. Consider `--json` instead for programmatic scenarios, but take care: in JSON output mode, built-in property names are abbreviated.
+
+## Type System
+
+Stored data and intermediate values in expression evaluation are typed dynamically. Values are always one of the following types.
+
+| Type name | Description | Example literals |
+|---|---|---|
+| **null** | The atom `null`. Null is a value in Seq's type system. | `null` |
+| **boolean** | The atoms `true` and `false`. | `true`, `false` |
+| **number** | Decimal numbers with the range and precision of .NET's `decimal` type. | `0`, `12.34`, `56ms`, `DateTime('2026-05-29T10:56:01.43278Z')`, `0xa1b234ff` |
+| **array** | An ordered array of values. | `[]`, `[17, null, {a: 'test'}]` |
+| **object** | An unordered set of name/value pairs. | `{}`, `{a: 'test', 'b c': 17, d: []}` |
+
+In expression evaluation, Seq does not perform any type coercion.
+
+The results of functions and operators that receive invalid arguments are undefined, which is the absence of a value 
+(_undefined_ has the same "poison" semantics as `NULL` in standard SQL).
+
+Type notation in this document column uses the suffix `?` on a type name to indicate values that may be undefined.
+The synthetic type name `any` is used as an alias for `null | boolean | number | array | object`.
+
+## Scalar Functions and Operators
+
+These built-in functions and operators work with individual values. See Aggregate Functions for information on functions like `count()` and `distinct()` that work with sets of values.
+
+| Function signature | Description |
+|---|---|
+| `Arrived(eventId: string): number?` | Evaluates to the arrival order encoded in `eventId`. The arrival order is a hint that preserves the order of events from the same source that have the same timestamp. |
+| `Bucket(n: number, err: number?): number` | Reduce precision of `n` by computing the midpoint of the closest logarithmic bucket. The optional `err` parameter specifies the maximum permissible error fraction. |
+| `Coalesce(arg0: any?, arg1: any?, ...): any?` | Evaluates to the first defined, non-`null` argument. If no argument meets this requirement, `Coalesce` returns the value of its final argument. |
+| `Concat(str0: string, str1: string, ...): string` | Concatenate all string arguments. No type coercion is performed. |
+| `Contains(text: string, substring: string): boolean` | Evaluates to `true` if text contains substring. Supports the `ci` modifier. |
+| `DatePart(datetime: number, part: string, offset: number): number?` | Compute the value of `part` for the date/time `datetime` at time zone offset `offset`. Both `datetime` and `offset` are 100-nanosecond tick values. If `part` is not a recognized part name, the result is undefined. See the documentation section on date and time handling for more information. |
+| `DateTime(str: string): number?` | Attempt to parse the date/time value encoded in the string `str`. If the value cannot be parsed as a date/time, the result is undefined. |
+| `ElementAt(collection: array \| object, index: number \| string): any?` | Access the element of the array or object `collection` at the index or key `index`. Supports the `ci` modifier. |
+| `EndsWith(text: string, substring: string): boolean` | Evaluates to `true` if `text` ends with `substring`. Supports the `ci` modifier. |
+| `Every(collection: array \| object, predicate): boolean` | Evaluates to `true` if the function predicate evaluates to `true` for all elements of the array or object `collection`. |
+| `FromJson(json: string): any?` | Parse JSON, producing a value of the corresponding Seq native type. If `json` is not valid JSON, the result is undefined. High runtime cost, should be avoided when possible. |
+| `Has(arg: any?): boolean` | Evaluates to `true` if `arg` is defined. Otherwise, if `arg` is undefined, the result is `false`. |
+| `IndexOf(text: string, substring: string): number` | Return the zero-based index of the first occurrence of `substring` in `text`. If substring is not present in text, the result is `-1`. Supports the `ci` modifier. |
+| `Keys(obj: object): array` | Evaluates to an array containing the keys of the object `obj`. |
+| `LastIndexOf(text: string, substring: string): number` | Return the zero-based index of the last occurrence of `substring` in `text`. If substring is not present in text, the result is `-1`. Supports the `ci` modifier. |
+| `Length(arg: string \| array): number` | Evaluates to the length of the string or array `arg`. |
+| `Now(): number` | Evaluates to the current time, as 100-nanosecond ticks since 00:00:00 on 0001-01-01. |
+| `OffsetIn(timezone: string, instant: number): number?` | Determine the offset from UTC in time zone `timezone` at instant `instant`. The time zone name must be an IANA time zone name. The instant value is specified in 100-nanosecond ticks. See the documentation section on date and time handling for more information. |
+| `Replace(text: string, substring: string, replacement: string): string` | Replace all occurrences of `substring` in `text` with `replacement`. Accepts a `/regular expression/` in place of `substring`, in which case replacement may use `$0` to refer to the match, `$1` the first capturing group, and so on. Regular expression replacements use `$$` to escape a single dollar sign. Supports the `ci` modifier. |
+| `Round(value: number, places: number): number` | Round `value` to specified number of decimal places. Midpoint values (0.5) are rounded up. |
+| `Some(collection: array \| object, predicate): boolean` | Evaluates to `true` if the function predicate evaluates to `true` for any element of the array or object `collection`. |
+| `StartsWith(text: string, substring: string): boolean` | Evaluates to `true` if text starts with substring. Supports the `ci` modifier. |
+| `Substring(str: string, start: number, length: number?): string?` | Evaluates to the substring of string `str` from the zero-based index `start`, of `length` characters. If `length` is not specified, or exceeds the number of characters remaining after `start`, the result is the remainder of the string. The result is undefined if `start` is out of bounds, or if either `start` or `length` is negative. |
+| `TimeOfDay(datetime: number, offsetHours: number): number` | Compute the time of day of the date/time `datetime` in the time zone offset `offsetHours`. |
+| `TimeSpan(str: string): number?` | Attempt to parse the `d.HH:mm:ss.f` formatted time value encoded in the string `str`. If the value cannot be parsed as a time, the result is undefined. |
+| `ToEventType(str: string): any` | Compute the event type that Seq automatically assigns to `@EventType` from the message template `str`. |
+| `ToHexString(num: number): string` | Format `num` as a hexadecimal string, including leading `0x`. Decimal digits are discarded. |
+| `ToIsoString(datetime: number, offset: number?): string` | Format `datetime` as an ISO-8601 string, with an optional time zone offset. Both the datetime and offset arguments are interpreted as 100-nanosecond ticks since the epoch. The offset argument defaults to 0, or UTC. Output is given with the UTC or full time zone designator, i.e. Z or ±hh:mm. |
+| `ToJson(arg: any): string` | Convert the value `arg` to JSON. Can be used to convert a value (e.g. number) to a string. |
+| `ToLower(str: string): string` | Convert string `str` to lowercase. To compare strings in a case-insensitive manner, use the equality operator and `ci` modifier instead. |
+| `ToNumber(str: string): number?` | Parse string `str` as a number. |
+| `TotalMilliseconds(timespan: number \| string): number?` | Evaluates to the total number of milliseconds represented by the time span `timespan`. If `timespan` is a number, it will be interpreted as containing 100-nanosecond ticks. If `timespan` is a string, it will be parsed in the same manner as performed by `TimeSpan()`. |
+| `ToTimeString(timespan: number): string` | Format `timespan` as an `d.HH:mm:ss.f` string. The `timespan` argument is interpreted as 100-nanosecond ticks. The inverse of TimeSpan. |
+| `ToUpper(str: string): string` | Convert string `str` to uppercase. To compare strings in a case-insensitive manner, use the equality operator and `ci` modifier instead. |
+| `TypeOf(arg: any?): string` | Returns the type of value, either `'object'`, `'array'`, `'string'`, `'number'`, `'boolean'`, `'null'`, or `'undefined'`. |
+| `Values(obj: object): array` | Evaluates to an array containing the values of the members of object `obj`. |
+| Operator `+`, `-`, `*`, `/` | Arithmetic operators. If any argument is non-numeric, or if a divisor is zero, the result is undefined. |
+| Operator `-` (prefix) | Negate a number. If any argument is non-numeric, the result is undefined. |
+| Operator `%` | Remainder after dividing the left-hand operand by the right. If the right-hand operand is zero, the result is undefined. If any argument is non-numeric, the result is undefined. |
+| Operator `^` | Raise a number to the specified power. If any argument is non-numeric, the result is undefined. |
+| Operator `=` | Compare two values, returning `true` if the values are equal, and `false` otherwise. Structural comparison is supported, so the values may be of any type including objects and arrays. If the right-hand operand is a `/regular expression/`, the result is `true` if the left-hand operand is a string that is an exact match for the regular expression. If any argument is undefined, the result is undefined. Supports the `ci` modifier. |
+| Operator `<>` | Compare two values, returning `true` if the values are unequal, and `false` otherwise. Structural comparison is supported, so the values may be of any type including objects and arrays. If any argument is undefined, the result is undefined. Supports the `ci` modifier. |
+| Operator `<`, `<=`, `>`, `>=` | Compare two values. If any argument is undefined, the result is undefined. |
+| Operator `and` | The result of `a and b` is `true` if and only if both `a` and `b` are `true`; the result is otherwise `false`. Type coercion is not performed. If any argument is non-Boolean, the result is undefined. |
+| Operator `not` (prefix) | Evaluates to `true` if the operand is `false`, or if the operand is undefined. Type coercion is not performed. If any argument is non-Boolean, the result is undefined. |
+| Operator `or` | The result of `a or b` is `true` if either `a` or `b` is `true`; otherwise the result is `false`. Type coercion is not performed. If any argument is non-Boolean, the result is undefined. |
+| Operator `like` | Determine if the left-hand operand is a string matching the right-hand pattern. The pattern can contain `%` and `?` wildcards for zero-or-many, or zero-or-one characters. `%` and `?` are escaped by doubling. The inverse `not like` is also supported. |
+| Operator `is null` | Determine if the left-hand operand is `null` or undefined. The result is always a defined `boolean`. The inverse `is not null` is also supported. |
+| Operator `in` | Determine if the left-hand operand is an element of the right-hand array. |
+
+## Aggregate Functions
+
+`select` queries have access to the following aggregate functions.
+
+| Aggregate function signature | Description | Example |
+|---|---|---|
+| `all(expr: boolean): boolean` | Return `true` if `expr` is `true` for all events in the stream. | `all(@Level = 'Error')` |
+| `any(expr: boolean): boolean` | Return `true` if `expr` is `true` for any event in the stream. | `any(@Level = 'Error')` |
+| `bottom(expr: any, n: number): rowset` | Compute the last `n` values that appear for `expr`. The `bottom` function cannot appear with any other aggregate functions. The default ordering for `select` queries on `stream` is time-ascending. See also: `top()`, `first()`, `last()`. | `bottom(StatusCode, 5)` |
+| `count(property: any): number` | Computes the number of events that have a non-`null` value for `property`. The special property name `*` can be used to count all events. | `count(*)` |
+| `distinct(expr: any): rowset` | Computes the set of distinct values for `expr`. The `distinct` function cannot appear with any other aggregate functions. `distinct()` and `count()` can be combined to count distinct values without returning them all. | `distinct(ExceptionType)` |
+| `first(expr: any): any` | Returns the value of `expr` applied to the first events in the target range. | `first(Elapsed)` |
+| `last(expr: any): any` | Returns the value of `expr` applied to the last events in the target range. | `last(Elapsed)` |
+| `interval(): number` | In a query that groups by time, the duration of each time slice. | `count(*) / (interval() / 1d)` |
+| `min(expr: number): number` | Computes the smallest value for `expr`. | `min(Elapsed / 1000)` |
+| `max(expr: number): number` | Computes the largest value for `expr`. | `max(Elapsed / 1000)` |
+| `mean(expr: number): number` | Computes the arithmetic mean (average) of `expr`, i.e. `sum(expr) / count(expr)`. Events where the expression is `null` or not numeric are ignored and do not contribute to the final result. | `mean(ItemCount)` |
+| `percentile(expr: number, p: number [, err: number?]): number` | Given a percentage `p`, calculates the value of `expr` at or below which `p` percent of the results fall. The optional `err` parameter specifies the maximum permissible error fraction. Higher error values reduce compute and memory resource consumption. | `percentile(ResponseTime, 95 , 0.01)` |
+| `phist(expr: number, count: number, p: number): number` | Bucketed (histogram) percentile. | |
+| `sum(expr: number): number` | Calculates the sum of `expr`. Non-numeric values are ignored. | `sum(ItemsOrdered)` |
+| `top(expr: any, n: number): rowset` | Select the first `n` values of `expr`. The `top` function cannot appear with any other aggregate functions. | `top(StatusCode, 5)` |
+
+## Base Grammar
+
+```ebnf
+identifier = ( letter | '_' ) , { letter | digit | '_' } ;
+built_in_identifier = '@' , ( letter | digit | '_' ) , { letter | digit | '_' } ;
+variable = '$' , ( letter | digit | '_' ) , { letter | digit | '_' } ;
+letter = ? any Unicode letter ? ;
+digit = ? any Unicode digit ? ;
+string_literal = "'" , { string_char } , "'" ;
+string_char = "''" | ? any character except single quote ? ;
+number = natural , [ '.' , natural ] ;
+hex_number = '0x' , hex_digit , { hex_digit } ;
+natural = digit , { digit } ;
+hex_digit = digit | 'a' | 'b' | 'c' | 'd' | 'e' | 'f'
+   | 'A' | 'B' | 'C' | 'D' | 'E' | 'F' ;
+duration = { natural , time_unit }- ;
+time_unit = 'd' | 'h' | 'ms' | 'm' | 'us' | 'μs' | 'ns' | 's' ;
+regular_expression = '/' , { regex_char } , '/' ;
+regex_char = '\/' | ? any character except '/' ? ;
+```
+
+### Expression Grammar
+
+```ebnf
+Expr = Disjunction ;
+Disjunction = Conjunction , { 'or' , Conjunction } ;
+Conjunction = Comparison , { 'and' , Comparison } ;
+Comparison = Comparand , { comparison_op , Comparand , [ 'ci' ] } ;
+comparison_op = 'not' , 'like'
+   | 'like'
+   | 'not' , 'in'
+   | 'in'
+   | '<=' | '<>' | '<'
+   | '>=' | '>'
+   | '=' ;
+Comparand = Term , { ( '+' | '-' ) , Term }  ;
+Term = InnerTerm , { ( '*' | '/' | '%' ) , InnerTerm } ;
+InnerTerm = Operand , { '^' , Operand } ;
+
+Operand = ( unary_op , Operand | Path ) , [ 'is' , null_test ] ;
+unary_op = '-' | 'not' ;
+null_test = 'null' | 'not' , 'null' ;
+Path = Factor , { path_step } ;
+path_step = '.' , identifier
+   | '[' , ( wildcard | Expr ) , ']' ;
+wildcard = '?' | '*' ;
+Factor = '(' , Expr , ')'
+   | Item ;
+Item = Property
+   | Literal
+   | Function
+   | ArrayLiteral
+   | ObjectLiteral
+   | Conditional
+   | Block
+   | Lambda
+   | Variable ;
+Property = built_in_identifier
+   | identifier ;   (* when not followed by '(' *)
+Literal = string_literal
+   | number
+   | hex_number
+   | duration
+   | regular_expression
+   | 'true'
+   | 'false'
+   | 'null' ;
+Function = function_name , '(' , arg_list , ')' , [ 'ci' ] ;
+function_name = identifier
+   | 'and' | 'not' | 'or' ;
+arg_list = '*'   (* only valid for count(*) *)
+   | [ Expr , { ',' , Expr } ] ;
+ArrayLiteral = '[' , [ Expr , { ',' , Expr } ] , ']' ;
+ObjectLiteral = '{' , [ ObjectMember , { ',' , ObjectMember } ] , '}' ;
+ObjectMember  = ( identifier | string_literal ) , ':' , Expr ;
+Conditional = 'if' , Expr , 'then' , Expr , 'else' , Expr ;
+Block = 'let' , '|' , Binding , { ',' , Binding } , '|' , Expr ;
+Binding = identifier , ':' , Expr ;
+Lambda = '|' , [ identifier , { ',' , identifier } ] , '|' , Expr ;
+Variable = variable ;
+```
+
+### Query Grammar
+
+```ebnf
+Query = [ ExplainClause ]
+   SelectClause
+   [ IntoClause ]
+   [ FromClause ]
+   [ WhereClause ]
+   [ GroupByClause ]
+   [ HavingClause ]
+   [ OrderByClause ]
+   [ LimitClause ]
+   [ ForClause ] ;
+ExplainClause = 'explain' , [ 'analyze' | 'lower' ] ;
+SelectClause = 'select' , SelectColumn , { ',' , SelectColumn } ;
+SelectColumn = '*'
+   | Expr , [ 'as' , identifier ] ;
+IntoClause = 'into' , variable ;
+FromClause = 'from' , source , { LateralJoin } ;
+source = 'stream' | 'series' ;
+LateralJoin = 'lateral' , 'unnest' , '(' , Expr , ')' , 'as' , identifier ;
+WhereClause = 'where' , Expr ;
+GroupByClause = 'group' , 'by' , Grouping , { ',' , Grouping } ;
+Grouping = TimeGrouping
+   | Expr , [ 'ci' ] , [ 'as' , identifier ] ;
+TimeGrouping  = 'time' , '(' , duration , ')' ;
+HavingClause = 'having' , Expr ;
+OrderByClause = 'order' , 'by' , Ordering , { ',' , Ordering } ;
+Ordering = TimeOrdering
+   | identifier , [ 'ci' ] , [ 'asc' | 'desc' ] ; (* identifier must be a selected column or group key alias *)
+TimeOrdering  = 'time' ;
+LimitClause = 'limit' , natural ;
+ForClause = 'for' , ForOption , { ',' , ForOption } ;
+ForOption = identifier , [ '(' , [ Expr , { ',' , Expr } ] , ')' ] ;
+```
+
+Keywords are case-insensitive.
+
+## Example Expressions
+
+| Example | Purpose |
+|--------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
+| `@Timestamp >= Now() - 10m` | Match events that occurred in the last ten minutes. |
+| `@TraceId = '0af7651916cd43dd8448eb211c80319c'` | Match all events (both spans and log events) belonging to the given trace. |
+| `Has(@Start)` | Match all spans (excludes log events). |
+| `@Message like '%overflow%' ci or @Exception like '%overflow%' ci` | Given a piece of text, find events with that text in their message or exception/stack trace, using case-insensitive comparisons. |
+| `Items[?] = 'coffee'` | Wildcard "any" - check if element appears in collection. |
+| `ToIsoString(@Timestamp)` | Render a numeric timestamp as ISO-8601. |
+| `ToTimeString(@Elapsed)` | Render a numeric duration value as a human-readable time string. |
+| `@Resource.service.name = 'unknown_service'` | Match events from a specific service (OpenTelemetry semantic convention) |
+| `@TraceId = '...' and @SpanId = '...' and Has(@Start)` | Retrieve a specific trace span using a search expression |
+| `@Level like 'err%' ci` | Perform a case-insensitive prefix search |
+
+## Query Example
+
+Grouped query with ordering:
+
+```
+select count(*)
+from stream
+group by @Resource.service.name as service
+order by service
+```
+
+## Tracing Tactics
+
+* Find traces by searching for their included spans.
+* To reconstruct a trace as a call tree, use the `seq_load_trace` MCP tool, or run `seqcli trace -i <traceid>`.
+* Project additional Seq expressions per span with `columns` (CLI: repeated `--column`), for example
+  `@Resource.service.name` or `@SpanKind`; values appear in each node's `columns` object, keyed by expression.
+* On large traces, pass `span_id` (CLI: `--span-id`) to retrieve only the subtree rooted at one span.
+* If `complete` is `false`, the trace was truncated at the retrieval limit.
+* Log events recorded during a span are not included by `seq_load_trace`; find them by searching with
+  `@TraceId = '...' and @SpanId = '...' and not Has(@Start)` (CLI: pass `--logs`).
+* To see all properties of a single span, use the "retrieve a specific trace span" search recipe above.
+
+Rank services by span latency over a window:
+
+```
+select
+   count(*) as spans,
+   Round(TotalMilliseconds(percentile(@Elapsed, 95)), 2) as p95_ms,
+   Round(TotalMilliseconds(max(@Elapsed)), 2) as max_ms
+from stream
+where @Timestamp >= now() - 30m and Has(@Start)
+group by @Resource.service.name as service     -- group key: alias it, do NOT put it in the select list
+having spans > 50                              -- filter groups by the select alias, NOT by count(*) directly
+order by p95_ms desc                           -- order by a selected aggregate's alias
+limit 100
+```
+
+## User-Defined Property Schema
+
+Seq servers are compatible with a vast array of data sources. They may use a mix of OpenTelemetry and
+framework/ecosystem-specific property names, and may do so inconsistently.
+
+When available, **always use the MCP schema tool** to inspect the actual properties appearing on search results. In
+particular, don't skip schema checks early in investigations just because you've seen a few events. Events are
+inconsistent! Use the schema tool at least once just to be safe.
+
+## Gotchas
+
+ - Group keys are automatically included in result rowsets and **must not** be explicitly included in the `select` list.
+ - To order by group keys, apply an alias with `group by <expr> as <alias>` and use `order by <alias>`. Never add the group key to the `select` list, this will fail.
+ - OpenTelemetry dotted property names correspond to property accessor paths in Seq, so `@Resource.service.name` and `http.response.status_code` are written exactly like this.
+ - **Never** put a dotted OTel name inside `[...]`. `@Resource['a.b.c']` is a single literal key (almost always undefined); use `@Resource.a.b.c` for path navigation.
+ - Seq expression literals are not JSON, take care to use the Seq expression syntax when formatting literal values.
+ - Seq queries are not SQL. Don't expect standard SQL syntax, operators, or semantics to apply, always use the grammar and built-ins described above.
+ - Seq searches work backwards through the event stream and always return results in reverse-chronological order, from **most recent** to least recent.
+ - Data in Seq servers doesn't always use OpenTelemetry semantic conventions. When searching or querying, only use property names from the built-ins described above, that appear on search results, or that are returned from the schema tool.
+ - Bare identifiers like `SomeName` are synonymous with `@Properties['SomeName']`. The latter form allows irregular names to be used.
+ - The only escape sequence allowed and required in Seq strings is a doubled single quote - `''` - which evaluates to an embedded literal single quote. Backslash escaping is not recognized.
+ - `@Timestamp`, `@Start`, and `@Elapsed` are internally represented as .NET `DateTime` ticks (100 ns resolution) in order to support consistent timestamp/duration math. Comparing these properties with strings will fail: use duration literals for durations, and the `DateTime` function to convert from ISO-8601 strings.
+ - Although Seq's types resemble those from JavaScript, Seq does not support JavaScript operators and does not use JavaScript's system of comparisons.
+ - The expression `null = null` is `true` in Seq's type system; `null` is just a regular value.
+ - Timestamp bounds with inclusive starts and exclusive ends are the most efficient for Seq to work with.
+ - Regular expression evaluation is extremely expensive, avoid these as much as possible.
+ - Queries without `from stream` or `from series` are scalar (can't project out fields or compute aggregations).
+ - Searches and queries should always constrain results using `@Timestamp`, `@TraceId`, or `@Id`.
+ - `group by time(..)` requires an inclusive lower time bound on `@Timestamp`.
+ - Queries impose a default limit of 1024 rows, which can be changed with the `limit` clause. Set smaller limits to conserve resources when speculatively exploring.
+ - Use `ToIsoString()` and `ToTimeString()` to make timestamps or durations (even computed ones) readable. If you forget, you can convert individual values cheaply with a scalar query like `ToIsoString(12345)`.
+ - When grouping by `time(..)`, the time ordering leaves of the interval - just `order by time`, the interval isn't re-specified.
+ - All function calls and operators are case-sensitive unless the `ci` modifier is appended.

package/Skills/working-with-seq-metrics/SKILL.md

@@ -0,0 +1,157 @@
+---
+name: working-with-seq-metrics
+description: Query metrics stored in Seq. Use this when monitoring data or diagnostic metric information is required.
+license: Apache-2.0
+metadata:
+ author: Datalust and Contributors
+---
+
+Seq stores OpenTelemetry metric samples in the `series` data source, which is a columnar data store.
+
+Metrics are accessed **strictly** using the following workflow:
+
+ 1. Search for metric definitions
+ 2. Note the _kind_ of any relevant metrics
+ 3. List the dimensions that apply to the metric
+ 4. Optionally, list the values present in dimensions of interest
+ 5. Run aggregate queries on metric samples using metric names, kinds, and dimensions discovered in the previous steps
+
+**NEVER** query the `series` data source for metric definitions directly. **ALWAYS** use the dedicated definition search tools instead.
+
+**NEVER** query the `series` data source directly for available dimensions. **ALWAYS** use the dedicated dimension listing tools instead.
+
+**NEVER** list dimension values using the `distinct` aggregation. **ALWAYS** use the dedicated dimension value listing tools instead.
+
+**NEVER** evaluate projection queries against the `series` data source. This risks impacting the performance of Seq for other
+users and workloads.
+
+## Searching for Metric Definitions
+
+Metric samples carry an `@Definitions` object where metric names are keys into the object. Under each metric name key,
+a `description` property is available.
+
+Search for metrics with `cpu` in their **name** or **description** (case-insensitive):
+
+```
+Keys(@Definitions)[?] like '%cpu%' ci or @Definitions[?].description like '%cpu%' ci
+```
+
+Search for metrics reported by a specific service:
+
+```
+@Resource.service.name = 'accounting'
+```
+
+If metrics searches report syntax errors, you MUST load the `seq-search-and-query` skill.
+
+## Listing Dimensions
+
+Determine the dimensions that apply to a metric using the dimensions MCP tool or the `seqcli metrics dimensions` CLI
+command.
+
+## Listing Unique Dimension Values
+
+DO NOT use `distinct` to query for the values of a metric dimension like `@Resource.service.name`. Always use the
+dimension values MCP tool or `seqcli metrics dimensionvalues` CLI command.
+
+## Querying Metric Samples
+
+Metrics returned from definition searches are tagged with a kind:
+
+* `Sum` - counter with delta aggregation temporality
+* `Gauge` - gauge or counter with cumulative aggregation temporality
+* `Fixed` - histogram with fixed buckets
+* `Exponential` - histogram with exponential buckets
+
+Correctly querying metric sample data from `series` relies on knowing the metric kind.
+
+### `Sum` Query Example
+
+```
+select sum(app_product_review_counter) as reviews_submitted
+from series
+where Has(app_product_review_counter)
+  and @Timestamp >= now() - 15m
+group by time(1m)
+```
+
+### `Gauge` Query Examples
+
+```
+select max(dotnet.gc.last_collection.heap.size) as heap_size_bytes
+from series 
+where (Has(dotnet.gc.last_collection.heap.size))
+  and @Resource.service.name = 'accounting'
+  and @Timestamp >= now() - 15m
+group by time(1m), @Resource.service.instance.id
+having heap_size_bytes > 5000000
+limit 100
+```
+
+### `Fixed` Query Examples
+
+HTTP request timing percentiles:
+
+```
+select
+  phist(coalesce((bucket.from + bucket.to) / 2, bucket.from, bucket.to), bucket.count, 95) as p95,
+  phist(coalesce((bucket.from + bucket.to) / 2, bucket.from, bucket.to), bucket.count, 99) as p99
+from series
+  lateral unnest(http.server.request.duration.buckets) as bucket 
+where Has(http.server.request.duration.buckets)
+  and @Resource.service.name = 'cart'
+  and @Scope.name = 'Microsoft.AspNetCore.Hosting'
+  and @Timestamp >= now() - 15m
+group by time(1m)
+limit 100
+```
+
+HTTP request througput (counts):
+
+```
+select sum(http.server.request.duration.count) as request_throughput
+from series
+where Has(http.server.request.duration.buckets)
+  and @Resource.service.name = 'cart'
+  and @Scope.name = 'Microsoft.AspNetCore.Hosting'
+  and @Timestamp >= now() - 15m
+group by time(1m)
+limit 100
+```
+
+Maximium HTTP request body size:
+
+```
+select max(http.server.request.body.size.max) as max_body_size
+from series 
+where Has(http.server.request.body.size.buckets)
+  and @Resource.service.name = 'shipping'
+  and @Scope.name = 'opentelemetry-instrumentation-actix-web'
+  and @Timestamp >= now() - 15m
+group by time(1m)
+limit 100
+```
+
+### `Exponential` Query Examples
+
+Commit timing percentiles:
+
+```
+select
+  phist(bucket.midpoint, bucket.count, 95) as p95,
+  phist(bucket.midpoint, bucket.count, 99) as p99
+from series
+  lateral unnest(commit_duration.buckets) as bucket 
+where Has(commit_duration.buckets)
+  and @Timestamp >= now() - 15m
+group by time(1m)
+limit 100
+```
+
+## Gotchas
+
+- If metrics that should be _counters_ show up as _gauges_, aggregation temporality is likely to be misconfigured: danger! This can produce wildly inaccurate results if not detected.
+- When writing queries that aggregate metric samples, `where Has(<metric name>)` must always be included to avoid row count skew.
+- Do NOT use `select disctint(<dimension>)` with metric dimensions: aways use the optimized dimension value searches exposed by `seqcli` or the MCP tools.
+- Group keys are automatically included in result rowsets and **must not** be explicitly included in the `select` list.
+- To order by group keys, apply an alias with `group by <expr> as <alias>` and use `order by <alias>`. Never add the group key to the `select` list, this will fail.