@gordon.gan/specflow 1.4.0-beta → 1.4.2-beta

package/skills/database/elasticsearch/references/04-aggregations-bucket.md

@@ -0,0 +1,236 @@
+# 桶聚合 (Bucket Aggregations)
+
+桶聚合将文档分组到不同的"桶"中，类似 SQL 的 GROUP BY。每个桶可以嵌套子聚合。
+
+## terms — 分组聚合 (GROUP BY)
+
+```bash
+# 按标签分组统计
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "by_tags": {
+      "terms": {
+        "field": "tags",
+        "size": 20,
+        "order": { "_count": "desc" }
+      }
+    }
+  }
+}
+
+# 带子聚合: 每个分组的平均价格
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "by_brand": {
+      "terms": {
+        "field": "brand",
+        "size": 10,
+        "order": { "avg_price": "desc" }
+      },
+      "aggs": {
+        "avg_price": { "avg": { "field": "price" } },
+        "product_count": { "value_count": { "field": "id" } }
+      }
+    }
+  }
+}
+```
+
+**注意事项**：
+- `size` 控制返回桶数（默认 10），超大 size 消耗内存
+- `order` 支持按文档数 (`_count`)、聚合键 (`_key`)、子聚合排序
+- keyword 字段上聚合性能最佳，text 字段需开启 `fielddata`
+
+## range / date_range — 范围分组
+
+```bash
+# 价格区间分组
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "price_ranges": {
+      "range": {
+        "field": "price",
+        "ranges": [
+          { "key": "0-1000", "from": 0, "to": 1000 },
+          { "key": "1000-5000", "from": 1000, "to": 5000 },
+          { "key": "5000+", "from": 5000 }
+        ]
+      }
+    }
+  }
+}
+
+# 日期范围
+GET /orders/_search
+{
+  "size": 0,
+  "aggs": {
+    "date_ranges": {
+      "date_range": {
+        "field": "order_date",
+        "ranges": [
+          { "from": "now-30d/d", "to": "now" },
+          { "from": "now-90d/d", "to": "now-30d/d" }
+        ]
+      }
+    }
+  }
+}
+```
+
+## histogram / date_histogram — 直方图
+
+```bash
+# 价格直方图 (间隔 1000)
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "price_histogram": {
+      "histogram": {
+        "field": "price",
+        "interval": 1000,
+        "min_doc_count": 1
+      }
+    }
+  }
+}
+
+# 时间直方图 (按小时/天/月/年聚合)
+GET /orders/_search
+{
+  "size": 0,
+  "aggs": {
+    "orders_over_time": {
+      "date_histogram": {
+        "field": "order_date",
+        "calendar_interval": "day",     // month, quarter, year
+        "format": "yyyy-MM-dd",
+        "min_doc_count": 0,
+        "extended_bounds": {
+          "min": "2024-01-01",
+          "max": "2024-12-31"
+        }
+      }
+    }
+  }
+}
+```
+
+## filter / filters — 过滤聚合
+
+```bash
+# 单一过滤
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "active_products": {
+      "filter": { "term": { "status": "active" } },
+      "aggs": {
+        "avg_price": { "avg": { "field": "price" } }
+      }
+    }
+  }
+}
+
+# 多过滤
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "price_categories": {
+      "filters": {
+        "other_bucket": true,
+        "filters": {
+          "budget":  { "range": { "price": { "lte": 5000 } } },
+          "mid":     { "range": { "price": { "from": 5000, "to": 15000 } } },
+          "premium": { "range": { "price": { "gte": 15000 } } }
+        }
+      }
+    }
+  }
+}
+```
+
+## 管道聚合 (Pipeline Aggregations)
+
+```bash
+# derivative — 环比增量
+GET /orders/_search
+{
+  "size": 0,
+  "aggs": {
+    "sales_per_day": {
+      "date_histogram": {
+        "field": "order_date",
+        "calendar_interval": "day"
+      },
+      "aggs": {
+        "daily_sales": { "sum": { "field": "amount" } },
+        "sales_derivative": {
+          "derivative": { "buckets_path": "daily_sales" }
+        }
+      }
+    }
+  }
+}
+
+# bucket_script — 计算桶间比例
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "total_products": { "value_count": { "field": "id" } },
+    "active_products": {
+      "filter": { "term": { "status": "active" } }
+    },
+    "active_ratio": {
+      "bucket_script": {
+        "buckets_path": {
+          "activeCount": "active_products>_count",
+          "totalCount": "total_products"
+        },
+        "script": "params.activeCount / params.totalCount * 100"
+      }
+    }
+  }
+}
+
+# bucket_selector — 过滤桶 (类似 HAVING)
+GET /products/_search
+{
+  "size": 0,
+  "aggs": {
+    "by_brand": {
+      "terms": { "field": "brand", "size": 100 },
+      "aggs": {
+        "avg_price": { "avg": { "field": "price" } },
+        "brands_having_avg_gt_10000": {
+          "bucket_selector": {
+            "buckets_path": { "avgPrice": "avg_price" },
+            "script": "params.avgPrice > 10000"
+          }
+        }
+      }
+    }
+  }
+}
+```
+
+## 业务场景速查
+
+| 聚合 | 场景 |
+|------|------|
+| terms | 商品分类统计、品牌分布、标签统计 |
+| date_histogram | 销售日报/月报、API 请求时序、监控趋势 |
+| range | 价格区间分布、年龄段统计 |
+| filters | 多条件对比分析 |
+| derivative | 环比增长/下降分析 |
+| bucket_selector | 过滤出符合条件的组 (HAVING) |

package/skills/database/elasticsearch/references/05-mapping-types.md

@@ -0,0 +1,134 @@
+# 映射与字段类型详解
+
+Mapping 定义文档中每个字段的数据类型和分析方式，相当于关系型数据库的 Schema。
+
+## 字段类型速查
+
+| 字段类型 | 说明 | 适用场景 |
+|---------|------|---------|
+| `text` | 被分词的全文字段 | 全文搜索、文章内容 |
+| `keyword` | 精确值、不被分词 | 标签、状态、分类、ID |
+| `integer` | 32 位整数 | 年龄、计数 |
+| `long` | 64 位整数 | 时间戳、大数值 |
+| `float` | 单精度浮点 | 价格、分数 |
+| `double` | 双精度浮点 | 科学计算 |
+| `boolean` | 布尔值 | 开关、状态 |
+| `date` | 日期 (可多 format) | 时间字段 |
+| `ip` | IPv4/IPv6 | IP 分析、CIDR 匹配 |
+| `geo_point` | 经纬度点 | 地理位置、距离排序 |
+| `geo_shape` | 复杂地理形状 | 区域查询、地理围栏 |
+| `nested` | 嵌套对象 (独立索引) | 保持数组内对象关联 |
+| `object` | JSON 对象 (默认) | 普通嵌套数据 |
+| `completion` | 自动补全 | 搜索建议 |
+| `dense_vector` | 稠密向量 | KNN 语义搜索 |
+| `flattened` | 扁平化嵌套对象 | 未知结构的元数据 |
+| `percolator` | 反向搜索 | 告警规则匹配 |
+| `range` | 范围类型 (integer_range) | IP 段、时间范围 |
+
+## 动态映射 (Dynamic Mapping)
+
+| 设置 | 行为 |
+|------|------|
+| `"dynamic": true` | 自动检测并添加新字段 (默认) |
+| `"dynamic": "runtime"` | 运行时映射 (7.11+) |
+| `"dynamic": false` | 忽略新字段 (不索引，可查询 _source) |
+| `"dynamic": "strict"` | 遇到新字段抛出异常 |
+
+## 映射参数详解
+
+```json
+PUT /articles
+{
+  "mappings": {
+    "properties": {
+      "title": {
+        "type": "text",
+        "analyzer": "ik_max_word",
+        "search_analyzer": "ik_smart",
+        "fields": {
+          "keyword": {
+            "type": "keyword",
+            "ignore_above": 256
+          },
+          "pinyin": {
+            "type": "text",
+            "analyzer": "pinyin"
+          }
+        }
+      },
+      "price": {
+        "type": "float",
+        "coerce": false
+      },
+      "meta_data": {
+        "type": "flattened"
+      }
+    }
+  }
+}
+```
+
+## 关键参数
+
+| 参数 | 说明 | 默认值 |
+|------|------|--------|
+| `analyzer` | 索引分词器 | standard |
+| `search_analyzer` | 搜索分词器 | 同 analyzer |
+| `fields` | 多字段 | 无 |
+| `copy_to` | 复制到组合字段 | 无 |
+| `coerce` | 自动类型转换 | true |
+| `doc_values` | 列式存储（排序/聚合） | text 外 true |
+| `index` | 是否索引 | true |
+| `norms` | 归一化因子（算分用） | true |
+| `ignore_above` | 超长不索引 (keyword) | 无 |
+| `eager_global_ordinals` | 预加载全局序数 | false |
+
+## 别名 (Alias) 与索引模板
+
+```json
+// 原子切换别名 (零停机重建)
+POST /_aliases
+{
+  "actions": [
+    { "remove": { "index": "products_v1", "alias": "products" } },
+    { "add": { "index": "products_v2", "alias": "products" } }
+  ]
+}
+
+// 索引模板
+PUT /_index_template/logs_template
+{
+  "index_patterns": ["logs-*"],
+  "template": {
+    "settings": {
+      "number_of_shards": 3,
+      "number_of_replicas": 1
+    },
+    "mappings": {
+      "properties": {
+        "@timestamp": { "type": "date" },
+        "message": { "type": "text" },
+        "level": { "type": "keyword" }
+      }
+    }
+  }
+}
+```
+
+## 索引设置
+
+```json
+PUT /my_index
+{
+  "settings": {
+    "number_of_shards": 3,
+    "number_of_replicas": 1,
+    "refresh_interval": "30s",
+    "max_result_window": 100000,
+    "codec": "best_compression"
+  },
+  "mappings": { ... }
+}
+```
+
+**重要**：`number_of_shards` 建后不可修改。`refresh_interval` 大批量写入时可设为 `-1`（关闭）。

package/skills/database/elasticsearch/references/06-analyzers.md

@@ -0,0 +1,187 @@
+# 分词器 (Analyzer)
+
+分词器将文本拆分为词项 (Token)，用于建立倒排索引和搜索。
+
+## Analyzer 组成
+
+```
+Analyzer = Char Filter(s) + Tokenizer + Token Filter(s)
+           (字符过滤)      (分词器)     (词项过滤)
+
+输入: "I <3 Elasticsearch!"
+    ↓
+char_filter: HTML 解码 → "I <3 Elasticsearch!"
+    ↓
+tokenizer: 按空格/标点拆分 → ["I", "<3", "Elasticsearch"]
+    ↓
+token_filter: 小写化 → ["i", "<3", "elasticsearch"]
+    ↓
+输出: ["i", "<3", "elasticsearch"] (存入倒排索引)
+```
+
+## 测试分词器
+
+```bash
+# 测试分析器效果
+POST /_analyze
+{
+  "analyzer": "standard",
+  "text": "I love Elasticsearch 搜索引擎"
+}
+
+# 指定字段测试 (使用字段配置的分析器)
+POST /products/_analyze
+{
+  "field": "title",
+  "text": "Apple MacBook Pro 16英寸"
+}
+```
+
+## 内置分词器
+
+| 分词器 | 说明 | 示例: "I love ES" |
+|--------|------|------------------|
+| `standard` | Unicode 分词, 小写化 (默认) | ["i", "love", "es"] |
+| `simple` | 非字母分割, 小写化 | ["i", "love", "es"] |
+| `whitespace` | 空格分割 (不小写) | ["I", "love", "ES"] |
+| `keyword` | 不分词, 整个字符串输出 | ["I love ES"] |
+| `pattern` | 正则分割 | 取决于 pattern |
+| `stop` | 类似 simple + 去停用词 | ["love", "es"] |
+| `fingerprint` | 排序 + 去重 | ["es", "i", "love"] |
+
+## IK 分词器 (中文)
+
+```bash
+# 安装 (需重启 ES)
+./bin/elasticsearch-plugin install \
+  https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v8.12.0/elasticsearch-analysis-ik-8.12.0.zip
+
+# ik_smart (粗粒度)
+POST /_analyze { "analyzer": "ik_smart", "text": "中华人民共和国国歌" }
+# → ["中华人民共和国", "国歌"]
+
+# ik_max_word (细粒度, 穷尽所有可能)
+POST /_analyze { "analyzer": "ik_max_word", "text": "中华人民共和国国歌" }
+# → ["中华人民共和国", "中华人民", "中华", "华人", "人民共和国", "人民", "共和国", "共和", "国歌"]
+```
+
+| 场景 | 推荐 | 理由 |
+|------|------|------|
+| 索引 (建倒排索引) | `ik_max_word` | 细粒度, 覆盖更多可能性 |
+| 搜索 (用户输入) | `ik_smart` | 粗粒度, 提高搜索精度 |
+
+## 拼音分词器
+
+```bash
+./bin/elasticsearch-plugin install \
+  https://github.com/medcl/elasticsearch-analysis-pinyin/releases/download/v8.12.0/elasticsearch-analysis-pinyin-8.12.0.zip
+
+# 测试
+POST /_analyze { "analyzer": "pinyin", "text": "王大力" }
+# → ["wang", "da", "li", "wangda", "dali", "wangdali", "wdl"]
+
+# 拼音字段配置
+PUT /products
+{
+  "mappings": {
+    "properties": {
+      "title": {
+        "type": "text",
+        "analyzer": "ik_max_word",
+        "fields": {
+          "pinyin": {
+            "type": "text",
+            "analyzer": "pinyin",
+            "search_analyzer": "pinyin"
+          }
+        }
+      }
+    }
+  }
+}
+# 搜索 "wdali" → 匹配 "王大力"
+```
+
+## 自定义分析器
+
+```json
+PUT /my_index
+{
+  "settings": {
+    "analysis": {
+      "char_filter": {
+        "html_strip": { "type": "html_strip" },
+        "my_mapping": {
+          "type": "mapping",
+          "mappings": ["& => and", "| => or"]
+        }
+      },
+      "tokenizer": {
+        "my_standard": { "type": "standard", "max_token_length": 100 }
+      },
+      "filter": {
+        "my_stop": {
+          "type": "stop",
+          "stopwords": ["a", "an", "the", "is"]
+        },
+        "my_synonym": {
+          "type": "synonym",
+          "synonyms": [
+            "笔记本, 笔记本电脑, laptop",
+            "手机, 移动电话, smartphone"
+          ]
+        }
+      },
+      "analyzer": {
+        "my_custom_analyzer": {
+          "type": "custom",
+          "char_filter": ["html_strip", "my_mapping"],
+          "tokenizer": "my_standard",
+          "filter": ["lowercase", "my_stop", "my_synonym", "asciifolding"]
+        }
+      }
+    }
+  }
+}
+```
+
+## 自动补全 (Edge Ngram)
+
+```json
+PUT /autocomplete_index
+{
+  "settings": {
+    "analysis": {
+      "filter": {
+        "autocomplete_filter": {
+          "type": "edge_ngram",
+          "min_gram": 1,
+          "max_gram": 20
+        }
+      },
+      "analyzer": {
+        "autocomplete": {
+          "type": "custom",
+          "tokenizer": "standard",
+          "filter": ["lowercase", "autocomplete_filter"]
+        }
+      }
+    }
+  },
+  "mappings": {
+    "properties": {
+      "title": {
+        "type": "text",
+        "analyzer": "autocomplete",
+        "search_analyzer": "standard"
+      }
+    }
+  }
+}
+```
+
+## 同义词注意事项
+
+- **单向同义词**: `"laptop => 笔记本"` (搜索 laptop 映射到笔记本)
+- **双向同义词**: `"laptop, 笔记本"` (互相等价)
+- **最佳实践**: 索引时不用同义词（保持原始词），搜索时用 `search_analyzer`