@umacloud/knowledge 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/00-governance/governance-capabilities.md +557 -0
- package/00-governance/knowledge-map.md +39 -0
- package/00-governance/maintenance-policy.md +76 -0
- package/00-governance/review-checklist.md +81 -0
- package/README.md +13 -0
- package/ai/01-standards/agent-development-complete.md +691 -0
- package/ai/01-standards/llm-application-complete.md +488 -0
- package/ai/01-standards/mlops-complete.md +798 -0
- package/ai/01-standards/prompt-engineering-complete.md +646 -0
- package/ai/01-standards/rag-architecture-complete.md +649 -0
- package/ai/02-playbooks/llm-evaluation-playbook.md +847 -0
- package/ai/03-checklists/ai-project-checklist.md +215 -0
- package/ai/04-antipatterns/ai-antipatterns.md +661 -0
- package/ai/05-cases/case-rag-production.md +147 -0
- package/ai/06-glossary/ai-glossary.md +162 -0
- package/ai/agent-evaluation-benchmark.md +53 -0
- package/ai/ai-agent-memory-context-management.md +41 -0
- package/ai/ai-cost-capacity-optimization-playbook.md +42 -0
- package/ai/ai-data-security-and-compliance-playbook.md +37 -0
- package/ai/ai-domain-index-and-checklist.md +40 -0
- package/ai/ai-governance-maturity-model.md +50 -0
- package/ai/ai-model-selection-and-routing-strategy.md +47 -0
- package/ai/ai-observability-and-oncall-runbook.md +52 -0
- package/ai/ai-rag-engineering-playbook.md +42 -0
- package/ai/ai-red-team-and-safety-evaluation.md +42 -0
- package/ai/ai-release-readiness-and-rollback-gate.md +42 -0
- package/ai/llm-agent-engineering-deep-dive.md +57 -0
- package/ai/prompt-and-tool-guardrails.md +52 -0
- package/api/01-standards/enterprise-api-standards.md +198 -0
- package/api/01-standards/rest-api-design-guide.md +63 -0
- package/api/02-playbooks/api-pagination-playbook.md +93 -0
- package/api/02-playbooks/graphql-production-playbook.md +176 -0
- package/api/03-checklists/api-review-checklist.md +55 -0
- package/api/04-antipatterns/api-antipatterns.md +112 -0
- package/architecture/01-standards/api-gateway-patterns.md +496 -0
- package/architecture/01-standards/cloud-native-patterns.md +644 -0
- package/architecture/01-standards/distributed-systems-patterns.md +591 -0
- package/architecture/01-standards/event-driven-architecture.md +595 -0
- package/architecture/01-standards/microservices-patterns-complete.md +968 -0
- package/architecture/01-standards/microservices-patterns.md +495 -0
- package/architecture/01-standards/system-design-interview.md +664 -0
- package/architecture/02-playbooks/microservices-patterns-playbook.md +137 -0
- package/architecture/02-playbooks/migration-playbook.md +780 -0
- package/architecture/02-playbooks/system-design-playbook.md +779 -0
- package/architecture/03-checklists/architecture-decision-checklist.md +297 -0
- package/architecture/04-antipatterns/architecture-antipatterns.md +417 -0
- package/architecture/05-cases/case-netflix-microservices.md +413 -0
- package/architecture/06-glossary/architecture-glossary.md +164 -0
- package/architecture/adr-template-and-examples.md +38 -0
- package/architecture/api-gateway-deep-dive.md +1291 -0
- package/architecture/configuration-management.md +1162 -0
- package/architecture/distributed-transactions.md +1220 -0
- package/architecture/microservices-complete.md +735 -0
- package/architecture/resilience-and-disaster-patterns.md +37 -0
- package/architecture/service-governance.md +1198 -0
- package/architecture/system-architecture-deep-dive.md +37 -0
- package/backend/01-standards/analytics-and-growth.md +65 -0
- package/backend/01-standards/api-and-error-conventions.md +120 -0
- package/backend/01-standards/application-layering-and-packaging.md +160 -0
- package/backend/01-standards/auth-implementation.md +104 -0
- package/backend/01-standards/backend-framework-idioms.md +74 -0
- package/backend/01-standards/background-jobs-and-async.md +66 -0
- package/backend/01-standards/caching-strategies-complete.md +390 -0
- package/backend/01-standards/config-and-observability.md +77 -0
- package/backend/01-standards/data-modeling-and-persistence.md +94 -0
- package/backend/01-standards/django-complete.md +1765 -0
- package/backend/01-standards/email-and-notifications.md +64 -0
- package/backend/01-standards/fastapi-complete.md +925 -0
- package/backend/01-standards/file-upload-and-storage.md +66 -0
- package/backend/01-standards/graphql-api-complete.md +416 -0
- package/backend/01-standards/llm-application-standard.md +78 -0
- package/backend/01-standards/message-queue-patterns.md +379 -0
- package/backend/01-standards/microservices-and-distributed.md +78 -0
- package/backend/01-standards/nestjs-complete.md +2167 -0
- package/backend/01-standards/payment-integration.md +80 -0
- package/backend/01-standards/rate-limiting-complete.md +451 -0
- package/backend/01-standards/realtime-and-websocket.md +65 -0
- package/backend/01-standards/search-and-filtering.md +64 -0
- package/backend/01-standards/spring-boot-complete.md +445 -0
- package/backend/02-playbooks/api-design-playbook.md +718 -0
- package/backend/02-playbooks/email-send-playbook.md +130 -0
- package/backend/02-playbooks/file-upload-s3-playbook.md +153 -0
- package/backend/02-playbooks/typescript-enterprise-playbook.md +133 -0
- package/backend/02-playbooks/websocket-realtime-playbook.md +154 -0
- package/backend/03-checklists/api-launch-checklist.md +189 -0
- package/backend/04-antipatterns/backend-antipatterns.md +1051 -0
- package/blockchain/01-standards/blockchain-basics.md +557 -0
- package/blockchain/01-standards/smart-contract-development.md +1315 -0
- package/cicd/01-standards/deployment-and-delivery-standard.md +96 -0
- package/cicd/01-standards/github-actions-complete.md +473 -0
- package/cicd/01-standards/release-and-store-submission.md +75 -0
- package/cicd/02-playbooks/cicd-pipeline-playbook.md +144 -0
- package/cicd/02-playbooks/release-management-playbook.md +605 -0
- package/cicd/03-checklists/pipeline-security-checklist.md +168 -0
- package/cicd/04-antipatterns/cicd-antipatterns.md +589 -0
- package/cicd/05-cases/case-deployment-automation.md +221 -0
- package/cicd/05-cases/case-gitops-transformation.md +212 -0
- package/cicd/06-glossary/cicd-glossary.md +114 -0
- package/cicd/cicd-blueprint-deep-dive.md +38 -0
- package/cicd/release-readiness-gate.md +37 -0
- package/cloud-native/01-standards/container-security.md +741 -0
- package/cloud-native/01-standards/kubernetes-complete.md +812 -0
- package/cloud-native/02-playbooks/api-gateway-playbook.md +155 -0
- package/cloud-native/02-playbooks/gitops-with-argocd.md +760 -0
- package/cloud-native/02-playbooks/k8s-troubleshooting-playbook.md +1942 -0
- package/cloud-native/02-playbooks/message-queue-playbook.md +129 -0
- package/cloud-native/02-playbooks/multicloud-governance.md +726 -0
- package/cloud-native/02-playbooks/serverless-patterns.md +788 -0
- package/cloud-native/02-playbooks/service-mesh-playbook.md +612 -0
- package/cloud-native/02-playbooks/terraform-iac-playbook.md +143 -0
- package/cloud-native/03-checklists/container-security-checklist.md +431 -0
- package/cloud-native/03-checklists/k8s-production-readiness-checklist.md +460 -0
- package/cloud-native/04-antipatterns/container-antipatterns.md +660 -0
- package/cloud-native/04-antipatterns/k8s-antipatterns.md +743 -0
- package/cloud-native/05-cases/case-k8s-migration.md +478 -0
- package/cloud-native/05-cases/case-k8s-scaling.md +642 -0
- package/cloud-native/05-cases/case-k8s-security-incident.md +397 -0
- package/cloud-native/06-glossary/cloud-native-glossary.md +337 -0
- package/cross-platform/01-standards/cross-platform-frameworks.md +83 -0
- package/cross-platform/01-standards/platform-selection-and-architecture.md +77 -0
- package/data/01-standards/elasticsearch-complete.md +2098 -0
- package/data/01-standards/postgresql-complete.md +1613 -0
- package/data/01-standards/redis-complete.md +1527 -0
- package/data/02-playbooks/database-optimization-playbook.md +403 -0
- package/data/02-playbooks/elasticsearch-production-playbook.md +132 -0
- package/data/03-checklists/database-launch-checklist.md +187 -0
- package/data/04-antipatterns/database-antipatterns.md +873 -0
- package/data/05-cases/case-database-migration.md +310 -0
- package/data/06-glossary/database-glossary.md +440 -0
- package/data/data-governance-and-modeling-deep-dive.md +39 -0
- package/data-engineering/01-standards/airflow-complete.md +523 -0
- package/data-engineering/01-standards/kafka-complete.md +1521 -0
- package/data-engineering/02-playbooks/spark-etl-playbook.md +496 -0
- package/data-engineering/03-checklists/pipeline-launch-checklist.md +194 -0
- package/data-engineering/04-antipatterns/data-pipeline-antipatterns.md +684 -0
- package/data-engineering/05-cases/case-real-time-pipeline.md +355 -0
- package/data-engineering/06-glossary/data-engineering-glossary.md +429 -0
- package/database/01-standards/database-schema-standards.md +147 -0
- package/database/02-playbooks/postgresql-optimization-quick.md +52 -0
- package/database/02-playbooks/postgresql-performance-optimization.md +58 -0
- package/database/02-playbooks/postgresql-production-playbook.md +146 -0
- package/database/02-playbooks/redis-caching-playbook.md +117 -0
- package/database/03-checklists/database-review-checklist.md +50 -0
- package/database/04-antipatterns/database-antipatterns.md +112 -0
- package/design/01-standards/ui-design-system-complete.md +423 -0
- package/design/02-playbooks/design-handoff-playbook.md +254 -0
- package/design/02-playbooks/design-review-playbook.md +388 -0
- package/design/03-checklists/design-review-checklist.md +246 -0
- package/design/04-antipatterns/design-antipatterns.md +378 -0
- package/design/05-cases/case-design-system-adoption.md +328 -0
- package/design/06-glossary/design-glossary.md +329 -0
- package/design/ui-full-lifecycle-cross-platform-playbook.md +571 -0
- package/design/ux-system-deep-dive.md +38 -0
- package/design-systems/00-craft-rules.md +71 -0
- package/design-systems/aesthetic-families.md +43 -0
- package/design-systems/anti-ai-slop.md +162 -0
- package/design-systems/bold-geometric.md +120 -0
- package/design-systems/brutalist-bold.md +103 -0
- package/design-systems/editorial-clean.md +109 -0
- package/design-systems/glass-aurora.md +108 -0
- package/design-systems/modern-minimal.md +145 -0
- package/design-systems/premium-luxury.md +106 -0
- package/design-systems/product-type-design-map.md +48 -0
- package/design-systems/soft-warm.md +123 -0
- package/design-systems/tech-utility.md +113 -0
- package/desktop/01-standards/desktop-app-standard.md +72 -0
- package/desktop/01-standards/desktop-design.md +71 -0
- package/development/00-governance/document-template.md +41 -0
- package/development/01-standards/api-versioning-strategies.md +432 -0
- package/development/01-standards/authentication-patterns-complete.md +479 -0
- package/development/01-standards/css-architecture-complete.md +550 -0
- package/development/01-standards/database-migration-strategies.md +484 -0
- package/development/01-standards/elasticsearch-complete.md +347 -0
- package/development/01-standards/git-complete.md +371 -0
- package/development/01-standards/golang-complete.md +1565 -0
- package/development/01-standards/graphql-complete.md +298 -0
- package/development/01-standards/javascript-bundlers-complete.md +469 -0
- package/development/01-standards/javascript-typescript-complete.md +528 -0
- package/development/01-standards/jest-complete.md +275 -0
- package/development/01-standards/linux-complete.md +234 -0
- package/development/01-standards/logging-observability-complete.md +526 -0
- package/development/01-standards/microservices-communication.md +502 -0
- package/development/01-standards/mongodb-complete.md +406 -0
- package/development/01-standards/oauth2-complete.md +285 -0
- package/development/01-standards/performance-optimization-complete.md +289 -0
- package/development/01-standards/playwright-complete.md +247 -0
- package/development/01-standards/postgresql-complete.md +456 -0
- package/development/01-standards/pytest-complete.md +340 -0
- package/development/01-standards/python-async-programming.md +902 -0
- package/development/01-standards/python-complete.md +956 -0
- package/development/01-standards/python-decorators-complete.md +799 -0
- package/development/01-standards/python-design-patterns.md +2854 -0
- package/development/01-standards/python-packaging-distribution.md +420 -0
- package/development/01-standards/python-testing-strategies.md +607 -0
- package/development/01-standards/python-web-frameworks-comparison.md +471 -0
- package/development/01-standards/redis-complete.md +317 -0
- package/development/01-standards/rest-api-complete.md +316 -0
- package/development/01-standards/rust-complete.md +578 -0
- package/development/01-standards/typescript-advanced-types.md +1513 -0
- package/development/01-standards/web-security-complete.md +292 -0
- package/development/02-playbooks/api-design-playbook.md +810 -0
- package/development/02-playbooks/database-migration-playbook.md +580 -0
- package/development/02-playbooks/debugging-playbook.md +692 -0
- package/development/02-playbooks/feature-delivery-playbook.md +430 -0
- package/development/02-playbooks/incident-hotfix-playbook.md +387 -0
- package/development/02-playbooks/performance-optimization-playbook.md +531 -0
- package/development/02-playbooks/performance-tuning-playbook.md +652 -0
- package/development/02-playbooks/refactor-playbook.md +403 -0
- package/development/02-playbooks/release-playbook.md +469 -0
- package/development/03-checklists/architecture-review-checklist.md +168 -0
- package/development/03-checklists/data-migration-checklist.md +157 -0
- package/development/03-checklists/oncall-handover-checklist.md +173 -0
- package/development/03-checklists/pr-checklist.md +158 -0
- package/development/03-checklists/production-readiness-checklist.md +190 -0
- package/development/03-checklists/release-readiness-checklist.md +154 -0
- package/development/03-checklists/security-review-checklist.md +182 -0
- package/development/04-antipatterns/api-antipatterns.md +657 -0
- package/development/04-antipatterns/architecture-antipatterns.md +686 -0
- package/development/04-antipatterns/backend-antipatterns.md +648 -0
- package/development/04-antipatterns/cicd-antipatterns.md +540 -0
- package/development/04-antipatterns/code-smell-antipatterns.md +571 -0
- package/development/04-antipatterns/data-antipatterns.md +658 -0
- package/development/04-antipatterns/database-antipatterns.md +578 -0
- package/development/04-antipatterns/frontend-antipatterns.md +635 -0
- package/development/04-antipatterns/reliability-antipatterns.md +700 -0
- package/development/04-antipatterns/security-antipatterns.md +747 -0
- package/development/05-cases/case-api-version-migration.md +428 -0
- package/development/05-cases/case-authorization-hardening.md +383 -0
- package/development/05-cases/case-bluegreen-rollback.md +466 -0
- package/development/05-cases/case-cache-snowball-protection.md +485 -0
- package/development/05-cases/case-ci-cd-pipeline.md +544 -0
- package/development/05-cases/case-database-scaling.md +500 -0
- package/development/05-cases/case-db-hotspot-optimization.md +487 -0
- package/development/05-cases/case-incident-mttr-reduction.md +563 -0
- package/development/05-cases/case-microservice-migration.md +375 -0
- package/development/05-cases/case-performance-optimization.md +406 -0
- package/development/05-cases/case-security-incident-response.md +345 -0
- package/development/06-glossary/full-stack-glossary.md +166 -0
- package/development/09-maturity/quarterly-audit-template.md +35 -0
- package/development/11-ui-excellence/ui-aesthetic-system.md +41 -0
- package/development/11-ui-excellence/ui-engineering-excellence.md +435 -0
- package/development/12-scenarios/development-scenarios-guide.md +565 -0
- package/development/13-implementation-assets/implementation-toolkit.md +282 -0
- package/development/13-implementation-assets/knowledge-gates-execution.md +43 -0
- package/development/14-full-lifecycle/software-lifecycle-gates.md +511 -0
- package/development/15-lifecycle-templates/project-templates-collection.md +791 -0
- package/development/api-contract-and-versioning-guide.md +36 -0
- package/development/api-governance-complete.md +43 -0
- package/development/backend-engineering-complete.md +43 -0
- package/development/code-review-quality-complete.md +43 -0
- package/development/concurrency-reliability-complete.md +43 -0
- package/development/database-engineering-complete.md +43 -0
- package/development/engineering-effectiveness-complete.md +43 -0
- package/development/engineering-standards-deep-dive.md +38 -0
- package/development/frontend-engineering-complete.md +43 -0
- package/development/performance-capacity-complete.md +43 -0
- package/development/refactor-migration-complete.md +42 -0
- package/development/refactoring-and-techdebt-playbook.md +37 -0
- package/development/security-in-development-complete.md +43 -0
- package/devops/01-standards/cicd-pipeline-complete.md +262 -0
- package/devops/01-standards/docker-complete.md +1490 -0
- package/devops/01-standards/github-actions-complete.md +337 -0
- package/devops/01-standards/kubernetes-complete.md +638 -0
- package/devops/01-standards/terraform-complete.md +2117 -0
- package/devops/02-playbooks/docker-compose-playbook.md +233 -0
- package/devops/02-playbooks/docker-k8s-production-playbook.md +186 -0
- package/devops/02-playbooks/docker-production-playbook.md +952 -0
- package/edge-iot/01-standards/edge-iot-complete.md +473 -0
- package/experts/architect/api-design.md +178 -0
- package/experts/architect/methodology.md +124 -0
- package/experts/architect/security.md +75 -0
- package/experts/backend-lead/methodology.md +216 -0
- package/experts/devops/methodology.md +160 -0
- package/experts/frontend-lead/methodology.md +178 -0
- package/experts/product-manager/industry/ecommerce.md +43 -0
- package/experts/product-manager/industry/saas.md +40 -0
- package/experts/product-manager/methodology.md +97 -0
- package/experts/qa-lead/methodology.md +123 -0
- package/experts/qa-lead/test-strategy.md +128 -0
- package/experts/uiux-designer/methodology.md +125 -0
- package/frontend/01-standards/accessibility-complete.md +532 -0
- package/frontend/01-standards/accessibility-standard.md +74 -0
- package/frontend/01-standards/admin-dashboard-and-crud.md +72 -0
- package/frontend/01-standards/design-tokens-complete.md +444 -0
- package/frontend/01-standards/forms-and-validation.md +77 -0
- package/frontend/01-standards/frontend-architecture-and-layering.md +119 -0
- package/frontend/01-standards/i18n-and-localization.md +65 -0
- package/frontend/01-standards/nextjs-complete.md +451 -0
- package/frontend/01-standards/react-complete.md +713 -0
- package/frontend/01-standards/react-hooks-complete-guide.md +1100 -0
- package/frontend/01-standards/react-hooks-complete.md +1171 -0
- package/frontend/01-standards/seo-and-web-vitals.md +77 -0
- package/frontend/01-standards/state-management-complete.md +444 -0
- package/frontend/01-standards/vue-complete.md +499 -0
- package/frontend/01-standards/vue3-complete.md +2002 -0
- package/frontend/01-standards/web-framework-best-practices.md +64 -0
- package/frontend/01-standards/web-performance-complete.md +495 -0
- package/frontend/02-playbooks/accessibility-a11y-playbook.md +161 -0
- package/frontend/02-playbooks/frontend-performance-playbook.md +707 -0
- package/frontend/02-playbooks/i18n-internationalization-playbook.md +120 -0
- package/frontend/02-playbooks/performance-optimization-playbook.md +163 -0
- package/frontend/02-playbooks/react-nextjs-production-playbook.md +167 -0
- package/frontend/02-playbooks/react-state-management-playbook.md +173 -0
- package/frontend/03-checklists/component-quality-checklist.md +166 -0
- package/frontend/03-checklists/frontend-launch-checklist.md +299 -0
- package/frontend/04-antipatterns/frontend-antipatterns.md +886 -0
- package/frontend/05-cases/case-performance-optimization.md +274 -0
- package/harmony/01-standards/harmonyos-arkts-standard.md +75 -0
- package/harmony/01-standards/harmonyos-design.md +65 -0
- package/high-quality-engineering-playbook.md +54 -0
- package/incident/01-standards/incident-response-complete.md +303 -0
- package/incident/02-playbooks/chaos-engineering-playbook.md +883 -0
- package/incident/02-playbooks/postmortem-playbook.md +398 -0
- package/incident/03-checklists/incident-readiness-checklist.md +181 -0
- package/incident/04-antipatterns/incident-antipatterns.md +490 -0
- package/incident/05-cases/case-cascade-failure.md +176 -0
- package/incident/06-glossary/incident-glossary.md +114 -0
- package/incident/postmortem-and-response-deep-dive.md +39 -0
- package/industries/ecommerce/ecommerce-complete.md +631 -0
- package/industries/education/education-complete.md +555 -0
- package/industries/fintech/fintech-complete.md +501 -0
- package/industries/gaming/gaming-complete.md +587 -0
- package/industries/healthcare/healthcare-complete.md +452 -0
- package/low-code/01-standards/low-code-complete.md +944 -0
- package/miniprogram/01-standards/ai-common-mistakes.md +61 -0
- package/miniprogram/01-standards/miniprogram-custom-navbar-capsule.md +77 -0
- package/miniprogram/01-standards/miniprogram-design.md +61 -0
- package/miniprogram/01-standards/miniprogram-standard.md +81 -0
- package/mobile/01-standards/android-material-design.md +70 -0
- package/mobile/01-standards/flutter-complete.md +384 -0
- package/mobile/01-standards/ios-design-hig.md +78 -0
- package/mobile/01-standards/mobile-app-standard.md +85 -0
- package/mobile/01-standards/react-native-complete.md +352 -0
- package/mobile/02-playbooks/mobile-cross-platform-playbook.md +175 -0
- package/mobile/02-playbooks/mobile-performance.md +473 -0
- package/mobile/03-checklists/mobile-release-checklist.md +234 -0
- package/mobile/04-antipatterns/mobile-antipatterns.md +798 -0
- package/mobile/05-cases/case-app-performance.md +500 -0
- package/mobile/05-cases/case-app-startup-optimization.md +218 -0
- package/mobile/06-glossary/mobile-glossary.md +484 -0
- package/observability/01-standards/observability-standards.md +103 -0
- package/observability/02-playbooks/prometheus-grafana-playbook.md +135 -0
- package/observability/02-playbooks/structured-logging-playbook.md +73 -0
- package/observability/03-checklists/observability-checklist.md +54 -0
- package/observability/04-antipatterns/observability-antipatterns.md +106 -0
- package/operations/01-standards/prometheus-monitoring-complete.md +1578 -0
- package/operations/02-playbooks/capacity-planning-playbook.md +620 -0
- package/operations/03-checklists/production-launch-checklist.md +365 -0
- package/operations/04-antipatterns/operations-antipatterns.md +664 -0
- package/operations/05-cases/case-sre-practices.md +581 -0
- package/operations/06-glossary/operations-glossary.md +120 -0
- package/operations/aiops-anomaly-detection.md +758 -0
- package/operations/capacity-planning.md +1061 -0
- package/operations/chaos-engineering.md +659 -0
- package/operations/incident-command-system.md +38 -0
- package/operations/observability-complete.md +442 -0
- package/operations/slo-sli-playbook.md +517 -0
- package/operations/sre-operations-deep-dive.md +39 -0
- package/package.json +8 -0
- package/performance/01-standards/performance-and-scalability.md +80 -0
- package/performance/01-standards/performance-standards.md +156 -0
- package/performance/02-playbooks/query-optimization-playbook.md +103 -0
- package/performance/03-checklists/performance-checklist.md +56 -0
- package/performance/04-antipatterns/performance-antipatterns.md +146 -0
- package/product/01-standards/product-management-complete.md +285 -0
- package/product/02-playbooks/feature-launch-playbook.md +207 -0
- package/product/02-playbooks/user-research-playbook.md +532 -0
- package/product/03-checklists/feature-launch-checklist.md +275 -0
- package/product/04-antipatterns/product-antipatterns.md +355 -0
- package/product/05-cases/case-mvp-to-scale.md +384 -0
- package/product/06-glossary/product-glossary.md +462 -0
- package/product/feature-prioritization-framework.md +40 -0
- package/product/kpi-and-metric-tree.md +37 -0
- package/product/product-discovery-and-prd-deep-dive.md +41 -0
- package/quantum/01-standards/quantum-complete.md +1186 -0
- package/security/01-standards/api-security-complete.md +511 -0
- package/security/01-standards/container-runtime-security.md +574 -0
- package/security/01-standards/data-protection-gdpr.md +543 -0
- package/security/01-standards/owasp-top10-complete.md +1890 -0
- package/security/01-standards/secure-coding-baseline.md +90 -0
- package/security/01-standards/supply-chain-security.md +441 -0
- package/security/01-standards/web-security-checklist.md +108 -0
- package/security/01-standards/zero-trust-architecture.md +521 -0
- package/security/02-playbooks/auth-sso-playbook.md +166 -0
- package/security/02-playbooks/incident-response-security-playbook.md +588 -0
- package/security/02-playbooks/owasp-api-security-playbook.md +129 -0
- package/security/02-playbooks/payment-integration-playbook.md +119 -0
- package/security/02-playbooks/penetration-testing-playbook.md +517 -0
- package/security/03-checklists/security-audit-checklist.md +356 -0
- package/security/04-antipatterns/security-coding-antipatterns.md +580 -0
- package/security/05-cases/case-log4shell-incident.md +537 -0
- package/security/05-cases/case-major-breaches.md +468 -0
- package/security/06-glossary/security-glossary.md +212 -0
- package/security/compliance-automation.md +993 -0
- package/security/container-security.md +680 -0
- package/security/devsecops-complete.md +426 -0
- package/security/sast-dast-sca.md +775 -0
- package/security/secrets-management.md +594 -0
- package/security/security-architecture-deep-dive.md +37 -0
- package/security/threat-modeling-stride-playbook.md +40 -0
- package/seed-templates/auth-system.md +59 -0
- package/seed-templates/blog-content.md +94 -0
- package/seed-templates/dashboard.md +89 -0
- package/seed-templates/docs-site.md +73 -0
- package/seed-templates/e-commerce.md +50 -0
- package/seed-templates/saas-landing.md +92 -0
- package/seed-templates/settings-page.md +51 -0
- package/testing/01-standards/test-strategy-and-layering.md +83 -0
- package/testing/01-standards/testing-strategy-complete.md +422 -0
- package/testing/01-standards/unit-testing-best-practices.md +118 -0
- package/testing/02-playbooks/e2e-testing-playbook.md +988 -0
- package/testing/02-playbooks/testing-strategy-playbook.md +126 -0
- package/testing/03-checklists/test-strategy-checklist.md +208 -0
- package/testing/04-antipatterns/testing-antipatterns.md +718 -0
- package/testing/05-cases/case-testing-transformation.md +300 -0
- package/testing/06-glossary/testing-glossary.md +110 -0
- package/testing/risk-based-test-matrix.md +36 -0
- package/testing/testing-strategy-deep-dive.md +37 -0
|
@@ -0,0 +1,1527 @@
|
|
|
1
|
+
---
|
|
2
|
+
id: redis-complete
|
|
3
|
+
title: Redis 数据领域完整指南
|
|
4
|
+
domain: data
|
|
5
|
+
category: 01-standards
|
|
6
|
+
difficulty: intermediate
|
|
7
|
+
tags: [complete, data, redis, 分布式锁, 持久化, 核心数据结构, 概述, 消息队列]
|
|
8
|
+
quality_score: 70
|
|
9
|
+
last_updated: 2026-06-15
|
|
10
|
+
---
|
|
11
|
+
# Redis 数据领域完整指南
|
|
12
|
+
|
|
13
|
+
> 文档版本: v1.0 | 最后更新: 2026-03-28
|
|
14
|
+
|
|
15
|
+
## 概述
|
|
16
|
+
|
|
17
|
+
Redis (Remote Dictionary Server) 是一款开源的内存键值数据库,以极低的读写延迟(微秒级)著称。它不仅是缓存层的首选方案,还广泛应用于会话管理、排行榜、实时分析、分布式锁、消息队列等场景。Redis 7.x 支持多线程 I/O、ACL 2.0、Function API 和 Sharded Pub/Sub,在数据领域承担着从缓存加速到流式计算的关键角色。
|
|
18
|
+
|
|
19
|
+
### 核心特性
|
|
20
|
+
|
|
21
|
+
- **内存优先**: 所有数据驻留内存,读写延迟 < 1ms
|
|
22
|
+
- **丰富数据结构**: String / Hash / List / Set / Sorted Set / Stream / HyperLogLog / Bitmap / Geospatial
|
|
23
|
+
- **持久化可选**: RDB 快照 + AOF 日志 + 混合持久化
|
|
24
|
+
- **高可用**: 主从复制 / Sentinel 哨兵 / Cluster 集群
|
|
25
|
+
- **原子操作**: 单线程命令执行模型,天然避免竞态
|
|
26
|
+
- **可扩展**: Lua 脚本、Module API、Function API
|
|
27
|
+
|
|
28
|
+
### 典型数据领域使用场景
|
|
29
|
+
|
|
30
|
+
| 场景 | 数据结构 | 说明 |
|
|
31
|
+
|------|----------|------|
|
|
32
|
+
| ETL 中间缓存 | String / Hash | 加速重复查询,减轻源库压力 |
|
|
33
|
+
| 实时指标聚合 | Sorted Set / HyperLogLog | 排行榜、UV 去重统计 |
|
|
34
|
+
| 流式数据管道 | Stream | 替代轻量级 Kafka 场景 |
|
|
35
|
+
| 数据血缘锁 | String (SET NX EX) | 防止并发血缘刷新冲突 |
|
|
36
|
+
| 特征存储 | Hash | ML 在线特征毫秒级读取 |
|
|
37
|
+
| 数据目录缓存 | String + JSON | 元数据目录热点加速 |
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## 核心数据结构
|
|
42
|
+
|
|
43
|
+
### 1. String(字符串)
|
|
44
|
+
|
|
45
|
+
最基础的数据类型,可存储字符串、整数、浮点数或二进制数据(最大 512MB)。
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
# 基本读写
|
|
49
|
+
SET user:1001 "Alice"
|
|
50
|
+
GET user:1001
|
|
51
|
+
|
|
52
|
+
# 带过期时间
|
|
53
|
+
SET session:abc123 "token_data" EX 3600 # 秒级过期
|
|
54
|
+
SET session:abc123 "token_data" PX 3600000 # 毫秒级过期
|
|
55
|
+
SETEX cache:product:99 300 '{"name":"Widget"}'
|
|
56
|
+
|
|
57
|
+
# 条件写入
|
|
58
|
+
SET lock:order:5001 "owner_a" NX EX 30 # 仅当 key 不存在时写入
|
|
59
|
+
SET counter:daily 100 XX # 仅当 key 存在时写入
|
|
60
|
+
|
|
61
|
+
# 原子自增/自减
|
|
62
|
+
SET counter:pageview 0
|
|
63
|
+
INCR counter:pageview # -> 1
|
|
64
|
+
INCRBY counter:pageview 10 # -> 11
|
|
65
|
+
DECR counter:pageview # -> 10
|
|
66
|
+
INCRBYFLOAT price:item:1 2.5 # 浮点自增
|
|
67
|
+
|
|
68
|
+
# 批量操作
|
|
69
|
+
MSET k1 "v1" k2 "v2" k3 "v3"
|
|
70
|
+
MGET k1 k2 k3
|
|
71
|
+
|
|
72
|
+
# 位操作(适合标记类场景)
|
|
73
|
+
SETBIT user:1001:login 0 1 # 第0天登录
|
|
74
|
+
GETBIT user:1001:login 0 # -> 1
|
|
75
|
+
BITCOUNT user:1001:login # 统计登录天数
|
|
76
|
+
|
|
77
|
+
# 获取子串
|
|
78
|
+
GETRANGE user:1001 0 2 # -> "Ali"
|
|
79
|
+
STRLEN user:1001 # -> 5
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
**Python redis-py 示例:**
|
|
83
|
+
|
|
84
|
+
```python
|
|
85
|
+
import redis
|
|
86
|
+
|
|
87
|
+
r = redis.Redis(host="localhost", port=6379, db=0, decode_responses=True)
|
|
88
|
+
|
|
89
|
+
# 基本操作
|
|
90
|
+
r.set("user:1001", "Alice", ex=3600)
|
|
91
|
+
value = r.get("user:1001") # "Alice"
|
|
92
|
+
|
|
93
|
+
# 原子自增
|
|
94
|
+
r.set("counter:pv", 0)
|
|
95
|
+
r.incr("counter:pv") # 1
|
|
96
|
+
r.incrby("counter:pv", 10) # 11
|
|
97
|
+
|
|
98
|
+
# 批量操作
|
|
99
|
+
r.mset({"k1": "v1", "k2": "v2", "k3": "v3"})
|
|
100
|
+
values = r.mget("k1", "k2", "k3") # ["v1", "v2", "v3"]
|
|
101
|
+
|
|
102
|
+
# 条件写入(分布式锁基础)
|
|
103
|
+
acquired = r.set("lock:order:5001", "owner_a", nx=True, ex=30)
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
### 2. Hash(哈希)
|
|
107
|
+
|
|
108
|
+
适合存储对象,每个 key 下可以有多个 field-value 对。
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
# 设置字段
|
|
112
|
+
HSET user:1001 name "Alice" age 30 email "alice@example.com" role "analyst"
|
|
113
|
+
|
|
114
|
+
# 读取
|
|
115
|
+
HGET user:1001 name # -> "Alice"
|
|
116
|
+
HMGET user:1001 name age # -> ["Alice", "30"]
|
|
117
|
+
HGETALL user:1001 # 返回所有字段
|
|
118
|
+
|
|
119
|
+
# 字段自增
|
|
120
|
+
HINCRBY user:1001 age 1 # -> 31
|
|
121
|
+
HINCRBYFLOAT user:1001 score 0.5
|
|
122
|
+
|
|
123
|
+
# 字段操作
|
|
124
|
+
HDEL user:1001 role
|
|
125
|
+
HEXISTS user:1001 name # -> 1
|
|
126
|
+
HLEN user:1001 # 字段数量
|
|
127
|
+
HKEYS user:1001 # 所有字段名
|
|
128
|
+
HVALS user:1001 # 所有字段值
|
|
129
|
+
|
|
130
|
+
# 扫描大 Hash(生产环境推荐)
|
|
131
|
+
HSCAN user:1001 0 MATCH "na*" COUNT 100
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
**Python redis-py 示例:**
|
|
135
|
+
|
|
136
|
+
```python
|
|
137
|
+
# 对象存储
|
|
138
|
+
r.hset("user:1001", mapping={
|
|
139
|
+
"name": "Alice",
|
|
140
|
+
"age": 30,
|
|
141
|
+
"email": "alice@example.com",
|
|
142
|
+
})
|
|
143
|
+
|
|
144
|
+
user = r.hgetall("user:1001")
|
|
145
|
+
# {"name": "Alice", "age": "30", "email": "alice@example.com"}
|
|
146
|
+
|
|
147
|
+
r.hincrby("user:1001", "age", 1) # 31
|
|
148
|
+
exists = r.hexists("user:1001", "name") # True
|
|
149
|
+
|
|
150
|
+
# 特征存储示例
|
|
151
|
+
r.hset("feature:user:1001", mapping={
|
|
152
|
+
"click_rate": "0.032",
|
|
153
|
+
"avg_session": "245",
|
|
154
|
+
"last_active": "2026-03-28",
|
|
155
|
+
})
|
|
156
|
+
features = r.hmget("feature:user:1001", "click_rate", "avg_session")
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
### 3. List(列表)
|
|
160
|
+
|
|
161
|
+
双向链表,支持头/尾插入和弹出,适合队列和栈场景。
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
# 左/右推入
|
|
165
|
+
LPUSH queue:tasks "task1" "task2" "task3"
|
|
166
|
+
RPUSH queue:tasks "task4"
|
|
167
|
+
|
|
168
|
+
# 弹出
|
|
169
|
+
LPOP queue:tasks # -> "task3"(LIFO 栈)
|
|
170
|
+
RPOP queue:tasks # -> "task4"(FIFO 队列)
|
|
171
|
+
|
|
172
|
+
# 阻塞弹出(消费者模式)
|
|
173
|
+
BLPOP queue:tasks 30 # 阻塞等待最多30秒
|
|
174
|
+
BRPOP queue:tasks 30
|
|
175
|
+
|
|
176
|
+
# 范围查询
|
|
177
|
+
LRANGE queue:tasks 0 -1 # 所有元素
|
|
178
|
+
LRANGE queue:tasks 0 9 # 前10个
|
|
179
|
+
|
|
180
|
+
# 长度和索引
|
|
181
|
+
LLEN queue:tasks
|
|
182
|
+
LINDEX queue:tasks 0 # 第一个元素
|
|
183
|
+
|
|
184
|
+
# 修剪(保留最近N条)
|
|
185
|
+
LTRIM queue:tasks 0 99 # 只保留前100个
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
**Python redis-py 示例:**
|
|
189
|
+
|
|
190
|
+
```python
|
|
191
|
+
# 简单任务队列
|
|
192
|
+
r.lpush("queue:tasks", "task1", "task2", "task3")
|
|
193
|
+
|
|
194
|
+
# 消费者
|
|
195
|
+
task = r.brpop("queue:tasks", timeout=30)
|
|
196
|
+
# ("queue:tasks", "task1")
|
|
197
|
+
|
|
198
|
+
# 保留最近100条日志
|
|
199
|
+
r.lpush("log:app", "2026-03-28 10:00 INFO started")
|
|
200
|
+
r.ltrim("log:app", 0, 99)
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
### 4. Set(集合)
|
|
204
|
+
|
|
205
|
+
无序不重复集合,支持集合运算。
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
# 添加成员
|
|
209
|
+
SADD tags:article:1 "python" "redis" "database"
|
|
210
|
+
SADD tags:article:2 "python" "django" "web"
|
|
211
|
+
|
|
212
|
+
# 查询
|
|
213
|
+
SMEMBERS tags:article:1 # 所有成员
|
|
214
|
+
SISMEMBER tags:article:1 "redis" # -> 1
|
|
215
|
+
SCARD tags:article:1 # 成员数量
|
|
216
|
+
|
|
217
|
+
# 集合运算
|
|
218
|
+
SINTER tags:article:1 tags:article:2 # 交集 -> {"python"}
|
|
219
|
+
SUNION tags:article:1 tags:article:2 # 并集
|
|
220
|
+
SDIFF tags:article:1 tags:article:2 # 差集
|
|
221
|
+
|
|
222
|
+
# 随机成员
|
|
223
|
+
SRANDMEMBER tags:article:1 2 # 随机取2个
|
|
224
|
+
SPOP tags:article:1 # 随机弹出1个
|
|
225
|
+
|
|
226
|
+
# 删除
|
|
227
|
+
SREM tags:article:1 "database"
|
|
228
|
+
```
|
|
229
|
+
|
|
230
|
+
**Python redis-py 示例:**
|
|
231
|
+
|
|
232
|
+
```python
|
|
233
|
+
# 标签系统
|
|
234
|
+
r.sadd("tags:article:1", "python", "redis", "database")
|
|
235
|
+
r.sadd("tags:article:2", "python", "django", "web")
|
|
236
|
+
|
|
237
|
+
common_tags = r.sinter("tags:article:1", "tags:article:2")
|
|
238
|
+
# {"python"}
|
|
239
|
+
|
|
240
|
+
all_tags = r.sunion("tags:article:1", "tags:article:2")
|
|
241
|
+
# {"python", "redis", "database", "django", "web"}
|
|
242
|
+
|
|
243
|
+
is_member = r.sismember("tags:article:1", "redis") # True
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
### 5. Sorted Set(有序集合 / ZSet)
|
|
247
|
+
|
|
248
|
+
带分值的有序集合,按 score 排序,适合排行榜和范围查询。
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
# 添加(score + member)
|
|
252
|
+
ZADD leaderboard 100 "alice" 95 "bob" 88 "charlie" 120 "diana"
|
|
253
|
+
|
|
254
|
+
# 排名查询(升序,0-based)
|
|
255
|
+
ZRANK leaderboard "alice" # -> 2
|
|
256
|
+
ZREVRANK leaderboard "alice" # -> 1(降序排名)
|
|
257
|
+
|
|
258
|
+
# 分值查询
|
|
259
|
+
ZSCORE leaderboard "alice" # -> 100
|
|
260
|
+
|
|
261
|
+
# 范围查询
|
|
262
|
+
ZRANGE leaderboard 0 2 # 升序前3
|
|
263
|
+
ZREVRANGE leaderboard 0 2 WITHSCORES # 降序前3(带分值)
|
|
264
|
+
ZRANGEBYSCORE leaderboard 90 110 # 分值 90-110
|
|
265
|
+
ZRANGEBYSCORE leaderboard -inf +inf LIMIT 0 10 # 分页
|
|
266
|
+
|
|
267
|
+
# 分值自增
|
|
268
|
+
ZINCRBY leaderboard 5 "alice" # -> 105
|
|
269
|
+
|
|
270
|
+
# 集合运算
|
|
271
|
+
ZUNIONSTORE combined 2 leaderboard:week1 leaderboard:week2 WEIGHTS 1 2
|
|
272
|
+
ZINTERSTORE overlap 2 set1 set2
|
|
273
|
+
|
|
274
|
+
# 删除
|
|
275
|
+
ZREM leaderboard "charlie"
|
|
276
|
+
ZREMRANGEBYRANK leaderboard 0 0 # 删除最低分
|
|
277
|
+
ZREMRANGEBYSCORE leaderboard 0 60 # 删除低于60分的
|
|
278
|
+
|
|
279
|
+
# 数量统计
|
|
280
|
+
ZCARD leaderboard
|
|
281
|
+
ZCOUNT leaderboard 90 110 # 分值在 90-110 之间的数量
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
**Python redis-py 示例:**
|
|
285
|
+
|
|
286
|
+
```python
|
|
287
|
+
# 实时排行榜
|
|
288
|
+
r.zadd("leaderboard", {"alice": 100, "bob": 95, "charlie": 88, "diana": 120})
|
|
289
|
+
|
|
290
|
+
# Top 3(降序)
|
|
291
|
+
top3 = r.zrevrange("leaderboard", 0, 2, withscores=True)
|
|
292
|
+
# [("diana", 120.0), ("alice", 100.0), ("bob", 95.0)]
|
|
293
|
+
|
|
294
|
+
# 用户排名
|
|
295
|
+
rank = r.zrevrank("leaderboard", "alice") # 1
|
|
296
|
+
|
|
297
|
+
# 分值自增
|
|
298
|
+
r.zincrby("leaderboard", 5, "alice") # 105.0
|
|
299
|
+
|
|
300
|
+
# 分页查询
|
|
301
|
+
page = r.zrevrangebyscore("leaderboard", "+inf", "-inf", start=0, num=10, withscores=True)
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
### 6. Stream(流)
|
|
305
|
+
|
|
306
|
+
Redis 5.0+ 引入的日志型数据结构,支持消费者组,适合事件流和消息队列。
|
|
307
|
+
|
|
308
|
+
```bash
|
|
309
|
+
# 写入消息
|
|
310
|
+
XADD events:orders * action "created" order_id "5001" amount "99.50"
|
|
311
|
+
XADD events:orders * action "paid" order_id "5001" payment "stripe"
|
|
312
|
+
|
|
313
|
+
# 读取
|
|
314
|
+
XLEN events:orders
|
|
315
|
+
XRANGE events:orders - + # 全部
|
|
316
|
+
XRANGE events:orders - + COUNT 10 # 前10条
|
|
317
|
+
XREVRANGE events:orders + - COUNT 5 # 最近5条
|
|
318
|
+
|
|
319
|
+
# 消费者组
|
|
320
|
+
XGROUP CREATE events:orders cg_analytics $ MKSTREAM
|
|
321
|
+
XREADGROUP GROUP cg_analytics consumer1 COUNT 10 BLOCK 5000 STREAMS events:orders >
|
|
322
|
+
|
|
323
|
+
# 确认消费
|
|
324
|
+
XACK events:orders cg_analytics 1679012345678-0
|
|
325
|
+
|
|
326
|
+
# 查看待处理消息
|
|
327
|
+
XPENDING events:orders cg_analytics - + 10
|
|
328
|
+
|
|
329
|
+
# 裁剪(保留最近1000条)
|
|
330
|
+
XTRIM events:orders MAXLEN ~ 1000
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
**Python redis-py 示例:**
|
|
334
|
+
|
|
335
|
+
```python
|
|
336
|
+
# 生产者
|
|
337
|
+
msg_id = r.xadd("events:orders", {
|
|
338
|
+
"action": "created",
|
|
339
|
+
"order_id": "5001",
|
|
340
|
+
"amount": "99.50",
|
|
341
|
+
})
|
|
342
|
+
|
|
343
|
+
# 创建消费者组
|
|
344
|
+
try:
|
|
345
|
+
r.xgroup_create("events:orders", "cg_analytics", id="$", mkstream=True)
|
|
346
|
+
except redis.exceptions.ResponseError:
|
|
347
|
+
pass # 组已存在
|
|
348
|
+
|
|
349
|
+
# 消费者
|
|
350
|
+
messages = r.xreadgroup(
|
|
351
|
+
groupname="cg_analytics",
|
|
352
|
+
consumername="consumer1",
|
|
353
|
+
streams={"events:orders": ">"},
|
|
354
|
+
count=10,
|
|
355
|
+
block=5000,
|
|
356
|
+
)
|
|
357
|
+
|
|
358
|
+
for stream, msgs in messages:
|
|
359
|
+
for msg_id, fields in msgs:
|
|
360
|
+
print(f"Processing {msg_id}: {fields}")
|
|
361
|
+
r.xack("events:orders", "cg_analytics", msg_id)
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
### 7. HyperLogLog
|
|
365
|
+
|
|
366
|
+
概率数据结构,用极小内存(12KB)估算集合基数(误差 < 0.81%),适合 UV 统计。
|
|
367
|
+
|
|
368
|
+
```bash
|
|
369
|
+
# 添加元素
|
|
370
|
+
PFADD uv:2026-03-28 "user1" "user2" "user3" "user1"
|
|
371
|
+
|
|
372
|
+
# 估算基数
|
|
373
|
+
PFCOUNT uv:2026-03-28 # -> 3
|
|
374
|
+
|
|
375
|
+
# 合并多天
|
|
376
|
+
PFMERGE uv:week uv:2026-03-28 uv:2026-03-27 uv:2026-03-26
|
|
377
|
+
PFCOUNT uv:week
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
**Python redis-py 示例:**
|
|
381
|
+
|
|
382
|
+
```python
|
|
383
|
+
# 日活统计
|
|
384
|
+
r.pfadd("uv:2026-03-28", "user1", "user2", "user3", "user1")
|
|
385
|
+
daily_uv = r.pfcount("uv:2026-03-28") # 3
|
|
386
|
+
|
|
387
|
+
# 周活统计
|
|
388
|
+
keys = [f"uv:2026-03-{d}" for d in range(22, 29)]
|
|
389
|
+
r.pfmerge("uv:week", *keys)
|
|
390
|
+
weekly_uv = r.pfcount("uv:week")
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
### 8. Bitmap(位图)
|
|
394
|
+
|
|
395
|
+
基于 String 的位操作,极致节省空间,适合布尔状态标记。
|
|
396
|
+
|
|
397
|
+
```bash
|
|
398
|
+
# 用户签到(day_offset 从年初算)
|
|
399
|
+
SETBIT sign:user:1001:2026 86 1 # 第87天签到
|
|
400
|
+
GETBIT sign:user:1001:2026 86 # -> 1
|
|
401
|
+
|
|
402
|
+
# 统计签到天数
|
|
403
|
+
BITCOUNT sign:user:1001:2026
|
|
404
|
+
|
|
405
|
+
# 位运算(统计连续签到)
|
|
406
|
+
BITOP AND result sign:user:1001:2026 sign:user:1002:2026
|
|
407
|
+
BITPOS sign:user:1001:2026 0 # 第一个未签到的天
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
**Python redis-py 示例:**
|
|
411
|
+
|
|
412
|
+
```python
|
|
413
|
+
# 用户签到系统
|
|
414
|
+
day_of_year = 86 # 3月28日大约是第87天
|
|
415
|
+
r.setbit("sign:user:1001:2026", day_of_year, 1)
|
|
416
|
+
|
|
417
|
+
# 统计本年签到天数
|
|
418
|
+
total_sign = r.bitcount("sign:user:1001:2026")
|
|
419
|
+
|
|
420
|
+
# 检查某天是否签到
|
|
421
|
+
signed = r.getbit("sign:user:1001:2026", day_of_year) # 1
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
---
|
|
425
|
+
|
|
426
|
+
## 持久化
|
|
427
|
+
|
|
428
|
+
### RDB(快照)
|
|
429
|
+
|
|
430
|
+
将内存数据以二进制快照形式写入磁盘。
|
|
431
|
+
|
|
432
|
+
```bash
|
|
433
|
+
# redis.conf 配置
|
|
434
|
+
save 900 1 # 900秒内至少1次写入则触发
|
|
435
|
+
save 300 10 # 300秒内至少10次写入
|
|
436
|
+
save 60 10000 # 60秒内至少10000次写入
|
|
437
|
+
|
|
438
|
+
dbfilename dump.rdb
|
|
439
|
+
dir /var/lib/redis/
|
|
440
|
+
|
|
441
|
+
# 手动触发
|
|
442
|
+
BGSAVE # 后台异步保存(推荐)
|
|
443
|
+
SAVE # 同步保存(阻塞,生产禁用)
|
|
444
|
+
LASTSAVE # 最近一次成功保存时间
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
**优点**: 文件紧凑,恢复速度快,适合灾备。
|
|
448
|
+
**缺点**: 可能丢失最近一次快照后的数据。
|
|
449
|
+
|
|
450
|
+
### AOF(追加日志)
|
|
451
|
+
|
|
452
|
+
将每条写命令追加到日志文件。
|
|
453
|
+
|
|
454
|
+
```bash
|
|
455
|
+
# redis.conf 配置
|
|
456
|
+
appendonly yes
|
|
457
|
+
appendfilename "appendonly.aof"
|
|
458
|
+
appenddirname "appendonlydir"
|
|
459
|
+
|
|
460
|
+
# 刷盘策略
|
|
461
|
+
appendfsync always # 每条命令刷盘(最安全,最慢)
|
|
462
|
+
appendfsync everysec # 每秒刷盘(推荐,最多丢1秒)
|
|
463
|
+
appendfsync no # OS 决定(最快,最不安全)
|
|
464
|
+
|
|
465
|
+
# AOF 重写(压缩日志)
|
|
466
|
+
BGREWRITEAOF
|
|
467
|
+
|
|
468
|
+
# 自动重写阈值
|
|
469
|
+
auto-aof-rewrite-percentage 100
|
|
470
|
+
auto-aof-rewrite-min-size 64mb
|
|
471
|
+
```
|
|
472
|
+
|
|
473
|
+
**优点**: 数据丢失少(everysec 最多丢1秒),日志可读。
|
|
474
|
+
**缺点**: 文件比 RDB 大,恢复速度较慢。
|
|
475
|
+
|
|
476
|
+
### 混合持久化(Redis 4.0+,推荐)
|
|
477
|
+
|
|
478
|
+
RDB + AOF 结合:重写时先写 RDB 格式,之后增量命令用 AOF 格式追加。
|
|
479
|
+
|
|
480
|
+
```bash
|
|
481
|
+
# redis.conf
|
|
482
|
+
aof-use-rdb-preamble yes # 默认开启
|
|
483
|
+
appendonly yes
|
|
484
|
+
```
|
|
485
|
+
|
|
486
|
+
**优点**: 兼顾快速恢复(RDB 部分)和低数据丢失(AOF 增量)。
|
|
487
|
+
|
|
488
|
+
**Python 验证持久化状态:**
|
|
489
|
+
|
|
490
|
+
```python
|
|
491
|
+
info = r.info("persistence")
|
|
492
|
+
print(f"RDB 上次保存: {info['rdb_last_save_time']}")
|
|
493
|
+
print(f"RDB 状态: {info['rdb_last_bgsave_status']}")
|
|
494
|
+
print(f"AOF 开启: {info['aof_enabled']}")
|
|
495
|
+
print(f"AOF 重写进行中: {info['aof_rewrite_in_progress']}")
|
|
496
|
+
```
|
|
497
|
+
|
|
498
|
+
---
|
|
499
|
+
|
|
500
|
+
## 高可用
|
|
501
|
+
|
|
502
|
+
### 主从复制
|
|
503
|
+
|
|
504
|
+
```bash
|
|
505
|
+
# 从节点配置
|
|
506
|
+
# redis.conf
|
|
507
|
+
replicaof 192.168.1.100 6379
|
|
508
|
+
masterauth "your_master_password"
|
|
509
|
+
|
|
510
|
+
# 动态设置
|
|
511
|
+
REPLICAOF 192.168.1.100 6379
|
|
512
|
+
REPLICAOF NO ONE # 提升为主节点
|
|
513
|
+
|
|
514
|
+
# 查看复制状态
|
|
515
|
+
INFO replication
|
|
516
|
+
```
|
|
517
|
+
|
|
518
|
+
**关键参数:**
|
|
519
|
+
|
|
520
|
+
```bash
|
|
521
|
+
repl-backlog-size 256mb # 复制积压缓冲区
|
|
522
|
+
repl-diskless-sync yes # 无盘复制(网络快于磁盘时开启)
|
|
523
|
+
min-replicas-to-write 1 # 至少1个从节点确认写入
|
|
524
|
+
min-replicas-max-lag 10 # 从节点最大延迟10秒
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
### Sentinel(哨兵)
|
|
528
|
+
|
|
529
|
+
自动故障转移,适合中小规模部署。
|
|
530
|
+
|
|
531
|
+
```bash
|
|
532
|
+
# sentinel.conf
|
|
533
|
+
sentinel monitor mymaster 192.168.1.100 6379 2 # 2个哨兵同意才切换
|
|
534
|
+
sentinel down-after-milliseconds mymaster 5000 # 5秒无响应判定下线
|
|
535
|
+
sentinel failover-timeout mymaster 60000 # 故障转移超时60秒
|
|
536
|
+
sentinel parallel-syncs mymaster 1 # 切换后同时同步的从节点数
|
|
537
|
+
|
|
538
|
+
sentinel auth-pass mymaster "your_password"
|
|
539
|
+
```
|
|
540
|
+
|
|
541
|
+
**Python redis-py 连接 Sentinel:**
|
|
542
|
+
|
|
543
|
+
```python
|
|
544
|
+
from redis.sentinel import Sentinel
|
|
545
|
+
|
|
546
|
+
sentinel = Sentinel(
|
|
547
|
+
[("sentinel1", 26379), ("sentinel2", 26379), ("sentinel3", 26379)],
|
|
548
|
+
socket_timeout=0.5,
|
|
549
|
+
)
|
|
550
|
+
|
|
551
|
+
# 获取主节点连接
|
|
552
|
+
master = sentinel.master_for("mymaster", socket_timeout=0.5, password="your_password")
|
|
553
|
+
master.set("key", "value")
|
|
554
|
+
|
|
555
|
+
# 获取从节点连接(读操作)
|
|
556
|
+
slave = sentinel.slave_for("mymaster", socket_timeout=0.5, password="your_password")
|
|
557
|
+
value = slave.get("key")
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
### Cluster(集群)
|
|
561
|
+
|
|
562
|
+
数据分片(16384 个 slot),适合大规模高可用部署。
|
|
563
|
+
|
|
564
|
+
```bash
|
|
565
|
+
# 创建集群(至少6个节点:3主3从)
|
|
566
|
+
redis-cli --cluster create \
|
|
567
|
+
192.168.1.101:6379 192.168.1.102:6379 192.168.1.103:6379 \
|
|
568
|
+
192.168.1.104:6379 192.168.1.105:6379 192.168.1.106:6379 \
|
|
569
|
+
--cluster-replicas 1
|
|
570
|
+
|
|
571
|
+
# 查看集群信息
|
|
572
|
+
redis-cli -c -h 192.168.1.101 -p 6379
|
|
573
|
+
CLUSTER INFO
|
|
574
|
+
CLUSTER NODES
|
|
575
|
+
CLUSTER SLOTS
|
|
576
|
+
|
|
577
|
+
# 添加节点
|
|
578
|
+
redis-cli --cluster add-node 192.168.1.107:6379 192.168.1.101:6379
|
|
579
|
+
|
|
580
|
+
# 重新分片
|
|
581
|
+
redis-cli --cluster reshard 192.168.1.101:6379
|
|
582
|
+
|
|
583
|
+
# 检查集群健康
|
|
584
|
+
redis-cli --cluster check 192.168.1.101:6379
|
|
585
|
+
```
|
|
586
|
+
|
|
587
|
+
**Python redis-py 集群连接:**
|
|
588
|
+
|
|
589
|
+
```python
|
|
590
|
+
from redis.cluster import RedisCluster
|
|
591
|
+
|
|
592
|
+
rc = RedisCluster(
|
|
593
|
+
host="192.168.1.101",
|
|
594
|
+
port=6379,
|
|
595
|
+
password="your_password",
|
|
596
|
+
decode_responses=True,
|
|
597
|
+
)
|
|
598
|
+
|
|
599
|
+
rc.set("key", "value")
|
|
600
|
+
value = rc.get("key")
|
|
601
|
+
|
|
602
|
+
# 注意:集群模式下多 key 操作需要 Hash Tag
|
|
603
|
+
rc.mset({"{user:1001}.name": "Alice", "{user:1001}.age": "30"})
|
|
604
|
+
```
|
|
605
|
+
|
|
606
|
+
---
|
|
607
|
+
|
|
608
|
+
## 缓存策略
|
|
609
|
+
|
|
610
|
+
### Cache-Aside(旁路缓存,最常用)
|
|
611
|
+
|
|
612
|
+
```python
|
|
613
|
+
def get_user(user_id: int) -> dict:
|
|
614
|
+
cache_key = f"user:{user_id}"
|
|
615
|
+
|
|
616
|
+
# 1. 先查缓存
|
|
617
|
+
cached = r.get(cache_key)
|
|
618
|
+
if cached:
|
|
619
|
+
return json.loads(cached)
|
|
620
|
+
|
|
621
|
+
# 2. 缓存未命中,查数据库
|
|
622
|
+
user = db.query("SELECT * FROM users WHERE id = %s", (user_id,))
|
|
623
|
+
if user is None:
|
|
624
|
+
# 防止缓存穿透:缓存空值(短TTL)
|
|
625
|
+
r.set(cache_key, json.dumps(None), ex=60)
|
|
626
|
+
return None
|
|
627
|
+
|
|
628
|
+
# 3. 写入缓存
|
|
629
|
+
r.set(cache_key, json.dumps(user), ex=3600)
|
|
630
|
+
return user
|
|
631
|
+
|
|
632
|
+
def update_user(user_id: int, data: dict):
|
|
633
|
+
# 1. 先更新数据库
|
|
634
|
+
db.execute("UPDATE users SET ... WHERE id = %s", (user_id,))
|
|
635
|
+
# 2. 再删除缓存(而非更新缓存,避免并发不一致)
|
|
636
|
+
r.delete(f"user:{user_id}")
|
|
637
|
+
```
|
|
638
|
+
|
|
639
|
+
### Write-Through(写穿透)
|
|
640
|
+
|
|
641
|
+
```python
|
|
642
|
+
def write_through_set(key: str, value: str, ttl: int = 3600):
|
|
643
|
+
"""写入时同时更新缓存和数据库"""
|
|
644
|
+
# 1. 更新数据库
|
|
645
|
+
db.execute("INSERT INTO kv_store (key, value) VALUES (%s, %s) "
|
|
646
|
+
"ON CONFLICT (key) DO UPDATE SET value = %s", (key, value, value))
|
|
647
|
+
# 2. 更新缓存
|
|
648
|
+
r.set(key, value, ex=ttl)
|
|
649
|
+
```
|
|
650
|
+
|
|
651
|
+
### Write-Behind(写回,异步写入数据库)
|
|
652
|
+
|
|
653
|
+
```python
|
|
654
|
+
def write_behind_set(key: str, value: str):
|
|
655
|
+
"""先写缓存,异步写数据库"""
|
|
656
|
+
r.set(key, value, ex=3600)
|
|
657
|
+
# 将写操作放入队列,异步批量写入数据库
|
|
658
|
+
r.lpush("queue:db_writes", json.dumps({"key": key, "value": value}))
|
|
659
|
+
|
|
660
|
+
# 异步消费者(独立进程)
|
|
661
|
+
def db_write_consumer():
|
|
662
|
+
while True:
|
|
663
|
+
item = r.brpop("queue:db_writes", timeout=5)
|
|
664
|
+
if item:
|
|
665
|
+
data = json.loads(item[1])
|
|
666
|
+
db.execute("UPSERT ...", (data["key"], data["value"]))
|
|
667
|
+
```
|
|
668
|
+
|
|
669
|
+
### TTL 策略
|
|
670
|
+
|
|
671
|
+
```python
|
|
672
|
+
# 分级 TTL 策略
|
|
673
|
+
TTL_CONFIG = {
|
|
674
|
+
"hot_data": 300, # 热点数据:5分钟
|
|
675
|
+
"warm_data": 3600, # 温数据:1小时
|
|
676
|
+
"cold_data": 86400, # 冷数据:1天
|
|
677
|
+
"static_data": 604800, # 静态数据:7天
|
|
678
|
+
}
|
|
679
|
+
|
|
680
|
+
# 带随机偏移的 TTL(防止缓存雪崩)
|
|
681
|
+
import random
|
|
682
|
+
|
|
683
|
+
def set_with_jitter(key: str, value: str, base_ttl: int):
|
|
684
|
+
jitter = random.randint(0, int(base_ttl * 0.1)) # 10% 随机偏移
|
|
685
|
+
r.set(key, value, ex=base_ttl + jitter)
|
|
686
|
+
```
|
|
687
|
+
|
|
688
|
+
### 缓存穿透
|
|
689
|
+
|
|
690
|
+
查询不存在的数据,请求直接打到数据库。
|
|
691
|
+
|
|
692
|
+
```python
|
|
693
|
+
# 方案1:缓存空值
|
|
694
|
+
def get_with_null_cache(key: str):
|
|
695
|
+
cached = r.get(key)
|
|
696
|
+
if cached == "NULL":
|
|
697
|
+
return None
|
|
698
|
+
if cached:
|
|
699
|
+
return json.loads(cached)
|
|
700
|
+
|
|
701
|
+
result = db.query(key)
|
|
702
|
+
if result is None:
|
|
703
|
+
r.set(key, "NULL", ex=60) # 短 TTL 缓存空值
|
|
704
|
+
return None
|
|
705
|
+
|
|
706
|
+
r.set(key, json.dumps(result), ex=3600)
|
|
707
|
+
return result
|
|
708
|
+
|
|
709
|
+
# 方案2:布隆过滤器(Redis Stack / RedisBloom)
|
|
710
|
+
# BF.RESERVE valid_ids 0.001 1000000
|
|
711
|
+
# BF.ADD valid_ids "id_1001"
|
|
712
|
+
# BF.EXISTS valid_ids "id_9999" -> 0 (不存在,直接拒绝)
|
|
713
|
+
```
|
|
714
|
+
|
|
715
|
+
### 缓存雪崩
|
|
716
|
+
|
|
717
|
+
大量 key 同时过期,请求同时打到数据库。
|
|
718
|
+
|
|
719
|
+
```python
|
|
720
|
+
# 方案1:TTL 随机化(见上方 set_with_jitter)
|
|
721
|
+
|
|
722
|
+
# 方案2:互斥锁 + 异步刷新
|
|
723
|
+
def get_with_mutex(key: str):
|
|
724
|
+
cached = r.get(key)
|
|
725
|
+
if cached:
|
|
726
|
+
return json.loads(cached)
|
|
727
|
+
|
|
728
|
+
lock_key = f"lock:{key}"
|
|
729
|
+
if r.set(lock_key, "1", nx=True, ex=10):
|
|
730
|
+
try:
|
|
731
|
+
result = db.query(key)
|
|
732
|
+
r.set(key, json.dumps(result), ex=3600)
|
|
733
|
+
return result
|
|
734
|
+
finally:
|
|
735
|
+
r.delete(lock_key)
|
|
736
|
+
else:
|
|
737
|
+
# 其他线程正在刷新,等待后重试
|
|
738
|
+
import time
|
|
739
|
+
time.sleep(0.1)
|
|
740
|
+
return get_with_mutex(key)
|
|
741
|
+
```
|
|
742
|
+
|
|
743
|
+
### 缓存击穿
|
|
744
|
+
|
|
745
|
+
单个热点 key 过期,大量并发请求同时打到数据库。
|
|
746
|
+
|
|
747
|
+
```python
|
|
748
|
+
# 方案1:互斥锁(同上 get_with_mutex)
|
|
749
|
+
|
|
750
|
+
# 方案2:逻辑过期(永不真正过期)
|
|
751
|
+
import time
|
|
752
|
+
|
|
753
|
+
def set_with_logical_expiry(key: str, value: str, ttl: int):
|
|
754
|
+
data = {"value": value, "expire_at": time.time() + ttl}
|
|
755
|
+
r.set(key, json.dumps(data)) # 不设 Redis TTL
|
|
756
|
+
|
|
757
|
+
def get_with_logical_expiry(key: str):
|
|
758
|
+
cached = r.get(key)
|
|
759
|
+
if not cached:
|
|
760
|
+
return None
|
|
761
|
+
|
|
762
|
+
data = json.loads(cached)
|
|
763
|
+
if time.time() < data["expire_at"]:
|
|
764
|
+
return data["value"]
|
|
765
|
+
|
|
766
|
+
# 已逻辑过期,异步刷新
|
|
767
|
+
lock_key = f"lock:refresh:{key}"
|
|
768
|
+
if r.set(lock_key, "1", nx=True, ex=10):
|
|
769
|
+
# 触发异步刷新(线程池/消息队列)
|
|
770
|
+
import threading
|
|
771
|
+
threading.Thread(target=refresh_cache, args=(key,)).start()
|
|
772
|
+
|
|
773
|
+
# 返回过期数据(而非 None)
|
|
774
|
+
return data["value"]
|
|
775
|
+
```
|
|
776
|
+
|
|
777
|
+
---
|
|
778
|
+
|
|
779
|
+
## 分布式锁
|
|
780
|
+
|
|
781
|
+
### SET NX EX(单节点锁)
|
|
782
|
+
|
|
783
|
+
```python
|
|
784
|
+
import uuid
|
|
785
|
+
import time
|
|
786
|
+
|
|
787
|
+
def acquire_lock(lock_name: str, timeout: int = 10) -> str | None:
|
|
788
|
+
"""获取分布式锁,返回 token 用于安全释放"""
|
|
789
|
+
token = str(uuid.uuid4())
|
|
790
|
+
acquired = r.set(f"lock:{lock_name}", token, nx=True, ex=timeout)
|
|
791
|
+
return token if acquired else None
|
|
792
|
+
|
|
793
|
+
# Lua 脚本安全释放(原子操作,防止误删他人的锁)
|
|
794
|
+
RELEASE_LOCK_SCRIPT = """
|
|
795
|
+
if redis.call('get', KEYS[1]) == ARGV[1] then
|
|
796
|
+
return redis.call('del', KEYS[1])
|
|
797
|
+
else
|
|
798
|
+
return 0
|
|
799
|
+
end
|
|
800
|
+
"""
|
|
801
|
+
|
|
802
|
+
def release_lock(lock_name: str, token: str) -> bool:
|
|
803
|
+
result = r.eval(RELEASE_LOCK_SCRIPT, 1, f"lock:{lock_name}", token)
|
|
804
|
+
return result == 1
|
|
805
|
+
|
|
806
|
+
# 使用示例
|
|
807
|
+
token = acquire_lock("order:5001", timeout=30)
|
|
808
|
+
if token:
|
|
809
|
+
try:
|
|
810
|
+
# 业务逻辑
|
|
811
|
+
process_order(5001)
|
|
812
|
+
finally:
|
|
813
|
+
release_lock("order:5001", token)
|
|
814
|
+
```
|
|
815
|
+
|
|
816
|
+
### Redlock(多节点锁,Martin Kleppmann 有争议但广泛使用)
|
|
817
|
+
|
|
818
|
+
```python
|
|
819
|
+
# pip install redis
|
|
820
|
+
# 需要至少5个独立 Redis 实例(非集群节点)
|
|
821
|
+
|
|
822
|
+
import time
|
|
823
|
+
import uuid
|
|
824
|
+
|
|
825
|
+
REDIS_INSTANCES = [
|
|
826
|
+
redis.Redis(host=f"redis{i}", port=6379) for i in range(1, 6)
|
|
827
|
+
]
|
|
828
|
+
QUORUM = len(REDIS_INSTANCES) // 2 + 1 # 3
|
|
829
|
+
|
|
830
|
+
def redlock_acquire(resource: str, ttl_ms: int = 10000) -> str | None:
|
|
831
|
+
token = str(uuid.uuid4())
|
|
832
|
+
start_time = time.monotonic()
|
|
833
|
+
acquired_count = 0
|
|
834
|
+
|
|
835
|
+
for instance in REDIS_INSTANCES:
|
|
836
|
+
try:
|
|
837
|
+
if instance.set(f"lock:{resource}", token, nx=True, px=ttl_ms):
|
|
838
|
+
acquired_count += 1
|
|
839
|
+
except redis.exceptions.ConnectionError:
|
|
840
|
+
continue
|
|
841
|
+
|
|
842
|
+
elapsed_ms = (time.monotonic() - start_time) * 1000
|
|
843
|
+
# 检查:多数节点获取成功 且 耗时未超过 TTL
|
|
844
|
+
if acquired_count >= QUORUM and elapsed_ms < ttl_ms:
|
|
845
|
+
return token
|
|
846
|
+
|
|
847
|
+
# 获取失败,释放所有已获取的锁
|
|
848
|
+
for instance in REDIS_INSTANCES:
|
|
849
|
+
try:
|
|
850
|
+
instance.eval(RELEASE_LOCK_SCRIPT, 1, f"lock:{resource}", token)
|
|
851
|
+
except redis.exceptions.ConnectionError:
|
|
852
|
+
continue
|
|
853
|
+
return None
|
|
854
|
+
|
|
855
|
+
def redlock_release(resource: str, token: str):
|
|
856
|
+
for instance in REDIS_INSTANCES:
|
|
857
|
+
try:
|
|
858
|
+
instance.eval(RELEASE_LOCK_SCRIPT, 1, f"lock:{resource}", token)
|
|
859
|
+
except redis.exceptions.ConnectionError:
|
|
860
|
+
continue
|
|
861
|
+
```
|
|
862
|
+
|
|
863
|
+
> **注意**: 生产环境建议使用 `python-redis-lock` 或 `pottery` 库的 Redlock 实现,而非自行编写。
|
|
864
|
+
|
|
865
|
+
---
|
|
866
|
+
|
|
867
|
+
## 消息队列
|
|
868
|
+
|
|
869
|
+
### Stream(推荐,Redis 5.0+)
|
|
870
|
+
|
|
871
|
+
见上方 Stream 数据结构部分。Stream 提供:
|
|
872
|
+
- 消费者组(Consumer Group)
|
|
873
|
+
- 消息确认(ACK)
|
|
874
|
+
- 待处理消息列表(PEL)
|
|
875
|
+
- 消息回溯与重放
|
|
876
|
+
|
|
877
|
+
### Pub/Sub(发布/订阅)
|
|
878
|
+
|
|
879
|
+
适合实时通知,不保证消息持久化。
|
|
880
|
+
|
|
881
|
+
```bash
|
|
882
|
+
# 订阅(消费者终端)
|
|
883
|
+
SUBSCRIBE channel:notifications
|
|
884
|
+
PSUBSCRIBE channel:* # 模式订阅
|
|
885
|
+
|
|
886
|
+
# 发布(生产者终端)
|
|
887
|
+
PUBLISH channel:notifications "order:5001 created"
|
|
888
|
+
|
|
889
|
+
# Redis 7.0+ Sharded Pub/Sub(集群环境推荐)
|
|
890
|
+
SSUBSCRIBE channel:notifications
|
|
891
|
+
SPUBLISH channel:notifications "order:5001 created"
|
|
892
|
+
```
|
|
893
|
+
|
|
894
|
+
**Python redis-py 示例:**
|
|
895
|
+
|
|
896
|
+
```python
|
|
897
|
+
# 发布者
|
|
898
|
+
r.publish("channel:notifications", "order:5001 created")
|
|
899
|
+
|
|
900
|
+
# 订阅者(独立线程/进程)
|
|
901
|
+
pubsub = r.pubsub()
|
|
902
|
+
pubsub.subscribe("channel:notifications")
|
|
903
|
+
|
|
904
|
+
for message in pubsub.listen():
|
|
905
|
+
if message["type"] == "message":
|
|
906
|
+
print(f"Received: {message['data']}")
|
|
907
|
+
```
|
|
908
|
+
|
|
909
|
+
**Stream vs Pub/Sub 对比:**
|
|
910
|
+
|
|
911
|
+
| 特性 | Stream | Pub/Sub |
|
|
912
|
+
|------|--------|---------|
|
|
913
|
+
| 消息持久化 | 是 | 否 |
|
|
914
|
+
| 消费者组 | 是 | 否 |
|
|
915
|
+
| 消息确认 | 是 | 否 |
|
|
916
|
+
| 历史回溯 | 是 | 否 |
|
|
917
|
+
| 延迟 | 较低 | 极低 |
|
|
918
|
+
| 适用场景 | 可靠消息队列 | 实时通知广播 |
|
|
919
|
+
|
|
920
|
+
---
|
|
921
|
+
|
|
922
|
+
## Lua 脚本
|
|
923
|
+
|
|
924
|
+
Redis 保证 Lua 脚本的原子执行,适合需要多步原子操作的场景。
|
|
925
|
+
|
|
926
|
+
```bash
|
|
927
|
+
# 内联执行
|
|
928
|
+
EVAL "return redis.call('get', KEYS[1])" 1 mykey
|
|
929
|
+
|
|
930
|
+
# 条件自增(库存扣减示例)
|
|
931
|
+
EVAL "
|
|
932
|
+
local stock = tonumber(redis.call('get', KEYS[1]))
|
|
933
|
+
if stock and stock >= tonumber(ARGV[1]) then
|
|
934
|
+
redis.call('decrby', KEYS[1], ARGV[1])
|
|
935
|
+
return 1
|
|
936
|
+
end
|
|
937
|
+
return 0
|
|
938
|
+
" 1 stock:product:1001 2
|
|
939
|
+
```
|
|
940
|
+
|
|
941
|
+
**Python redis-py 示例:**
|
|
942
|
+
|
|
943
|
+
```python
|
|
944
|
+
# 注册 Lua 脚本(推荐,避免重复传输脚本体)
|
|
945
|
+
deduct_stock_script = r.register_script("""
|
|
946
|
+
local stock = tonumber(redis.call('get', KEYS[1]))
|
|
947
|
+
if stock and stock >= tonumber(ARGV[1]) then
|
|
948
|
+
redis.call('decrby', KEYS[1], ARGV[1])
|
|
949
|
+
return 1
|
|
950
|
+
end
|
|
951
|
+
return 0
|
|
952
|
+
""")
|
|
953
|
+
|
|
954
|
+
# 使用
|
|
955
|
+
r.set("stock:product:1001", 100)
|
|
956
|
+
result = deduct_stock_script(keys=["stock:product:1001"], args=[2])
|
|
957
|
+
# result == 1 表示扣减成功
|
|
958
|
+
|
|
959
|
+
# 限流器(滑动窗口)
|
|
960
|
+
rate_limit_script = r.register_script("""
|
|
961
|
+
local key = KEYS[1]
|
|
962
|
+
local limit = tonumber(ARGV[1])
|
|
963
|
+
local window = tonumber(ARGV[2])
|
|
964
|
+
local now = tonumber(ARGV[3])
|
|
965
|
+
|
|
966
|
+
redis.call('zremrangebyscore', key, 0, now - window)
|
|
967
|
+
local count = redis.call('zcard', key)
|
|
968
|
+
|
|
969
|
+
if count < limit then
|
|
970
|
+
redis.call('zadd', key, now, now .. ':' .. math.random(100000))
|
|
971
|
+
redis.call('expire', key, window)
|
|
972
|
+
return 1
|
|
973
|
+
end
|
|
974
|
+
return 0
|
|
975
|
+
""")
|
|
976
|
+
|
|
977
|
+
import time
|
|
978
|
+
allowed = rate_limit_script(
|
|
979
|
+
keys=["ratelimit:api:user:1001"],
|
|
980
|
+
args=[100, 60, int(time.time())] # 100次/60秒
|
|
981
|
+
)
|
|
982
|
+
```
|
|
983
|
+
|
|
984
|
+
---
|
|
985
|
+
|
|
986
|
+
## Pipeline 与事务
|
|
987
|
+
|
|
988
|
+
### Pipeline(批量命令,减少网络往返)
|
|
989
|
+
|
|
990
|
+
```python
|
|
991
|
+
# 不使用 Pipeline:N 次命令 = N 次网络往返
|
|
992
|
+
# 使用 Pipeline:N 次命令 = 1 次网络往返
|
|
993
|
+
|
|
994
|
+
pipe = r.pipeline(transaction=False) # 非事务 Pipeline
|
|
995
|
+
for i in range(1000):
|
|
996
|
+
pipe.set(f"key:{i}", f"value:{i}")
|
|
997
|
+
results = pipe.execute() # 批量执行
|
|
998
|
+
|
|
999
|
+
# 带读取的 Pipeline
|
|
1000
|
+
pipe = r.pipeline(transaction=False)
|
|
1001
|
+
for i in range(100):
|
|
1002
|
+
pipe.get(f"key:{i}")
|
|
1003
|
+
values = pipe.execute() # 返回100个值
|
|
1004
|
+
```
|
|
1005
|
+
|
|
1006
|
+
### 事务(MULTI/EXEC)
|
|
1007
|
+
|
|
1008
|
+
```python
|
|
1009
|
+
# Redis 事务:原子执行一组命令(但不支持回滚)
|
|
1010
|
+
pipe = r.pipeline(transaction=True) # 默认就是 True
|
|
1011
|
+
pipe.multi()
|
|
1012
|
+
pipe.set("account:A:balance", 800)
|
|
1013
|
+
pipe.set("account:B:balance", 1200)
|
|
1014
|
+
pipe.execute() # 原子执行
|
|
1015
|
+
|
|
1016
|
+
# WATCH 乐观锁
|
|
1017
|
+
def transfer(from_account: str, to_account: str, amount: int) -> bool:
|
|
1018
|
+
with r.pipeline() as pipe:
|
|
1019
|
+
while True:
|
|
1020
|
+
try:
|
|
1021
|
+
pipe.watch(from_account, to_account)
|
|
1022
|
+
balance_from = int(pipe.get(from_account) or 0)
|
|
1023
|
+
balance_to = int(pipe.get(to_account) or 0)
|
|
1024
|
+
|
|
1025
|
+
if balance_from < amount:
|
|
1026
|
+
pipe.unwatch()
|
|
1027
|
+
return False
|
|
1028
|
+
|
|
1029
|
+
pipe.multi()
|
|
1030
|
+
pipe.set(from_account, balance_from - amount)
|
|
1031
|
+
pipe.set(to_account, balance_to + amount)
|
|
1032
|
+
pipe.execute()
|
|
1033
|
+
return True
|
|
1034
|
+
except redis.WatchError:
|
|
1035
|
+
continue # 被其他客户端修改,重试
|
|
1036
|
+
```
|
|
1037
|
+
|
|
1038
|
+
---
|
|
1039
|
+
|
|
1040
|
+
## 内存优化
|
|
1041
|
+
|
|
1042
|
+
### 编码优化
|
|
1043
|
+
|
|
1044
|
+
Redis 自动选择紧凑编码以节省内存:
|
|
1045
|
+
|
|
1046
|
+
| 数据结构 | 紧凑编码 | 条件 | 普通编码 |
|
|
1047
|
+
|----------|----------|------|----------|
|
|
1048
|
+
| Hash | listpack (ziplist) | field 数 <= hash-max-listpack-entries (128) 且值 <= hash-max-listpack-value (64B) | hashtable |
|
|
1049
|
+
| List | listpack | 元素数 <= list-max-listpack-size (128) | quicklist |
|
|
1050
|
+
| Set | intset | 全部为整数且数量 <= set-max-intset-entries (512) | hashtable |
|
|
1051
|
+
| Set | listpack | 元素数 <= set-max-listpack-entries (128) | hashtable |
|
|
1052
|
+
| ZSet | listpack | 元素数 <= zset-max-listpack-entries (128) 且值 <= zset-max-listpack-value (64B) | skiplist + hashtable |
|
|
1053
|
+
|
|
1054
|
+
```bash
|
|
1055
|
+
# 查看 key 的编码方式
|
|
1056
|
+
OBJECT ENCODING mykey
|
|
1057
|
+
|
|
1058
|
+
# 调整阈值(redis.conf)
|
|
1059
|
+
hash-max-listpack-entries 128
|
|
1060
|
+
hash-max-listpack-value 64
|
|
1061
|
+
list-max-listpack-size 128
|
|
1062
|
+
set-max-intset-entries 512
|
|
1063
|
+
zset-max-listpack-entries 128
|
|
1064
|
+
zset-max-listpack-value 64
|
|
1065
|
+
```
|
|
1066
|
+
|
|
1067
|
+
### 内存策略
|
|
1068
|
+
|
|
1069
|
+
```bash
|
|
1070
|
+
# 最大内存限制
|
|
1071
|
+
maxmemory 4gb
|
|
1072
|
+
|
|
1073
|
+
# 淘汰策略
|
|
1074
|
+
maxmemory-policy allkeys-lru # 推荐:所有 key 中淘汰最近最少使用
|
|
1075
|
+
# 可选策略:
|
|
1076
|
+
# noeviction - 不淘汰,写入报错(默认)
|
|
1077
|
+
# allkeys-lru - 所有 key LRU 淘汰
|
|
1078
|
+
# allkeys-lfu - 所有 key LFU 淘汰(Redis 4.0+)
|
|
1079
|
+
# volatile-lru - 仅带 TTL 的 key LRU 淘汰
|
|
1080
|
+
# volatile-lfu - 仅带 TTL 的 key LFU 淘汰
|
|
1081
|
+
# volatile-ttl - 淘汰 TTL 最短的 key
|
|
1082
|
+
# allkeys-random - 随机淘汰
|
|
1083
|
+
# volatile-random - 随机淘汰(仅带 TTL 的 key)
|
|
1084
|
+
```
|
|
1085
|
+
|
|
1086
|
+
### 内存碎片整理
|
|
1087
|
+
|
|
1088
|
+
```bash
|
|
1089
|
+
# 查看内存碎片率
|
|
1090
|
+
INFO memory
|
|
1091
|
+
# mem_fragmentation_ratio > 1.5 表示碎片严重
|
|
1092
|
+
|
|
1093
|
+
# 在线碎片整理(Redis 4.0+)
|
|
1094
|
+
CONFIG SET activedefrag yes
|
|
1095
|
+
CONFIG SET active-defrag-threshold-lower 10 # 碎片率 > 10% 开始
|
|
1096
|
+
CONFIG SET active-defrag-threshold-upper 100 # 碎片率 > 100% 全力整理
|
|
1097
|
+
CONFIG SET active-defrag-cycle-min 1 # CPU 占用最小 1%
|
|
1098
|
+
CONFIG SET active-defrag-cycle-max 25 # CPU 占用最大 25%
|
|
1099
|
+
```
|
|
1100
|
+
|
|
1101
|
+
**Python 内存分析:**
|
|
1102
|
+
|
|
1103
|
+
```python
|
|
1104
|
+
info = r.info("memory")
|
|
1105
|
+
print(f"已使用内存: {info['used_memory_human']}")
|
|
1106
|
+
print(f"峰值内存: {info['used_memory_peak_human']}")
|
|
1107
|
+
print(f"碎片率: {info['mem_fragmentation_ratio']}")
|
|
1108
|
+
print(f"淘汰策略: {info['maxmemory_policy']}")
|
|
1109
|
+
|
|
1110
|
+
# 大 key 分析
|
|
1111
|
+
# redis-cli --bigkeys # 扫描大 key
|
|
1112
|
+
# redis-cli --memkeys # 按内存排序
|
|
1113
|
+
```
|
|
1114
|
+
|
|
1115
|
+
---
|
|
1116
|
+
|
|
1117
|
+
## 监控
|
|
1118
|
+
|
|
1119
|
+
### INFO 命令
|
|
1120
|
+
|
|
1121
|
+
```bash
|
|
1122
|
+
INFO # 全部信息
|
|
1123
|
+
INFO server # 服务器信息
|
|
1124
|
+
INFO clients # 客户端连接
|
|
1125
|
+
INFO memory # 内存使用
|
|
1126
|
+
INFO stats # 统计信息
|
|
1127
|
+
INFO replication # 复制状态
|
|
1128
|
+
INFO keyspace # 数据库 key 统计
|
|
1129
|
+
INFO commandstats # 命令执行统计
|
|
1130
|
+
```
|
|
1131
|
+
|
|
1132
|
+
**Python 监控脚本:**
|
|
1133
|
+
|
|
1134
|
+
```python
|
|
1135
|
+
def redis_health_check():
|
|
1136
|
+
info = r.info()
|
|
1137
|
+
metrics = {
|
|
1138
|
+
"connected_clients": info["connected_clients"],
|
|
1139
|
+
"used_memory_human": info["used_memory_human"],
|
|
1140
|
+
"mem_fragmentation_ratio": info["mem_fragmentation_ratio"],
|
|
1141
|
+
"instantaneous_ops_per_sec": info["instantaneous_ops_per_sec"],
|
|
1142
|
+
"hit_rate": (
|
|
1143
|
+
info["keyspace_hits"]
|
|
1144
|
+
/ max(info["keyspace_hits"] + info["keyspace_misses"], 1)
|
|
1145
|
+
* 100
|
|
1146
|
+
),
|
|
1147
|
+
"connected_slaves": info.get("connected_slaves", 0),
|
|
1148
|
+
"rejected_connections": info["rejected_connections"],
|
|
1149
|
+
"evicted_keys": info["evicted_keys"],
|
|
1150
|
+
"expired_keys": info["expired_keys"],
|
|
1151
|
+
}
|
|
1152
|
+
|
|
1153
|
+
# 告警阈值
|
|
1154
|
+
if metrics["mem_fragmentation_ratio"] > 1.5:
|
|
1155
|
+
alert("内存碎片率过高", metrics["mem_fragmentation_ratio"])
|
|
1156
|
+
if metrics["hit_rate"] < 90:
|
|
1157
|
+
alert("缓存命中率低", metrics["hit_rate"])
|
|
1158
|
+
if metrics["connected_clients"] > 9000:
|
|
1159
|
+
alert("连接数接近上限", metrics["connected_clients"])
|
|
1160
|
+
|
|
1161
|
+
return metrics
|
|
1162
|
+
```
|
|
1163
|
+
|
|
1164
|
+
### SLOWLOG
|
|
1165
|
+
|
|
1166
|
+
```bash
|
|
1167
|
+
# 配置慢查询阈值(微秒)
|
|
1168
|
+
CONFIG SET slowlog-log-slower-than 10000 # 10ms
|
|
1169
|
+
CONFIG SET slowlog-max-len 128 # 保留最近128条
|
|
1170
|
+
|
|
1171
|
+
# 查看慢查询
|
|
1172
|
+
SLOWLOG GET 10 # 最近10条
|
|
1173
|
+
SLOWLOG LEN # 慢查询数量
|
|
1174
|
+
SLOWLOG RESET # 清空
|
|
1175
|
+
```
|
|
1176
|
+
|
|
1177
|
+
### 延迟监控
|
|
1178
|
+
|
|
1179
|
+
```bash
|
|
1180
|
+
# 实时延迟检测
|
|
1181
|
+
redis-cli --latency # 持续检测
|
|
1182
|
+
redis-cli --latency-history # 每15秒一组
|
|
1183
|
+
redis-cli --latency-dist # 延迟分布图
|
|
1184
|
+
|
|
1185
|
+
# 内部延迟监控
|
|
1186
|
+
CONFIG SET latency-monitor-threshold 100 # 超过100ms记录
|
|
1187
|
+
LATENCY LATEST # 最近事件
|
|
1188
|
+
LATENCY HISTORY event_name # 历史记录
|
|
1189
|
+
LATENCY RESET # 清空
|
|
1190
|
+
```
|
|
1191
|
+
|
|
1192
|
+
### CLIENT LIST
|
|
1193
|
+
|
|
1194
|
+
```bash
|
|
1195
|
+
# 查看所有客户端连接
|
|
1196
|
+
CLIENT LIST
|
|
1197
|
+
CLIENT LIST TYPE normal # 仅普通连接
|
|
1198
|
+
|
|
1199
|
+
# 关键字段
|
|
1200
|
+
# id=5 addr=127.0.0.1:50000 fd=8 name= db=0 cmd=get age=100 idle=0
|
|
1201
|
+
# age: 连接时长 idle: 空闲时长 cmd: 最近命令
|
|
1202
|
+
|
|
1203
|
+
# 关闭空闲连接
|
|
1204
|
+
CLIENT KILL ID 5
|
|
1205
|
+
CONFIG SET timeout 300 # 300秒空闲自动断开
|
|
1206
|
+
```
|
|
1207
|
+
|
|
1208
|
+
---
|
|
1209
|
+
|
|
1210
|
+
## 安全
|
|
1211
|
+
|
|
1212
|
+
### ACL(Redis 6.0+)
|
|
1213
|
+
|
|
1214
|
+
```bash
|
|
1215
|
+
# 查看当前用户
|
|
1216
|
+
ACL WHOAMI
|
|
1217
|
+
|
|
1218
|
+
# 创建用户
|
|
1219
|
+
ACL SETUSER analyst on >strong_password ~report:* &* +get +mget +hgetall +info -@dangerous
|
|
1220
|
+
|
|
1221
|
+
# 语法说明:
|
|
1222
|
+
# on/off - 启用/禁用
|
|
1223
|
+
# >password - 设置密码
|
|
1224
|
+
# ~key_pattern - 允许的 key 模式
|
|
1225
|
+
# &channel - 允许的 Pub/Sub channel
|
|
1226
|
+
# +command - 允许的命令
|
|
1227
|
+
# -command - 禁止的命令
|
|
1228
|
+
# +@category - 允许整个命令类别
|
|
1229
|
+
# -@dangerous - 禁止危险命令
|
|
1230
|
+
|
|
1231
|
+
# 查看所有用户
|
|
1232
|
+
ACL LIST
|
|
1233
|
+
ACL GETUSER analyst
|
|
1234
|
+
|
|
1235
|
+
# 持久化 ACL 规则
|
|
1236
|
+
ACL SAVE # 保存到 aclfile
|
|
1237
|
+
ACL LOAD # 从 aclfile 加载
|
|
1238
|
+
|
|
1239
|
+
# 常用命令类别
|
|
1240
|
+
ACL CAT # 列出所有类别
|
|
1241
|
+
# @read, @write, @set, @hash, @list, @sortedset, @string
|
|
1242
|
+
# @admin, @dangerous, @slow, @fast, @keyspace, @pubsub
|
|
1243
|
+
```
|
|
1244
|
+
|
|
1245
|
+
### TLS 加密
|
|
1246
|
+
|
|
1247
|
+
```bash
|
|
1248
|
+
# redis.conf
|
|
1249
|
+
tls-port 6380
|
|
1250
|
+
port 0 # 禁用非 TLS 端口
|
|
1251
|
+
tls-cert-file /path/to/redis.crt
|
|
1252
|
+
tls-key-file /path/to/redis.key
|
|
1253
|
+
tls-ca-cert-file /path/to/ca.crt
|
|
1254
|
+
tls-auth-clients optional # 客户端证书验证
|
|
1255
|
+
|
|
1256
|
+
# 客户端连接
|
|
1257
|
+
redis-cli --tls --cert /path/to/client.crt --key /path/to/client.key --cacert /path/to/ca.crt -h host -p 6380
|
|
1258
|
+
```
|
|
1259
|
+
|
|
1260
|
+
**Python TLS 连接:**
|
|
1261
|
+
|
|
1262
|
+
```python
|
|
1263
|
+
import ssl
|
|
1264
|
+
|
|
1265
|
+
ssl_context = ssl.create_default_context(cafile="/path/to/ca.crt")
|
|
1266
|
+
ssl_context.load_cert_chain(
|
|
1267
|
+
certfile="/path/to/client.crt",
|
|
1268
|
+
keyfile="/path/to/client.key",
|
|
1269
|
+
)
|
|
1270
|
+
|
|
1271
|
+
r = redis.Redis(
|
|
1272
|
+
host="redis-host",
|
|
1273
|
+
port=6380,
|
|
1274
|
+
ssl=True,
|
|
1275
|
+
ssl_ca_certs="/path/to/ca.crt",
|
|
1276
|
+
ssl_certfile="/path/to/client.crt",
|
|
1277
|
+
ssl_keyfile="/path/to/client.key",
|
|
1278
|
+
decode_responses=True,
|
|
1279
|
+
)
|
|
1280
|
+
```
|
|
1281
|
+
|
|
1282
|
+
### 密码认证
|
|
1283
|
+
|
|
1284
|
+
```bash
|
|
1285
|
+
# redis.conf
|
|
1286
|
+
requirepass "your_strong_password_here"
|
|
1287
|
+
|
|
1288
|
+
# 客户端认证
|
|
1289
|
+
AUTH "your_strong_password_here"
|
|
1290
|
+
redis-cli -a "your_strong_password_here" # 不推荐(密码出现在进程列表)
|
|
1291
|
+
redis-cli --askpass # 推荐(交互式输入)
|
|
1292
|
+
```
|
|
1293
|
+
|
|
1294
|
+
### 安全加固清单
|
|
1295
|
+
|
|
1296
|
+
```bash
|
|
1297
|
+
# 1. 绑定地址
|
|
1298
|
+
bind 127.0.0.1 192.168.1.100 # 不要 bind 0.0.0.0
|
|
1299
|
+
|
|
1300
|
+
# 2. 禁用危险命令
|
|
1301
|
+
rename-command FLUSHALL ""
|
|
1302
|
+
rename-command FLUSHDB ""
|
|
1303
|
+
rename-command CONFIG "" # 或重命名为随机字符串
|
|
1304
|
+
rename-command DEBUG ""
|
|
1305
|
+
rename-command KEYS "" # 用 SCAN 替代
|
|
1306
|
+
|
|
1307
|
+
# 3. 禁用 Lua 调试
|
|
1308
|
+
enable-debug-command no
|
|
1309
|
+
|
|
1310
|
+
# 4. 连接限制
|
|
1311
|
+
maxclients 10000
|
|
1312
|
+
timeout 300 # 空闲超时
|
|
1313
|
+
|
|
1314
|
+
# 5. 保护模式
|
|
1315
|
+
protected-mode yes # 默认开启
|
|
1316
|
+
```
|
|
1317
|
+
|
|
1318
|
+
---
|
|
1319
|
+
|
|
1320
|
+
## 常见陷阱
|
|
1321
|
+
|
|
1322
|
+
### 大 Key(Big Key)
|
|
1323
|
+
|
|
1324
|
+
大 Key 导致:阻塞其他命令、网络拥塞、内存倾斜、主从同步延迟。
|
|
1325
|
+
|
|
1326
|
+
```bash
|
|
1327
|
+
# 检测大 Key
|
|
1328
|
+
redis-cli --bigkeys
|
|
1329
|
+
redis-cli --memkeys --memkeys-samples 100
|
|
1330
|
+
|
|
1331
|
+
# 查看单个 key 内存
|
|
1332
|
+
MEMORY USAGE mykey SAMPLES 0
|
|
1333
|
+
OBJECT ENCODING mykey
|
|
1334
|
+
DEBUG OBJECT mykey # 生产慎用
|
|
1335
|
+
```
|
|
1336
|
+
|
|
1337
|
+
**治理方案:**
|
|
1338
|
+
|
|
1339
|
+
```python
|
|
1340
|
+
# 拆分大 Hash(>= 5000 fields 应拆分)
|
|
1341
|
+
def split_large_hash(key: str, data: dict, bucket_size: int = 500):
|
|
1342
|
+
for field, value in data.items():
|
|
1343
|
+
bucket = hash(field) % (len(data) // bucket_size + 1)
|
|
1344
|
+
r.hset(f"{key}:bucket:{bucket}", field, value)
|
|
1345
|
+
|
|
1346
|
+
def get_from_split_hash(key: str, field: str, total_buckets: int):
|
|
1347
|
+
bucket = hash(field) % total_buckets
|
|
1348
|
+
return r.hget(f"{key}:bucket:{bucket}", field)
|
|
1349
|
+
|
|
1350
|
+
# 异步删除大 Key(Redis 4.0+)
|
|
1351
|
+
r.unlink("large_key") # 异步删除,不阻塞主线程
|
|
1352
|
+
# 避免使用 DEL 删除大 Key
|
|
1353
|
+
```
|
|
1354
|
+
|
|
1355
|
+
**大 Key 阈值参考:**
|
|
1356
|
+
|
|
1357
|
+
| 数据类型 | 危险阈值 |
|
|
1358
|
+
|----------|----------|
|
|
1359
|
+
| String | > 10KB |
|
|
1360
|
+
| Hash | > 5000 fields 或 value > 10KB |
|
|
1361
|
+
| List | > 10000 元素 |
|
|
1362
|
+
| Set | > 10000 成员 |
|
|
1363
|
+
| ZSet | > 10000 成员 |
|
|
1364
|
+
|
|
1365
|
+
### 热 Key(Hot Key)
|
|
1366
|
+
|
|
1367
|
+
单个 key 被高并发访问,导致单节点成为瓶颈。
|
|
1368
|
+
|
|
1369
|
+
```python
|
|
1370
|
+
# 方案1:本地缓存 + Redis(两级缓存)
|
|
1371
|
+
from functools import lru_cache
|
|
1372
|
+
import time
|
|
1373
|
+
|
|
1374
|
+
LOCAL_CACHE = {}
|
|
1375
|
+
LOCAL_TTL = 5 # 本地缓存5秒
|
|
1376
|
+
|
|
1377
|
+
def get_hot_key(key: str):
|
|
1378
|
+
now = time.time()
|
|
1379
|
+
if key in LOCAL_CACHE and now < LOCAL_CACHE[key]["expire"]:
|
|
1380
|
+
return LOCAL_CACHE[key]["value"]
|
|
1381
|
+
|
|
1382
|
+
value = r.get(key)
|
|
1383
|
+
LOCAL_CACHE[key] = {"value": value, "expire": now + LOCAL_TTL}
|
|
1384
|
+
return value
|
|
1385
|
+
|
|
1386
|
+
# 方案2:读副本分散(key 加后缀)
|
|
1387
|
+
import random
|
|
1388
|
+
|
|
1389
|
+
def set_hot_key(key: str, value: str, replicas: int = 3):
|
|
1390
|
+
pipe = r.pipeline(transaction=False)
|
|
1391
|
+
for i in range(replicas):
|
|
1392
|
+
pipe.set(f"{key}:r{i}", value, ex=3600)
|
|
1393
|
+
pipe.execute()
|
|
1394
|
+
|
|
1395
|
+
def get_hot_key_replica(key: str, replicas: int = 3):
|
|
1396
|
+
replica = random.randint(0, replicas - 1)
|
|
1397
|
+
return r.get(f"{key}:r{replica}")
|
|
1398
|
+
```
|
|
1399
|
+
|
|
1400
|
+
### 内存碎片
|
|
1401
|
+
|
|
1402
|
+
长时间运行后频繁的创建/删除操作会产生内存碎片。
|
|
1403
|
+
|
|
1404
|
+
```python
|
|
1405
|
+
def check_fragmentation():
|
|
1406
|
+
info = r.info("memory")
|
|
1407
|
+
ratio = info["mem_fragmentation_ratio"]
|
|
1408
|
+
|
|
1409
|
+
if ratio > 1.5:
|
|
1410
|
+
print(f"WARNING: 内存碎片率 {ratio:.2f},建议启用在线碎片整理")
|
|
1411
|
+
r.config_set("activedefrag", "yes")
|
|
1412
|
+
elif ratio < 1.0:
|
|
1413
|
+
print(f"WARNING: 碎片率 < 1.0 ({ratio:.2f}),可能存在 swap,检查系统内存")
|
|
1414
|
+
else:
|
|
1415
|
+
print(f"OK: 内存碎片率 {ratio:.2f}")
|
|
1416
|
+
```
|
|
1417
|
+
|
|
1418
|
+
### 其他常见陷阱
|
|
1419
|
+
|
|
1420
|
+
```python
|
|
1421
|
+
# 1. KEYS 命令(生产禁用,用 SCAN 替代)
|
|
1422
|
+
# ❌ KEYS user:* # O(N) 全量扫描,阻塞
|
|
1423
|
+
# ✅ SCAN 0 MATCH user:* COUNT 100
|
|
1424
|
+
|
|
1425
|
+
def scan_keys(pattern: str, count: int = 100):
|
|
1426
|
+
"""安全扫描 key"""
|
|
1427
|
+
cursor = 0
|
|
1428
|
+
keys = []
|
|
1429
|
+
while True:
|
|
1430
|
+
cursor, batch = r.scan(cursor=cursor, match=pattern, count=count)
|
|
1431
|
+
keys.extend(batch)
|
|
1432
|
+
if cursor == 0:
|
|
1433
|
+
break
|
|
1434
|
+
return keys
|
|
1435
|
+
|
|
1436
|
+
# 2. 连接泄漏(务必使用连接池)
|
|
1437
|
+
# ❌
|
|
1438
|
+
r = redis.Redis(host="localhost") # 每次新建连接
|
|
1439
|
+
|
|
1440
|
+
# ✅
|
|
1441
|
+
pool = redis.ConnectionPool(host="localhost", port=6379, max_connections=50)
|
|
1442
|
+
r = redis.Redis(connection_pool=pool)
|
|
1443
|
+
|
|
1444
|
+
# 3. 序列化选择
|
|
1445
|
+
import json
|
|
1446
|
+
import pickle
|
|
1447
|
+
|
|
1448
|
+
# ❌ pickle(安全风险 + 不可跨语言)
|
|
1449
|
+
r.set("data", pickle.dumps(obj))
|
|
1450
|
+
|
|
1451
|
+
# ✅ JSON(安全 + 可跨语言)
|
|
1452
|
+
r.set("data", json.dumps(obj))
|
|
1453
|
+
|
|
1454
|
+
# ✅ msgpack(更紧凑)
|
|
1455
|
+
import msgpack
|
|
1456
|
+
r.set("data", msgpack.packb(obj))
|
|
1457
|
+
|
|
1458
|
+
# 4. 过期时间丢失
|
|
1459
|
+
r.set("key", "value", ex=3600)
|
|
1460
|
+
r.set("key", "new_value") # ⚠️ TTL 被清除!
|
|
1461
|
+
r.set("key", "new_value", keepttl=True) # ✅ 保留 TTL (Redis 6.0+)
|
|
1462
|
+
```
|
|
1463
|
+
|
|
1464
|
+
---
|
|
1465
|
+
|
|
1466
|
+
## Agent Checklist
|
|
1467
|
+
|
|
1468
|
+
开发和运维 Redis 相关任务时,按以下清单检查:
|
|
1469
|
+
|
|
1470
|
+
### 数据模型设计
|
|
1471
|
+
- [ ] 选择了最合适的数据结构(不是所有场景都用 String)
|
|
1472
|
+
- [ ] Hash/List/Set/ZSet 的元素数量在紧凑编码阈值内(或已评估超出的影响)
|
|
1473
|
+
- [ ] Key 命名规范统一(`业务:对象类型:ID:属性`,使用冒号分隔)
|
|
1474
|
+
- [ ] 已评估大 Key 风险(单个 value < 10KB,集合类 < 10000 元素)
|
|
1475
|
+
- [ ] 热 Key 已识别并有应对方案(本地缓存 / 读副本)
|
|
1476
|
+
|
|
1477
|
+
### 缓存策略
|
|
1478
|
+
- [ ] 明确选择了缓存模式(Cache-Aside / Write-Through / Write-Behind)
|
|
1479
|
+
- [ ] 所有缓存 key 设置了合理的 TTL
|
|
1480
|
+
- [ ] TTL 添加了随机偏移(防止缓存雪崩)
|
|
1481
|
+
- [ ] 缓存穿透防护已就位(空值缓存 / 布隆过滤器)
|
|
1482
|
+
- [ ] 热点 key 击穿防护已就位(互斥锁 / 逻辑过期)
|
|
1483
|
+
|
|
1484
|
+
### 高可用与持久化
|
|
1485
|
+
- [ ] 持久化策略已配置(推荐混合持久化)
|
|
1486
|
+
- [ ] 主从复制已验证 `INFO replication`
|
|
1487
|
+
- [ ] Sentinel / Cluster 故障转移已测试
|
|
1488
|
+
- [ ] 客户端使用 Sentinel/Cluster 感知的连接方式
|
|
1489
|
+
- [ ] `min-replicas-to-write` 和 `min-replicas-max-lag` 已配置
|
|
1490
|
+
|
|
1491
|
+
### 性能与内存
|
|
1492
|
+
- [ ] 使用 Pipeline 批量执行命令(减少网络往返)
|
|
1493
|
+
- [ ] 生产环境禁用 KEYS 命令,使用 SCAN 替代
|
|
1494
|
+
- [ ] `maxmemory` 和 `maxmemory-policy` 已配置
|
|
1495
|
+
- [ ] 内存碎片率定期检查(`mem_fragmentation_ratio`)
|
|
1496
|
+
- [ ] 大 Key 删除使用 `UNLINK` 而非 `DEL`
|
|
1497
|
+
- [ ] 连接使用连接池,max_connections 已合理设置
|
|
1498
|
+
|
|
1499
|
+
### 分布式锁
|
|
1500
|
+
- [ ] 锁有 token(UUID),释放时校验 token(Lua 原子操作)
|
|
1501
|
+
- [ ] 锁设置了合理的过期时间(防止死锁)
|
|
1502
|
+
- [ ] 已考虑锁续期场景(长任务需 watchdog 机制)
|
|
1503
|
+
- [ ] 跨数据中心场景评估了 Redlock 的适用性
|
|
1504
|
+
|
|
1505
|
+
### 安全
|
|
1506
|
+
- [ ] 启用了 ACL(Redis 6.0+),最小权限原则
|
|
1507
|
+
- [ ] 危险命令已禁用或重命名(FLUSHALL / FLUSHDB / CONFIG / KEYS / DEBUG)
|
|
1508
|
+
- [ ] 绑定了具体 IP(非 0.0.0.0)
|
|
1509
|
+
- [ ] TLS 加密已启用(公网 / 跨网段必须)
|
|
1510
|
+
- [ ] `protected-mode yes` 已确认
|
|
1511
|
+
|
|
1512
|
+
### 监控与运维
|
|
1513
|
+
- [ ] SLOWLOG 已配置(阈值 10ms)
|
|
1514
|
+
- [ ] 缓存命中率持续监控(`keyspace_hits / (hits + misses)`)
|
|
1515
|
+
- [ ] 连接数、内存、OPS 有告警
|
|
1516
|
+
- [ ] 定期执行 `redis-cli --bigkeys` 巡检
|
|
1517
|
+
- [ ] AOF 重写和 RDB 快照的资源影响已评估
|
|
1518
|
+
|
|
1519
|
+
---
|
|
1520
|
+
|
|
1521
|
+
**知识ID**: `redis-complete`
|
|
1522
|
+
**领域**: data
|
|
1523
|
+
**类型**: standards
|
|
1524
|
+
**难度**: intermediate-advanced
|
|
1525
|
+
**质量分**: 95
|
|
1526
|
+
**维护者**: data-team@umadev.com
|
|
1527
|
+
**最后更新**: 2026-03-28
|