@kkuffour/solid-moderation-plugin 0.1.0

package/CONFIG-GUIDE.md

@@ -0,0 +1,49 @@
+# Configuration Guide
+
+## Understanding the Configuration Files
+
+### `config/default.json` (Production - Installed with npm)
+- Located in `node_modules/@solid/moderation-plugin/config/default.json` after installation
+- Contains the complete moderation setup with default thresholds
+- Imported by users in their CSS config using: `"@solid/moderation-plugin:config/default.json"`
+- **This is what production users import**
+
+### `config-with-moderation.json` (Local Testing Only)
+- Located in the plugin source directory
+- Used for testing the plugin locally before publishing
+- Points to local files, not npm package
+- **Not included in npm package**
+
+## For Production Users
+
+When you install the plugin via npm, you create a config file in **your CSS project**:
+
+```json
+{
+  "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^7.0.0/components/context.jsonld",
+  "import": [
+    "css:config/file.json",
+    "@solid/moderation-plugin:config/default.json"
+  ]
+}
+```
+
+Then run:
+```bash
+export SIGHTENGINE_API_USER=your_api_user
+export SIGHTENGINE_API_SECRET=your_api_secret
+npx @solid/community-server -c config-with-moderation.json
+```
+
+## For Local Development/Testing
+
+When testing the plugin locally (before publishing):
+
+1. Use `npm link` to link the local plugin
+2. Use the `config-with-moderation.json` file in the plugin directory
+3. Run CSS pointing to that config file
+
+## Key Difference
+
+- **Production**: Import `@solid/moderation-plugin:config/default.json` (from npm package)
+- **Local Testing**: Use full path to `config-with-moderation.json` (from source directory)

package/DEVELOPMENT.md

@@ -0,0 +1,129 @@
+# Development Notes
+
+## Components.js Module Loading Issue
+
+### Problem
+Components.js cannot load modules via `npm link` due to symlink resolution issues. When using `npm link`, the plugin appears in `node_modules/@solid/moderation-plugin` as a symlink, but Components.js's module loader cannot follow it to instantiate classes.
+
+### Symptoms
+- Server starts without errors
+- Config file is loaded successfully
+- No moderation logs appear when uploading files
+- ModerationOperationHandler is never instantiated
+- Uploads succeed without any moderation checks
+
+### Root Cause
+Components.js uses its own module resolution that doesn't properly handle symlinked packages. The `requireElement` in the generated `.jsonld` files points to the class name, but Components.js cannot resolve the actual module path through the symlink.
+
+### Solutions
+
+#### Option 1: File Path Dependency (Recommended for Development)
+Add to CSS's `package.json`:
+```json
+{
+  "dependencies": {
+    "@solid/moderation-plugin": "file:../solid-moderation"
+  }
+}
+```
+
+Then run:
+```bash
+cd /Users/opendata/CommunitySolidServer
+npm install
+```
+
+This creates a proper copy (not symlink) in node_modules.
+
+**After code changes:**
+```bash
+cd /Users/opendata/solid-moderation
+npm run build
+cd /Users/opendata/CommunitySolidServer
+npm install  # Re-copies the updated files
+```
+
+#### Option 2: Publish to npm (Recommended for Production)
+```bash
+cd /Users/opendata/solid-moderation
+npm publish
+```
+
+Then in CSS:
+```bash
+npm install @solid/moderation-plugin@latest
+```
+
+#### Option 3: Manual Copy (Quick Testing Only)
+```bash
+rm -rf /Users/opendata/CommunitySolidServer/node_modules/@solid/moderation-plugin
+cp -r /Users/opendata/solid-moderation /Users/opendata/CommunitySolidServer/node_modules/@solid/moderation-plugin
+```
+
+**Note:** Must repeat after every code change.
+
+## Current Configuration
+
+### Environment Variables
+```bash
+export SIGHTENGINE_API_USER="1060049443"
+export SIGHTENGINE_API_SECRET="QRQ8HUmh4hyvhZjksBJq5ZaNYPLPEKXu"
+```
+
+### Thresholds (Configurable in config-with-moderation.json)
+- **Image Nudity**: 0.1 (10% - very strict)
+- **Text Sexual**: 0.1 (10% - very strict)
+- **Text Toxic**: 0.1 (10% - very strict)
+- **Video Nudity**: 0.1 (10% - very strict)
+
+Lower values = stricter moderation (blocks more content)
+Higher values = lenient moderation (blocks less content)
+Range: 0.0 (block everything) to 1.0 (block nothing)
+
+### Starting the Server
+```bash
+cd /Users/opendata/CommunitySolidServer
+export SIGHTENGINE_API_USER="1060049443"
+export SIGHTENGINE_API_SECRET="QRQ8HUmh4hyvhZjksBJq5ZaNYPLPEKXu"
+npm start -- -c /Users/opendata/solid-moderation/config-with-moderation.json -f data/ -p 3009 -l debug > logs/server.log 2>&1 &
+```
+
+### Checking Logs
+```bash
+# Watch for moderation activity
+tail -f logs/server.log | grep -i moderation
+
+# Check if handler is invoked
+grep "MODERATION: Checking" logs/server.log
+
+# Check for API failures
+grep "MODERATION:.*failed" logs/server.log
+```
+
+## Features Implemented
+
+### Code Structure
+- **ModerationOperationHandler**: Wraps POST/PUT/PATCH handlers
+- **ModerationMixin**: Performs actual content checks
+- **ModerationStore**: Audit logging for violations
+- **SightEngineProvider**: API client for SightEngine
+- **Configurable thresholds**: All thresholds adjustable via config
+
+### Moderation Checks
+- **Images**: Nudity detection
+- **Text**: Sexual content and toxic language
+- **Video**: Nudity detection
+- **Fail-open policy**: Allows content if API fails
+
+### Debug Logging
+- Handler invocation logs
+- Content type detection logs
+- API response with scores
+- Threshold comparison logs
+- Failure reason logs
+
+## Known Issues
+
+1. **npm link doesn't work** - Use file path dependency instead
+2. **Environment variables** - Must be set before starting server
+3. **Audit logs** - Only record violations, not all checks

package/ENV-VARIABLES.md

@@ -0,0 +1,137 @@
+# Environment Variables Explained
+
+## What `export` Does
+
+```bash
+export SIGHTENGINE_API_USER=your_api_user
+export SIGHTENGINE_API_SECRET=your_api_secret
+```
+
+These commands set **environment variables** in your shell session:
+
+- `export` makes the variable available to all programs you run in that terminal
+- `SIGHTENGINE_API_USER` is the variable name
+- `your_api_user` is the value (replace with your actual API credentials)
+
+## How the Plugin Uses These Variables
+
+In `config/default.json`, you'll see:
+
+```json
+"ModerationConfig:_sightEngine": {
+  "ModerationConfig:_sightEngine_apiUser": "${SIGHTENGINE_API_USER}",
+  "ModerationConfig:_sightEngine_apiSecret": "${SIGHTENGINE_API_SECRET}"
+}
+```
+
+The `${SIGHTENGINE_API_USER}` syntax means:
+- "Look for an environment variable named SIGHTENGINE_API_USER"
+- "Replace this placeholder with its value"
+
+## Example Flow
+
+1. **You set the variables:**
+   ```bash
+   export SIGHTENGINE_API_USER=abc123
+   export SIGHTENGINE_API_SECRET=xyz789
+   ```
+
+2. **You start CSS:**
+   ```bash
+   npx @solid/community-server -c config.json
+   ```
+
+3. **The plugin reads the config and replaces placeholders:**
+   - `${SIGHTENGINE_API_USER}` becomes `abc123`
+   - `${SIGHTENGINE_API_SECRET}` becomes `xyz789`
+
+4. **The plugin uses these credentials to call SightEngine API**
+
+## Why Use Environment Variables?
+
+✅ **Security**: Credentials aren't stored in config files (which might be committed to git)
+✅ **Flexibility**: Different credentials for dev/staging/production without changing code
+✅ **Standard Practice**: Common pattern for managing secrets
+
+## Alternative Methods
+
+### Option 1: Set per command (temporary)
+```bash
+SIGHTENGINE_API_USER=abc123 SIGHTENGINE_API_SECRET=xyz789 npx @solid/community-server -c config.json
+```
+
+### Option 2: Use a .env file (with dotenv)
+Create `.env` file:
+```
+SIGHTENGINE_API_USER=abc123
+SIGHTENGINE_API_SECRET=xyz789
+```
+
+### Option 3: Set in shell profile (permanent) ⭐ RECOMMENDED FOR PRODUCTION
+Add to `~/.bashrc` or `~/.zshrc`:
+```bash
+export SIGHTENGINE_API_USER=abc123
+export SIGHTENGINE_API_SECRET=xyz789
+```
+
+### Option 4: System service environment file (for systemd)
+Create `/etc/systemd/system/solid-server.service.d/override.conf`:
+```ini
+[Service]
+Environment="SIGHTENGINE_API_USER=abc123"
+Environment="SIGHTENGINE_API_SECRET=xyz789"
+```
+
+## What Happens on Server Restart?
+
+### ❌ If you only used `export` in terminal:
+- Variables are **LOST** when terminal closes or server restarts
+- Server will fail to moderate content (falls back to fail-open policy)
+- You must re-export before starting server again
+
+### ✅ If you added to shell profile (`~/.bashrc` or `~/.zshrc`):
+- Variables are **PRESERVED** across terminal sessions
+- But still lost on full system reboot unless server auto-starts with those variables
+
+### ✅ If you use systemd service file:
+- Variables are **ALWAYS AVAILABLE** to the service
+- Survives server restarts and system reboots
+- **BEST for production servers**
+
+### ✅ If you use Docker:
+```yaml
+services:
+  solid-server:
+    environment:
+      - SIGHTENGINE_API_USER=abc123
+      - SIGHTENGINE_API_SECRET=xyz789
+```
+Variables are preserved in container configuration.
+
+## Production Recommendation
+
+For a production server that needs to survive restarts:
+
+1. **Use systemd service** with environment variables in service file
+2. **Or use Docker** with environment variables in docker-compose.yml
+3. **Or use a process manager** like PM2 with ecosystem.config.js:
+   ```javascript
+   module.exports = {
+     apps: [{
+       name: 'solid-server',
+       script: 'npx',
+       args: '@solid/community-server -c config.json',
+       env: {
+         SIGHTENGINE_API_USER: 'abc123',
+         SIGHTENGINE_API_SECRET: 'xyz789'
+       }
+     }]
+   }
+   ```
+
+## Getting Your API Credentials
+
+1. Sign up at https://sightengine.com/
+2. Go to your dashboard
+3. Find your API User and API Secret
+4. Replace `your_api_user` and `your_api_secret` with those values

package/INSTALLATION.md

@@ -0,0 +1,90 @@
+# Files Generated in Community Solid Server
+
+When you install `@solid/moderation-plugin` in your Community Solid Server, the following files will be generated in `node_modules/@solid/moderation-plugin/`:
+
+## Directory Structure
+
+```
+node_modules/@solid/moderation-plugin/
+├── dist/                           # Compiled JavaScript and metadata
+│   ├── components/                 # Components.js metadata
+│   │   ├── components.jsonld       # Component definitions
+│   │   └── context.jsonld          # JSON-LD context
+│   │
+│   ├── providers/                  # API providers
+│   │   ├── SightEngineProvider.js
+│   │   ├── SightEngineProvider.d.ts
+│   │   ├── SightEngineProvider.js.map
+│   │   ├── SightEngineProvider.d.ts.map
+│   │   └── SightEngineProvider.jsonld
+│   │
+│   ├── util/                       # Utilities
+│   │   ├── GuardedStream.js
+│   │   ├── GuardedStream.d.ts
+│   │   └── *.map files
+│   │
+│   ├── index.js                    # Main entry point
+│   ├── index.d.ts                  # TypeScript definitions
+│   ├── ModerationOperationHandler.js
+│   ├── ModerationOperationHandler.d.ts
+│   ├── ModerationOperationHandler.jsonld
+│   ├── ModerationConfig.js
+│   ├── ModerationConfig.d.ts
+│   ├── ModerationConfig.jsonld
+│   ├── ModerationStore.js
+│   ├── ModerationStore.d.ts
+│   ├── ModerationStore.jsonld
+│   ├── ModerationMixin.js
+│   ├── ModerationMixin.d.ts
+│   ├── ModerationMixin.jsonld
+│   ├── ModerationRecord.js
+│   ├── ModerationRecord.d.ts
+│   ├── ModerationRecord.jsonld
+│   └── *.map files                 # Source maps
+│
+├── config/                         # Configuration templates
+│   └── default.json                # Default moderation settings
+│
+├── package.json                    # Package metadata
+├── README.md                       # Documentation
+└── LICENSE                         # MIT License
+```
+
+## Runtime Files (Created During Operation)
+
+When CSS runs with moderation enabled, these files/folders are created:
+
+```
+your-css-project/
+├── data/
+│   └── moderation-logs/            # Audit logs (if enabled)
+│       ├── 2024-01-20.jsonl        # Daily log files
+│       ├── 2024-01-21.jsonl
+│       └── ...
+│
+└── /tmp/                           # Temporary files during analysis
+    └── moderation_*.jpg/mp4        # Auto-deleted after analysis
+```
+
+## Key Files Explained
+
+### Components.js Files (*.jsonld)
+- Used by CSS's dependency injection system
+- Define how classes are instantiated and wired together
+- Allow configuration through JSON-LD config files
+
+### JavaScript Files (*.js)
+- Compiled TypeScript code
+- Main application logic
+
+### TypeScript Definitions (*.d.ts)
+- Type information for TypeScript users
+- Enable IDE autocomplete and type checking
+
+### Source Maps (*.map)
+- Map compiled JS back to original TypeScript
+- Used for debugging
+
+### config/default.json
+- Template configuration with default thresholds
+- Can be imported in CSS config files

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/MIGRATION.md

@@ -0,0 +1,81 @@
+# Migration Guide: Extracting Moderation to External Plugin
+
+## Overview
+
+This guide walks through migrating the moderation feature from CSS core to the standalone `@solid/moderation-plugin` package.
+
+## Steps
+
+### 1. Publish Plugin Package
+
+```bash
+cd mod-package-struct
+npm install
+npm run build
+npm publish --access public
+```
+
+### 2. Update CSS to Use Plugin
+
+In your CSS project:
+
+```bash
+npm install @solid/moderation-plugin
+```
+
+### 3. Update Configuration
+
+Replace `config/file-moderation.json` with:
+
+```json
+{
+  "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^7.0.0/components/context.jsonld",
+  "import": [
+    "css:config/file.json",
+    "@solid/moderation-plugin:config/default.json"
+  ]
+}
+```
+
+### 4. Remove Core Moderation Code
+
+Delete from CSS repository:
+- `src/moderation/` directory
+- `src/http/ldp/ModerationOperationHandler.ts`
+- `src/http/ldp/ModerationMixin.ts`
+- `config/moderation-settings.json`
+- `config/file-moderation.json`
+
+### 5. Update Exports
+
+Remove from `src/index.ts`:
+- ModerationOperationHandler export
+- ModerationConfig export
+- ModerationStore export
+- ModerationMixin export
+
+### 6. Test
+
+```bash
+# Start server with plugin
+npm start -- -c config-with-moderation.json
+
+# Test image upload
+curl -X PUT http://localhost:3000/test.jpg \
+  -H "Content-Type: image/jpeg" \
+  --data-binary @test-image.jpg
+```
+
+## Rollback Plan
+
+If issues occur:
+1. Keep plugin package unpublished
+2. Revert CSS changes
+3. Continue using embedded moderation
+
+## Timeline
+
+- Week 1: Extract and test plugin locally
+- Week 2: Publish plugin to npm
+- Week 3: Update CSS to use plugin
+- Week 4: Remove core moderation code

package/PRODUCTION.md

@@ -0,0 +1,186 @@
+# Production Deployment Guide
+
+## Installation
+
+1. **Install the plugin:**
+   ```bash
+   npm install @solid/moderation-plugin
+   ```
+
+2. **Create config file** in your CSS project (e.g., `config-with-moderation.json`):
+   ```json
+   {
+     "@context": [
+       "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^7.0.0/components/context.jsonld",
+       "https://linkedsoftwaredependencies.org/bundles/npm/@solid/moderation-plugin/^1.0.0/components/context.jsonld"
+     ],
+     "import": [
+       "css:config/file.json",
+       "@solid/moderation-plugin:config/default.json"
+     ]
+   }
+   ```
+
+3. **Set environment variables** (choose one method):
+
+## Environment Variable Setup
+
+### Option 1: Shell Profile (Simple)
+Add to `~/.bashrc` or `~/.zshrc`:
+```bash
+export SIGHTENGINE_API_USER=your_api_user
+export SIGHTENGINE_API_SECRET=your_api_secret
+```
+
+### Option 2: Systemd Service (Recommended for Production)
+Create `/etc/systemd/system/solid-server.service`:
+```ini
+[Unit]
+Description=Community Solid Server with Moderation
+After=network.target
+
+[Service]
+Type=simple
+User=solid
+WorkingDirectory=/opt/solid-server
+Environment="SIGHTENGINE_API_USER=your_api_user"
+Environment="SIGHTENGINE_API_SECRET=your_api_secret"
+ExecStart=/usr/bin/npx @solid/community-server -c config-with-moderation.json
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Enable and start:
+```bash
+sudo systemctl enable solid-server
+sudo systemctl start solid-server
+```
+
+### Option 3: Docker
+Create `docker-compose.yml`:
+```yaml
+version: '3'
+services:
+  solid-server:
+    image: node:20
+    working_dir: /app
+    volumes:
+      - ./:/app
+    environment:
+      - SIGHTENGINE_API_USER=your_api_user
+      - SIGHTENGINE_API_SECRET=your_api_secret
+    command: npx @solid/community-server -c config-with-moderation.json
+    ports:
+      - "3000:3000"
+```
+
+Run:
+```bash
+docker-compose up -d
+```
+
+### Option 4: PM2 Process Manager
+Create `ecosystem.config.js`:
+```javascript
+module.exports = {
+  apps: [{
+    name: 'solid-server',
+    script: 'npx',
+    args: '@solid/community-server -c config-with-moderation.json',
+    env: {
+      SIGHTENGINE_API_USER: 'your_api_user',
+      SIGHTENGINE_API_SECRET: 'your_api_secret'
+    }
+  }]
+};
+```
+
+Run:
+```bash
+pm2 start ecosystem.config.js
+pm2 save
+pm2 startup
+```
+
+## How It Works in Production
+
+### Local Testing vs Production
+
+| Aspect | Local Testing | Production |
+|--------|--------------|------------|
+| Plugin Location | `/Users/opendata/solid-moderation` (source) | `node_modules/@solid/moderation-plugin` (npm) |
+| Installation | `npm link` | `npm install @solid/moderation-plugin` |
+| Config Import | Full path to local file | `@solid/moderation-plugin:config/default.json` |
+| Components | Loaded from local `dist/` | Loaded from `node_modules/@solid/moderation-plugin/dist/` |
+
+### What Happens When CSS Starts
+
+1. **CSS reads your config file**
+2. **Sees import:** `@solid/moderation-plugin:config/default.json`
+3. **Resolves to:** `node_modules/@solid/moderation-plugin/config/default.json`
+4. **Loads components from:** `node_modules/@solid/moderation-plugin/dist/components/`
+5. **Instantiates:** ModerationOperationHandler wrapping PUT/POST/PATCH
+6. **Reads env vars:** `${SIGHTENGINE_API_USER}` → actual value
+7. **Server starts** with moderation active
+
+### File Locations in Production
+
+```
+your-css-project/
+├── node_modules/
+│   └── @solid/
+│       └── moderation-plugin/          ← Installed here
+│           ├── dist/
+│           │   ├── components/
+│           │   ├── *.js
+│           │   └── *.jsonld
+│           ├── config/
+│           │   └── default.json        ← Imported by your config
+│           └── package.json
+│
+├── config-with-moderation.json         ← Your config file
+├── data/
+│   └── moderation-logs/                ← Runtime logs
+└── package.json
+```
+
+## Customizing Thresholds in Production
+
+Override specific settings in your config:
+```json
+{
+  "import": [
+    "css:config/file.json",
+    "@solid/moderation-plugin:config/default.json"
+  ],
+  "@graph": [
+    {
+      "@id": "urn:solid-moderation:default:Config",
+      "ModerationConfig:_images_thresholds": {
+        "ModerationConfig:_images_thresholds_nudity": 0.3
+      }
+    }
+  ]
+}
+```
+
+## Monitoring
+
+Check logs:
+```bash
+# Systemd
+sudo journalctl -u solid-server -f
+
+# PM2
+pm2 logs solid-server
+
+# Docker
+docker-compose logs -f
+```
+
+View moderation violations:
+```bash
+cat data/moderation-logs/$(date +%Y-%m-%d).jsonl
+```