datajunction-ui 0.0.30 → 0.0.34

package/TODO.md

@@ -0,0 +1,265 @@
+# UI Test Improvements TODO
+
+## Phase 1: Fix Test Warnings (Completed)
+
+### 1.1 Fix `act()` Warnings ✅
+
+- [x] Wrap async state updates in `waitFor()` properly
+- [x] Add proper cleanup/waiting in tests for components with async effects
+- [x] Fixed: SQLBuilderPage, NotificationsPage, CubeBuilderPage tests
+- [ ] Remaining: Some AddEditNodePage/AddEditTagPage warnings (minor, from Formik internals)
+
+### 1.2 Fix Key Prop Warnings ✅
+
+- [x] Fix `key={rowValue}` in SQLBuilderPage (line 185) - use index-based key
+- [x] Fix `key={rowValue}` in NodeValidateTab (line 346) - use index-based key
+- [x] Fix missing key in QueryInfo.jsx error mapping (line 125)
+- [x] Fix incorrect mock data in Collapse.test.jsx causing undefined keys
+- [x] Fix missing key in NodeInfoTab.jsx: nodeTags, primary_key, required_dimensions, owners
+
+### 1.3 Fix DOM/React Warnings ✅
+
+- [x] Fix `value` prop on `textarea` should not be null - changed `custom_metadata: null` to `''` in AddEditNodePage
+- [x] Fix invalid DOM property `class` -> `className` in AddEditTagPage
+- [x] Fix nested elements: changed `<div>` to `<span>` in NodeInfoTab nodeTags
+- [x] Fix nested `<p>` tags: changed outer `<p>` to `<div>` in ListGroupItem (Markdown can render `<p>`)
+
+---
+
+## Phase 2: GraphQL Type Safety (In Progress)
+
+### Goal
+
+Migrate all REST GET endpoints to GraphQL for:
+
+- Field selection (fetch only what's needed)
+- Type safety via GraphQL codegen
+- Single source of truth for read operations
+
+Keep REST for mutations (POST/PATCH/DELETE) since they're already implemented and mutations are simpler.
+
+### Completed ✅
+
+#### 2.1 Type Generation Setup
+
+- [x] Added `@graphql-codegen/cli`, `@graphql-codegen/typescript`, `@graphql-codegen/typescript-operations`
+- [x] Added `openapi-typescript` for REST type generation
+- [x] Created `codegen.yml` pointing to static `schema.graphql` (no live server needed)
+- [x] Added scripts: `yarn codegen`, `yarn codegen:graphql`, `yarn codegen:openapi`, `yarn codegen:check`
+- [x] Generated types:
+  - `src/types/graphql.ts` (GraphQL types + query operation types)
+  - `src/types/openapi.ts` (REST endpoint types)
+- [x] Server Makefile updated with `make generate-schemas` and `make check-schemas`
+
+#### 2.2 GraphQL Query Files
+
+- [x] Created `src/graphql/queries/` directory with 10 `.graphql` files:
+  - `listNodesForLanding.graphql`
+  - `listCubesForPreset.graphql`
+  - `cubeForPlanner.graphql`
+  - `getNodeForEditing.graphql`
+  - `getNodesByNames.graphql`
+  - `getMetric.graphql`
+  - `getCubeForEditing.graphql`
+  - `upstreamNodes.graphql`
+  - `downstreamNodes.graphql`
+  - `getNodeColumnsWithPartitions.graphql`
+
+#### 2.3 Typed GraphQL Service
+
+- [x] Created `src/app/services/DJGraphQLService.ts` - fully typed GraphQL service
+- [x] All 10 existing GraphQL methods are now typed
+- [x] DJService.js GraphQL methods now delegate to DJGraphQLService.ts
+- [x] Components continue to use djClient context pattern (no changes needed)
+
+### Usage
+
+Components continue to use the existing djClient pattern - no changes needed:
+
+```jsx
+// Existing pattern (still works, now backed by typed service)
+const djClient = useContext(DJClientContext).DataJunctionAPI;
+const node = await djClient.getNodeForEditing('my.node');
+```
+
+For direct typed access (optional):
+
+```typescript
+import { DJGraphQLService } from '../services/DJGraphQLService';
+import type { GetNodeForEditingQuery } from '../../types/graphql';
+
+const node = await DJGraphQLService.getNodeForEditing('my.node');
+// node.name, node.type, node.current.displayName are all typed
+```
+
+### Development Workflow
+
+**Schema files (committed to repo):**
+| File | Source |
+|------|--------|
+| `openapi.json` (monorepo root) | Server OpenAPI spec |
+| `datajunction-server/.../schema.graphql` | Server GraphQL schema |
+| `datajunction-ui/src/types/graphql.ts` | Generated GraphQL types |
+| `datajunction-ui/src/types/openapi.ts` | Generated REST types |
+
+**When you change the server API:**
+
+```bash
+cd datajunction-server
+make generate-schemas  # Regenerates openapi.json + schema.graphql
+git add ../openapi.json datajunction_server/api/graphql/schema.graphql
+```
+
+**When you change UI GraphQL queries or pull server changes:**
+
+```bash
+cd datajunction-ui
+yarn codegen           # Regenerates graphql.ts + openapi.ts
+git add src/types/
+```
+
+**CI validation:**
+
+- Server: `make check` includes `make check-schemas` → fails if schemas are stale
+- UI: `yarn codegen:check` → fails if generated types are stale
+
+### Current State
+
+**Typed GraphQL methods (in DJGraphQLService.ts):**
+| Method | Purpose |
+|--------|---------|
+| `listNodesForLanding` | Paginated node listing |
+| `listCubesForPreset` | Cube dropdown |
+| `cubeForPlanner` | Query planner page |
+| `getNodeForEditing` | Edit node form |
+| `getNodesByNames` | Batch node fetch |
+| `getMetric` | Metric details |
+| `getCubeForEditing` | Edit cube form |
+| `upstreamNodes` | Upstream nodes |
+| `downstreamNodes` | Downstream nodes |
+| `getNodeColumnsWithPartitions` | Column partitions |
+
+### REST GET Endpoints to Migrate (~45 methods)
+
+#### High Priority (core functionality, frequently used)
+
+- [ ] `node` - GET /nodes/{name}/ → Full node details page
+- [ ] `nodes` - GET /nodes/?prefix= → Node listings
+- [ ] `nodesWithType` - GET /nodes/?node_type= → Filtered node lists
+- [ ] `metric` - GET /metrics/{name}/ → Metric details (REST version)
+- [ ] `metrics` - GET /metrics/ → All metrics list
+- [ ] `cube` - GET /cubes/{name}/ → Full cube details
+- [ ] `commonDimensions` - GET /metrics/common/dimensions/ → Dimension lookups
+- [ ] `upstreams` - GET /nodes/{name}/upstream/ → (has GQL version, deprecate REST)
+- [ ] `downstreams` - GET /nodes/{name}/downstream/ → (has GQL version, deprecate REST)
+- [ ] `history` - GET /history?node= → Activity history
+- [ ] `revisions` - GET /nodes/{name}/revisions/ → Node revision history
+
+#### Medium Priority (namespace/catalog/tags)
+
+- [ ] `namespaces` - GET /namespaces/ → Namespace list
+- [ ] `namespace` - GET /namespaces/{nmspce} → Namespace details
+- [ ] `namespaceSources` - GET /namespaces/{namespace}/sources
+- [ ] `catalogs` - GET /catalogs → Catalog list
+- [ ] `engines` - GET /engines → Engine list
+- [ ] `dimensions` - GET /dimensions → All dimensions
+- [ ] `nodeDimensions` - GET /nodes/{nodeName}/dimensions
+- [ ] `nodesWithDimension` - GET /dimensions/{name}/nodes/
+- [ ] `attributes` - GET /attributes
+- [ ] `listTags` - GET /tags → Tag list
+- [ ] `getTag` - GET /tags/{tagName} → Tag details
+- [ ] `listNodesForTag` - GET /tags/{tagName}/nodes
+- [ ] `users` - GET /users?with_activity=true
+
+#### Medium Priority (materializations)
+
+- [ ] `materializations` - GET /nodes/{node}/materializations
+- [ ] `availabilityStates` - GET /nodes/{node}/availability/
+- [ ] `materializationInfo` - GET /materialization/info
+- [ ] `listPreaggs` - GET /preaggs/
+- [ ] `getPreagg` - GET /preaggs/{preaggId}
+- [ ] `getCubeDetails` - GET /cubes/{cubeName}
+
+#### Lower Priority (SQL/data - may need special handling)
+
+- [ ] `sql` - GET /sql/{metric_name}?... → Generated SQL
+- [ ] `sqls` - GET /sql/?metrics=&dimensions= → Multi-metric SQL
+- [ ] `measuresV3` - GET /sql/measures/v3/ → V3 measures SQL
+- [ ] `metricsV3` - GET /sql/metrics/v3/ → V3 metrics SQL
+- [ ] `data` - GET /data/?metrics=&dimensions= → Query execution
+- [ ] `nodeData` - GET /data/{nodeName}? → Node data query
+- [ ] `compiledSql` - GET /sql/{node}/ → Compiled SQL
+
+#### Lower Priority (system metrics/admin)
+
+- [ ] `querySystemMetric` - GET /system/data/{metric} → Dashboard analytics
+- [ ] `system.node_counts_by_*` - System metrics
+- [ ] `system.dimensions` - GET /system/dimensions
+- [ ] `nodeDetails` - GET /nodes/details/
+- [ ] `node_dag` - GET /nodes/{name}/dag/
+- [ ] `node_lineage` - GET /nodes/{name}/lineage/
+- [ ] `listDeployments` - GET /deployments
+- [ ] `listMetricMetadata` - GET /metrics/metadata
+
+#### Lower Priority (notifications/export)
+
+- [ ] `getNotificationPreferences` - GET /notifications/
+- [ ] `getSubscribedHistory` - GET /history/?only_subscribed=true
+- [ ] `listServiceAccounts` - GET /service-accounts
+- [ ] `clientCode` - GET /datajunction-clients/python/new_node/{name}
+- [ ] `notebookExportCube` - GET /datajunction-clients/python/notebook/?cube=
+- [ ] `notebookExportNamespace` - GET /datajunction-clients/python/notebook/?namespace=
+
+#### Keep as REST (streaming - EventSource)
+
+- `stream` - EventSource /stream/... (GraphQL subscriptions are different tech)
+- `streamNodeData` - EventSource /stream/{nodeName}...
+
+### Implementation Tasks
+
+1. **Server-side: Extend GraphQL schema**
+
+   - [ ] Ensure all required fields are available in GraphQL schema
+   - [ ] Add any missing queries to the GraphQL API
+   - [ ] Verify introspection is enabled for codegen
+
+2. **UI-side: Set up GraphQL codegen**
+
+   - [ ] Add `@graphql-codegen/cli` and `@graphql-codegen/typescript` as dev dependencies
+   - [ ] Create `codegen.yml` configuration
+   - [ ] Add npm script: `yarn generate-types`
+   - [ ] Generate types to `src/types/graphql.ts`
+
+3. **UI-side: Migrate DJService methods**
+
+   - [ ] Start with high-priority methods
+   - [ ] Create GraphQL queries in `src/graphql/queries/`
+   - [ ] Update DJService.js methods to use GraphQL
+   - [ ] Remove deprecated REST calls
+
+4. **Type the mutation payloads**
+   - [ ] Keep OpenAPI types for REST mutations
+   - [ ] Add `openapi-typescript` for mutation request/response types
+   - [ ] Generate to `src/types/api-mutations.ts`
+
+### Benefits
+
+- **Field selection**: UI fetches only needed data
+- **Type safety**: Compile-time errors when code doesn't match schema
+- **Single type source for reads**: GraphQL schema is authoritative
+- **Performance**: Already proven with `cubeForPlanner`, `listNodesForLanding`
+
+---
+
+## Phase 3: Integration Tests (Long-term)
+
+### Goal
+
+Add integration test suite that runs against a real DJ server to catch real integration issues.
+
+### Tasks
+
+- [ ] Add Docker Compose setup for test DJ server
+- [ ] Create separate test suite for integration tests
+- [ ] Add CI workflow to run integration tests
+- [ ] Consider using Playwright or Cypress for E2E tests

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "datajunction-ui",
-  "version": "0.0.30",
+  "version": "0.0.34",
   "description": "DataJunction Metrics Platform UI",
   "module": "src/index.tsx",
   "repository": {

package/src/app/components/ListGroupItem.jsx

@@ -9,14 +9,14 @@ export default class ListGroupItem extends Component {
         <div className="d-flex gap-2 w-100 justify-content-between py-3">
           <div>
             <h6 className="mb-0 w-100">{label}</h6>
-            <p
+            <div
               className="mb-0 opacity-75"
               role="dialog"
               aria-hidden="false"
               aria-label={label}
             >
               <Markdown>{value}</Markdown>
-            </p>
+            </div>
           </div>
         </div>
       </div>