@studyportals/fawkes 7.3.4 → 7.3.6

package/README.md

@@ -1,176 +1,176 @@
-# fawkes
-
-This project is used to centralize indexability logic for our Search
-microservices and Sitemap Generator. The main purpose is to completely remove
-discrepancies between pages indexed by Search and page URLs stored in Sitemap.
-
-Link to documentation: https://studyportals.atlassian.net/wiki/x/BwAdvw
-
-## Base structure
-
-- organisations-seo (used by Organisation Search)
-- scholarships-seo (used by Scholarship Search)
-- sitemap-generator-seo (used by Sitemap Generator)
-- src
-  - common
-  - organisations
-  - scholarships
-  - sitemap-generator
-
-# Automatic Documentation System
-
-## Overview
-
-This system automatically generates and maintains documentation for Fawkes
-indexing rules across multiple projects. It creates a hierarchical structure of
-Confluence pages that document all indexing policies and rules, ensuring that
-the documentation is always up-to-date with the actual codebase.
-
-Link to documentation:
-[Fawkes Indexing Rules Documentation](https://studyportals.atlassian.net/wiki/spaces/T/pages/3494969354/Fawkes+Indexing+Rules+Documentation)
-
-## Features
-
-- **Dynamic Documentation**: Automatically extracts indexing rules and policies
-  directly from the codebase
-- **Multi-Page Structure**: Creates a main index page with separate pages for
-  each project
-- **Hierarchical Organization**: Structures content as Projects → Policies →
-  Rules
-- **Automatic Page Creation**: Intelligently creates or updates pages as needed
-- **Cross-Page Navigation**: Provides intuitive navigation between all pages
-
-## Architecture
-
-The system follows SOLID design principles to ensure maintainability and
-extensibility:
-
-- **Single Responsibility**: Each class has one specific responsibility
-- **Open/Closed**: Easily extendable without modifying existing code
-- **Liskov Substitution**: Interchangeable components through well-defined
-  interfaces
-- **Interface Segregation**: Clean, focused interfaces
-- **Dependency Inversion**: High-level modules depend on abstractions
-
-### Key Components
-
-1. **Content Builder**: Extracts project, policy, and rule data from the Fawkes
-   system
-2. **Policy Fetchers**: Project-specific components that fetch and format policy
-   data
-3. **Document Builder**: Generates structured HTML content for Confluence pages
-4. **Confluence Updater**: Handles creation and updating of Confluence pages
-5. **Token Retriever**: Securely retrieves API token for Confluence access
-
-## How It Works
-
-1. The system starts by collecting all project data using the `ContentBuilder`
-2. For each project, it extracts policies and rules using project-specific
-   fetchers
-3. The `DocumentBuilder` generates the content for the main index page and each
-   project page
-4. The `ConfluenceUpdater` creates or updates pages in Confluence as needed
-5. Each project page links back to the main index, and the main index links to
-   all project pages
-
-## Setup and Configuration
-
-### Environment Variables
-
-The system requires the following environment variables:
-
-```
-CONFLUENCE_EMAIL=your-email@example.com
-CONFLUENCE_API_TOKEN=your-api-token
-CONFLUENCE_BASE_URL=https://your-instance.atlassian.net
-CONFLUENCE_SPACE_KEY=YOUR_SPACE_KEY
-CONFLUENCE_MAIN_PAGE_TITLE=Fawkes Indexing Rules Documentation
-```
-
-> **Important Note on Credentials**: Currently, the system uses personal
-> credentials (email and API token) for Confluence access. This is a temporary
-> solution and should be replaced with a team or service account in the future
-> to avoid dependency on a specific individual's account.
-
-### Deployment and Triggering
-
-The system runs as an AWS Lambda function and is automatically triggered as part
-of the CI/CD pipeline. This ensures documentation is always updated whenever new
-code is deployed:
-
-```yaml
-# buildspec.yml (excerpt)
-post_build:
-  commands:
-    - |
-      if [ -z "$PR_ID" ]; then
-        LAMBDA_NAME="FawkesConfluenceUpdater-$STAGE"
-        echo "Invoking Lambda function $LAMBDA_NAME"
-        echo '{"source":"cicd-pipeline","timestamp":"'$(date -u +"%Y-%m-%dT%H:%M:%SZ")'"}' > payload.json
-        aws lambda invoke --function-name $LAMBDA_NAME --invocation-type Event --payload fileb://payload.json response.json || echo "Lambda invocation failed, but continuing"
-      fi
-```
-
-**Trigger Flow:**
-
-1. When code is pushed to the `main` branch, the CI/CD pipeline is triggered
-2. After successful deployment of a new Fawkes version, the post-build phase
-   executes
-3. The Lambda function is invoked with a simple payload containing the source
-   and timestamp
-4. The Lambda function authenticates with Confluence and executes the
-   documentation generation process
-5. This only happens for production deployments (not PR builds)
-
-This automated approach ensures documentation stays in sync with the actual
-codebase without manual intervention.
-
-## Scalability and Extension
-
-### Adding New Projects
-
-To add documentation for a new project:
-
-1. Create a new class that implements the `IPolicyFetcher` interface
-2. Implement the `getProjectName()` and `fetchPolicies()` methods
-3. Register the new fetcher in the `ContentBuilder` constructor
-
-Example:
-
-```typescript
-export class NewProjectPolicyFetcher implements IPolicyFetcher {
-  getProjectName(): string {
-    return 'New Project Name';
-  }
-
-  fetchPolicies(): IPolicy[] {
-    // Implement policy fetching logic
-    return [...policies];
-  }
-}
-
-// Then update ContentBuilder:
-constructor(fetchers?: IPolicyFetcher[]) {
-  this.policyFetchers = fetchers || [
-    // Existing fetchers
-    new NewProjectPolicyFetcher()
-  ];
-}
-```
-
-### Customizing Page Content
-
-The `DocumentBuilder` class contains methods for generating page content. To
-customize the format:
-
-1. Modify the `buildMainDocument()` method for changes to the main index page
-2. Modify the `buildProjectDocument()` method for changes to project pages
-3. Modify the `buildPolicySection()` and `buildRulesTable()` methods for changes
-   to policy and rule sections
-
-## Conclusion
-
-This documentation system ensures that Fawkes indexing rules are always
-well-documented and accessible to the team. By automatically generating
-documentation directly from the codebase, it eliminates the risk of
-documentation becoming outdated or inconsistent with the actual implementation.
+# fawkes
+
+This project is used to centralize indexability logic for our Search
+microservices and Sitemap Generator. The main purpose is to completely remove
+discrepancies between pages indexed by Search and page URLs stored in Sitemap.
+
+Link to documentation: https://studyportals.atlassian.net/wiki/x/BwAdvw
+
+## Base structure
+
+- organisations-seo (used by Organisation Search)
+- scholarships-seo (used by Scholarship Search)
+- sitemap-generator-seo (used by Sitemap Generator)
+- src
+  - common
+  - organisations
+  - scholarships
+  - sitemap-generator
+
+# Automatic Documentation System
+
+## Overview
+
+This system automatically generates and maintains documentation for Fawkes
+indexing rules across multiple projects. It creates a hierarchical structure of
+Confluence pages that document all indexing policies and rules, ensuring that
+the documentation is always up-to-date with the actual codebase.
+
+Link to documentation:
+[Fawkes Indexing Rules Documentation](https://studyportals.atlassian.net/wiki/spaces/T/pages/3494969354/Fawkes+Indexing+Rules+Documentation)
+
+## Features
+
+- **Dynamic Documentation**: Automatically extracts indexing rules and policies
+  directly from the codebase
+- **Multi-Page Structure**: Creates a main index page with separate pages for
+  each project
+- **Hierarchical Organization**: Structures content as Projects → Policies →
+  Rules
+- **Automatic Page Creation**: Intelligently creates or updates pages as needed
+- **Cross-Page Navigation**: Provides intuitive navigation between all pages
+
+## Architecture
+
+The system follows SOLID design principles to ensure maintainability and
+extensibility:
+
+- **Single Responsibility**: Each class has one specific responsibility
+- **Open/Closed**: Easily extendable without modifying existing code
+- **Liskov Substitution**: Interchangeable components through well-defined
+  interfaces
+- **Interface Segregation**: Clean, focused interfaces
+- **Dependency Inversion**: High-level modules depend on abstractions
+
+### Key Components
+
+1. **Content Builder**: Extracts project, policy, and rule data from the Fawkes
+   system
+2. **Policy Fetchers**: Project-specific components that fetch and format policy
+   data
+3. **Document Builder**: Generates structured HTML content for Confluence pages
+4. **Confluence Updater**: Handles creation and updating of Confluence pages
+5. **Token Retriever**: Securely retrieves API token for Confluence access
+
+## How It Works
+
+1. The system starts by collecting all project data using the `ContentBuilder`
+2. For each project, it extracts policies and rules using project-specific
+   fetchers
+3. The `DocumentBuilder` generates the content for the main index page and each
+   project page
+4. The `ConfluenceUpdater` creates or updates pages in Confluence as needed
+5. Each project page links back to the main index, and the main index links to
+   all project pages
+
+## Setup and Configuration
+
+### Environment Variables
+
+The system requires the following environment variables:
+
+```
+CONFLUENCE_EMAIL=your-email@example.com
+CONFLUENCE_API_TOKEN=your-api-token
+CONFLUENCE_BASE_URL=https://your-instance.atlassian.net
+CONFLUENCE_SPACE_KEY=YOUR_SPACE_KEY
+CONFLUENCE_MAIN_PAGE_TITLE=Fawkes Indexing Rules Documentation
+```
+
+> **Important Note on Credentials**: Currently, the system uses personal
+> credentials (email and API token) for Confluence access. This is a temporary
+> solution and should be replaced with a team or service account in the future
+> to avoid dependency on a specific individual's account.
+
+### Deployment and Triggering
+
+The system runs as an AWS Lambda function and is automatically triggered as part
+of the CI/CD pipeline. This ensures documentation is always updated whenever new
+code is deployed:
+
+```yaml
+# buildspec.yml (excerpt)
+post_build:
+  commands:
+    - |
+      if [ -z "$PR_ID" ]; then
+        LAMBDA_NAME="FawkesConfluenceUpdater-$STAGE"
+        echo "Invoking Lambda function $LAMBDA_NAME"
+        echo '{"source":"cicd-pipeline","timestamp":"'$(date -u +"%Y-%m-%dT%H:%M:%SZ")'"}' > payload.json
+        aws lambda invoke --function-name $LAMBDA_NAME --invocation-type Event --payload fileb://payload.json response.json || echo "Lambda invocation failed, but continuing"
+      fi
+```
+
+**Trigger Flow:**
+
+1. When code is pushed to the `main` branch, the CI/CD pipeline is triggered
+2. After successful deployment of a new Fawkes version, the post-build phase
+   executes
+3. The Lambda function is invoked with a simple payload containing the source
+   and timestamp
+4. The Lambda function authenticates with Confluence and executes the
+   documentation generation process
+5. This only happens for production deployments (not PR builds)
+
+This automated approach ensures documentation stays in sync with the actual
+codebase without manual intervention.
+
+## Scalability and Extension
+
+### Adding New Projects
+
+To add documentation for a new project:
+
+1. Create a new class that implements the `IPolicyFetcher` interface
+2. Implement the `getProjectName()` and `fetchPolicies()` methods
+3. Register the new fetcher in the `ContentBuilder` constructor
+
+Example:
+
+```typescript
+export class NewProjectPolicyFetcher implements IPolicyFetcher {
+  getProjectName(): string {
+    return 'New Project Name';
+  }
+
+  fetchPolicies(): IPolicy[] {
+    // Implement policy fetching logic
+    return [...policies];
+  }
+}
+
+// Then update ContentBuilder:
+constructor(fetchers?: IPolicyFetcher[]) {
+  this.policyFetchers = fetchers || [
+    // Existing fetchers
+    new NewProjectPolicyFetcher()
+  ];
+}
+```
+
+### Customizing Page Content
+
+The `DocumentBuilder` class contains methods for generating page content. To
+customize the format:
+
+1. Modify the `buildMainDocument()` method for changes to the main index page
+2. Modify the `buildProjectDocument()` method for changes to project pages
+3. Modify the `buildPolicySection()` and `buildRulesTable()` methods for changes
+   to policy and rule sections
+
+## Conclusion
+
+This documentation system ensures that Fawkes indexing rules are always
+well-documented and accessible to the team. By automatically generating
+documentation directly from the codebase, it eliminates the risk of
+documentation becoming outdated or inconsistent with the actual implementation.

package/dist/src/programmes/policies/CountryAttendanceDegree.js

@@ -10,7 +10,7 @@ import { AttendancePresenter } from '../../presenters/AttendancePresenter';
 import { DegreePresenter } from '../../presenters/DegreePresenter';
 export class CountryAttendanceDegree extends ProgrammesBaseIndexabilityPolicy {
     name = 'Country Attendance Degree Policy';
-    description = `Determines indexing rules for pages filtered by country,
+    description = `Determines indexing rules for pages filtered by country,
 		study format (online, on-campus, etc.), and degree type.`;
     filterKeys = [FilterKey.COUNTRY, FilterKey.DELIVERY_METHOD, FilterKey.DEGREE_TYPE];
     rules = [

package/dist/src/programmes/policies/CountryDurationDegree.js

@@ -10,7 +10,7 @@ import { DegreePresenter } from '../../presenters/DegreePresenter';
 import { DurationPresenter } from '../../presenters/DurationPresenter';
 export class CountryDurationDegree extends ProgrammesBaseIndexabilityPolicy {
     name = 'Country Duration Degree Policy';
-    description = `Determines indexing rules for pages filtered by country,
+    description = `Determines indexing rules for pages filtered by country,
 		duration (1 year, 2 years, etc.), and degree type.`;
     filterKeys = [FilterKey.COUNTRY, FilterKey.DURATION, FilterKey.DEGREE_TYPE];
     rules = [

package/dist/src/programmes/policies/CountryEducationalForm.js

@@ -8,7 +8,7 @@ import { CountryPresenter } from '../../presenters/CountryPresenter';
 import { EducationalFormPresenter } from '../../presenters/EducationalFormPresenter';
 export class CountryEducationalForm extends ProgrammesBaseIndexabilityPolicy {
     name = 'Country Educational Form Policy';
-    description = `Determines indexing rules for pages filtered by both country,
+    description = `Determines indexing rules for pages filtered by both country,
 		and educational form (academic, semester, summer school, etc.).`;
     filterKeys = [FilterKey.COUNTRY, FilterKey.EDUCATIONAL_FORM];
     rules = [

package/dist/src/programmes/policies/DisciplineCountryEducationalForm.js

@@ -9,7 +9,7 @@ import { CountryPresenter } from '../../presenters/CountryPresenter';
 import { EducationalFormPresenter } from '../../presenters/EducationalFormPresenter';
 export class DisciplineCountryEducationalForm extends ProgrammesBaseIndexabilityPolicy {
     name = 'Discipline Country Educational Form Policy';
-    description = `Determines indexing rules for pages filtered by discipline,
+    description = `Determines indexing rules for pages filtered by discipline,
 		country, and educational form (academic, semester, summer school, etc.).`;
     filterKeys = [FilterKey.DISCIPLINES, FilterKey.COUNTRY, FilterKey.EDUCATIONAL_FORM];
     rules = [

package/dist/src/programmes/policies/DisciplineEducationalForm.js

@@ -8,7 +8,7 @@ import { DisciplinePresenter } from '../../presenters/DisciplinePresenter';
 import { EducationalFormPresenter } from '../../presenters/EducationalFormPresenter';
 export class DisciplineEducationalForm extends ProgrammesBaseIndexabilityPolicy {
     name = 'Discipline Educational Form Policy';
-    description = `Determines indexing rules for pages filtered by both discipline,
+    description = `Determines indexing rules for pages filtered by both discipline,
 		and educational form (academic, semester, summer school, etc.).`;
     filterKeys = [FilterKey.DISCIPLINES, FilterKey.EDUCATIONAL_FORM];
     rules = [

package/dist/src/structured-data/ProgrammeStructuredDataFactory.d.ts

@@ -0,0 +1,18 @@
+import { SearchStructuredDataFactory } from "./SearchStructuredDataFactory";
+import { IProgrammeCard } from "@studyportals/domain-client";
+import { ReviewRatingDTO } from "./dto/ReviewRatingDTO";
+import { OfferDTO } from "./dto/OfferDTO";
+import { Course } from "schema-dts";
+import { EntityDTO } from "./dto/EntityDTO";
+import { StructuredDataCurrencyConversionService } from "../structured-data/interface/StructuredDataCurrencyConversionService";
+export declare class ProgrammeStructuredDataFactory extends SearchStructuredDataFactory<IProgrammeCard> {
+    private currencyConversionService;
+    constructor(currencyConversionService: StructuredDataCurrencyConversionService);
+    private regionalDataPriority;
+    protected getRating(card: IProgrammeCard): ReviewRatingDTO | undefined;
+    protected getOfferData(card: IProgrammeCard): Promise<OfferDTO | undefined>;
+    protected buildStructuredDataForCard(entity: EntityDTO<IProgrammeCard>): Course | undefined;
+    private getCourseMode;
+    private getCourseWorkload;
+    private getCourseStartDate;
+}

package/dist/src/structured-data/ProgrammeStructuredDataFactory.js

@@ -0,0 +1,91 @@
+import { SearchStructuredDataFactory } from "./SearchStructuredDataFactory";
+export class ProgrammeStructuredDataFactory extends SearchStructuredDataFactory {
+    currencyConversionService;
+    constructor(currencyConversionService) {
+        super();
+        this.currencyConversionService = currencyConversionService;
+    }
+    regionalDataPriority = ['international', 'general', 'eea', 'national'];
+    getRating(card) {
+        if (typeof card.getReviewRating === 'undefined')
+            return;
+        const rating = card.getReviewRating();
+        if (!rating)
+            return;
+        return {
+            rating: rating?.getAverageStarRating(),
+            quantity: rating?.getQuantity(),
+        };
+    }
+    async getOfferData(card) {
+        if (!card.canShowTuitionFees())
+            return;
+        const tuitionFees = card.getTuitionFees();
+        let tuition;
+        let timing;
+        for (const target of this.regionalDataPriority) {
+            if (!tuition) {
+                tuition = tuitionFees.find(t => t.getTarget() === target);
+            }
+            if (!timing) {
+                timing = card.canShowTimings() ? card.getTimings().find(t => t.getType() === target && t.canShowStartDate()) : undefined;
+            }
+        }
+        if (!tuition) {
+            return;
+        }
+        return {
+            price: Math.round(await this.currencyConversionService.convert(tuition.getAmount(), tuition.getCurrency(), 'USD')),
+            category: 'tuition',
+            validFrom: timing ? timing.getStartDate().getDate() : undefined
+        };
+    }
+    buildStructuredDataForCard(entity) {
+        const offer = this.constructOffer(entity.offer);
+        if (!offer)
+            return undefined;
+        const card = entity.card;
+        return {
+            "@type": "Course",
+            "name": card.getTitle(),
+            "description": card.getSummary(),
+            "url": card.getProgrammeLink().getUrl(),
+            "provider": {
+                "@type": "EducationalOrganization",
+                "name": card.getUniversityLink().getDescription(),
+                "url": `https://${window.location.host}${card.getUniversityLink().getUrl()}`
+            },
+            "offers": offer,
+            "aggregateRating": this.constructAggregateRating(entity.reviewRating),
+            "hasCourseInstance": {
+                "@type": "CourseInstance",
+                "startDate": this.getCourseStartDate(card),
+                "courseWorkload": this.getCourseWorkload(card),
+                "courseMode": this.getCourseMode(card),
+            }
+        };
+    }
+    getCourseMode(card) {
+        if (card.isBlended())
+            return 'blended';
+        if (card.isOnline())
+            return 'online';
+        if (card.isOnCampus())
+            return 'onsite';
+    }
+    getCourseWorkload(card) {
+        if (card.isFullTime())
+            return 'PT40H';
+        if (card.isPartTime())
+            return 'PT20H';
+        return undefined;
+    }
+    getCourseStartDate(card) {
+        for (const target of this.regionalDataPriority) {
+            const timing = card.canShowTimings() ? card.getTimings().find(t => t.getType() === target && t.canShowStartDate()) : undefined;
+            if (timing) {
+                return timing.getStartDate().getDate().toISOString().split('T')[0];
+            }
+        }
+    }
+}

package/dist/src/structured-data/SearchStructuredDataFactory.d.ts

@@ -0,0 +1,34 @@
+import { SearchResultsPage, WithContext, Offer, AggregateRating, Thing, FAQPage } from 'schema-dts';
+import { ReviewRatingDTO } from './dto/ReviewRatingDTO';
+import { OfferDTO } from './dto/OfferDTO';
+import { EntityDTO } from './dto/EntityDTO';
+import { FAQItemDto } from './dto/FAQItemDto';
+import { BreadcrumbDTO } from './dto/BreadcrumbDTO';
+export declare abstract class SearchStructuredDataFactory<TCard> {
+    buildStructuredData(title: string, description: string, cards: TCard[], faqItems?: FAQItemDto[], breadcrumbs?: BreadcrumbDTO[]): Promise<WithContext<SearchResultsPage>>;
+    protected abstract buildStructuredDataForCard(entity: EntityDTO<TCard>): Thing | undefined;
+    /**
+     * Get the rating for a card.
+     * This method should be overridden by subclasses to provide the specific rating logic.
+     * If no rating is available, return undefined and no aggregate rating will be constructed for the card & page.
+     *
+     * @param card
+     * @protected
+     */
+    protected getRating(card: TCard): ReviewRatingDTO | undefined;
+    /**
+     * Get the offer data for a card.
+     * This method should be overridden by subclasses to provide the specific offer logic.
+     * If no offer is available, return undefined and no aggregate offer will be constructed for the card & page.
+     *
+     * @param card
+     * @protected
+     */
+    protected getOfferData(card: TCard): Promise<OfferDTO | undefined>;
+    private constructOptionalPageOffers;
+    protected constructOptionalFaqPage(faqItems: FAQItemDto[]): FAQPage | undefined;
+    private constructOptionalBreadcrumbs;
+    protected constructAggregateRating(reviewRating?: ReviewRatingDTO): AggregateRating | undefined;
+    protected constructOffer(offerData?: OfferDTO): Offer | undefined;
+    private stripHtmlWithListRetain;
+}