waha-shared 1.0.1

package/README.md

@@ -0,0 +1,10 @@
+# The Shared Directory
+
+Several of Waha's other projects require data, types, functions, and translations from the main repo. Specifically:
+
+- [waha-upload](https://github.com/kingdomstrategies/waha-upload)
+- [waha-web](https://github.com/kingdomstrategies/waha-web)
+- [waha-website](https://github.com/kingdomstrategies/waha-website)
+- [waha-dashboard](https://github.com/kingdomstrategies/waha-dashboard)
+
+This directory contains all of that shared data in one place so it can be easily copied. The ['copy.ts'](copy.ts) file simply copies the `shared` directory and all its contents from this repo into the root directory of whatever project it gets called in.

package/copy.ts

@@ -0,0 +1,37 @@
+import { cpSync, existsSync, mkdirSync, rmSync, writeFileSync } from 'fs'
+import path from 'path'
+import { fileURLToPath } from 'url'
+
+const scriptPath = fileURLToPath(import.meta.url)
+const sharedDir = path.dirname(scriptPath)
+
+try {
+  if (existsSync('shared')) {
+    /** Clear out shared directory first to make sure we have a fresh copy. */
+    rmSync('shared', { recursive: true })
+  }
+  mkdirSync('shared', { recursive: true })
+  cpSync(sharedDir, 'shared', { recursive: true })
+
+  /**
+   * Copy script should ONLY be run from the waha-app and thus should not be
+   * copied.
+   */
+  rmSync('shared/copy.ts', { force: true })
+
+  writeFileSync(
+    'shared/README.md',
+    [
+      '# The Shared Directory',
+      'Do not edit any files in this directory as they are automatically copied over from the main `waha-app` repo. Instead, edit the files in the main `waha-app` repo and then re-copy them to this project.',
+      '## Re-copying the Shared Directory',
+      'To update the contents of this directory, simply run the command `npx tsx PATH_TO_COPY_FILE_IN_THE_WAHA_APP_REPO` where `PATH_TO_COPY_FILE_IN_THE_WAHA_APP_REPO` is the path to the `copy.ts` file in the `waha-app` repo. For example, if the `waha-app` repo has the same parent directory as this project, you would run `npx tsx ../waha-app/shared/copy.ts`',
+    ].join('\n\n')
+  )
+  console.log(
+    `Successfully copied the shared folder from ${sharedDir} to shared`
+  )
+} catch (error) {
+  console.error('Error copying shared directory:', error)
+  process.exit(1)
+}

package/data/README.md

@@ -0,0 +1,45 @@
+# The `data` directory
+
+This directory contains a bunch of json data files that are used in various parts of Waha. Some are used for the mobile app, some for leviathan, and some for both.
+
+## Directories
+
+- [`bibleResources`](./bibleResources): Contains some miscellaneous files related to the Bible.
+  - [`audioBibleLicenses.json`](./bibleResources/audioBibleLicenses.json): Contains all audio Bible licenses that Waha has access to on the Digital Bible Library (DBL). Generated [here](/leviathan/scripts/get_licenses.py).
+  - [`bibleBooks.json`](./bibleResources/bibleBooks.json): List of books/chapters of the Bible. Used for the [Bible navigator component](/src/components/Bible/Navigator.tsx).
+  - [`bibleChaptersList.json`](./bibleResources/bibleChaptersList.json): List of all chapters in the Bible named by their USFM ids.
+  - [`textBibleLicenses.json`](./bibleResources/textBibleLicenses.json): Contains all text Bible licenses that Waha has access to on the DBL. Generated [here](/leviathan/scripts/get_licenses.py).
+
+- [`languageResources`](./languageResources): Contains some extra miscellaneous data files related to world languages.
+  - [`countriesAndLanguages.json`](./languageResources/countriesAndLanguages.json): Contains a list of all countries in the world and which languages are spoken there. Data scraped from [ethnologue](https://www.ethnologue.com) [here](/scripts/archive/getCountriesAndLanguages.py). Used for 2 purposes. Firstly, to populate the `countries` field for each language in [languages](./languages.json). See [`here`](/scripts/mapCountriesToLanguages.ts). This information is shown on the website to show which countries a language is most spoken in. Secondly, it's used in the mobile app to determine the default meet languages selected for a user when they go through onboarding based on their phone's country.
+  - [`iso6933LanguageCodes`](./languageResources/iso6933LanguageCodes.json): Contains a list of ISO 639-3 language codes. See [here](https://iso639-3.sil.org/code_tables/639/data). This is used to make sure we use existing language codes when adding new Waha languages, or to make sure that, if we are adding a language that doesn't have an existing code, we don't accidentally use a code that is already in use. In this case, the new language code should be manually added to this file so it doesn't get used accidentally in the future.
+  - [`mmsLanguages`](./languageResources/mmsLanguages.json): Contains a list of languages supported by Facebook's mms model. We use this model all over the place in `leviathan`, and use this list to make sure we are only running it on supported languages.
+  - [`phoneLanguages`](./languageResources/phoneLanguages.json): Contains a list of languages that phones support and potential alternatives if Waha doesn't support them. Used to set the user's default app interface language in the mobile app. See [here](/shared/functions/languages.ts) for more info.
+
+- [`python`](./python): Contains auto-generated python pydantic types for the data files. Generated using `scripts/makeTypes.ts`. This script takes the schemas found in the `schema` folder and creates pydantic types for using with Leviathan.
+- [`schemas`](./schemas): Contains json schemas for the data files. You can use your own json schema validator to validate the data files against these schemas as you update them. If you use VSCode, they are matched against their files [here](/.vscode/settings.json).
+- [`scripture`](./scripture): Contains full Scripture passages for all Bible translations to be used in Waha lessons. Gets populated by `scripts/getScripture.ts`. These are separated by Bible translation for convenience and clarity, but are combined into a single file in [`prebuild`](/scripts/prebuild.ts) that can be easily imported into the mobile app.
+- [`timings`](./timings): Contains data for the start times of each section (questions and scripture passages) of each Waha lesson, separated into files by language. These are used to skip to a specific section in the lesson and to highlight the text of a section when the audio reaches that point. These get written automatically [here](/leviathan/scripts/make_video_lessons.py). Like [`scripture`](scripture), these are combined into a single file in [`prebuild`](/scripts/prebuild.ts) that can be easily imported into the mobile app.
+- [`typescript`](./typescript): Contains auto-generated typescript interfaces for the data files. Generated using `scripts/makeTypes.ts`. This script takes the schemas found in the `schema` folder and creates Typescript interfaces for importing into the mobile app.
+- [`youtube`](./youtube): Contains data for Waha's Youtube videos and playlists.
+  - [`playlists.json`](./youtube/playlists.json): Contains ids for each language's Disciple Making Course Youtube playlist. This is populated by and used [here](/leviathan/scripts/update_youtube_dmc.py) to add videos to the correct playlists.
+  - [`videos.json`](./youtube/videos.json): Contains individual ids for each language's Disciple Making Course Youtube video. This is used in the mobile app to determine if we can show a "watch on Youtube" button and if so, link to the correct video. See [here](/shared/functions/sets.ts) where a youtube link is attached to a lesson. This is updated automatically [here](/leviathan/scripts/update_youtube_dmc.py) when videos are updated in Youtube.
+
+## Files
+
+- [`__init__.py`](./__init__.py): This file is used to make this directory importable into a python file. It is empty and not used for anything else.
+- [`allFiles.json`](./allFiles.json): Contains an array of all uploaded Waha lesson audio and video assets. This makes it really fast to check if a file exists in the mobile app, even when the app is offline, so we can hide or disable features for lessons that don't have all necessary assets. Generated during [`prebuild`](/scripts/prebuild.ts).
+- [`bibleAudios.json`](./bibleAudios.json): Contains information about Waha's audio Bible translations.
+- [`bibleTexts.json`](./bibleTexts.json): Contains information about Waha's text Bible translations.
+- [`clones.json`](./clones.json): Contains information about Waha "clones" or "white-labels": altered versions of Waha that are built from the same source code but are listed separately on the stores with different names, icons, etc. See `docs/clones.md` for more information.
+- [`foundationsCurriculums.json`](./foundationsCurriculums.json): Waha's various foundations curriculums. Matched with sets found in `data/sets.json` to get a language's story sets.
+- [`languages`](./languages.json): Contains a list of Waha's languages and a bunch of information about them.
+- [`notification`](./notification.json): The current push notification to be sent out with [sendPush](/scripts/sendPush.ts). `eng` should be the only language defined first. The rest will be automatically translated before sending.
+- [`questions.json`](./questions.json): Contains a list of Waha's questions and their associated data. This is used in the mobile app to show the questions in the lessons and in Leviathan to determine how to generate the questions. Similar to [`sets.json`](./sets.json), when adding new questions curriculums, any new questions must be added to this file.
+- [`questionsCurriculums.json`](./questionsCurriculums.json): Waha's various question curriculums.
+- [`releaseNotes.json`](./releaseNotes.json): Contains Waha's release notes. This is used in the mobile app to [show the release notes to users when they update the app](/src/hooks/useStartup.ts) and for development logs. This is updated automatically during [building](/scripts/build.ts).
+- [`scripture.json`](./scripture.json): Contains all scripture passages for all Bible translations to be used in Waha lessons. This is combined from the files in the [`scripture`](./scripture) directory and is used in the mobile app to show the scripture passages in the lessons. This is updated automatically during [building](/scripts/build.ts).
+- [`sets.json`](./sets.json): Contains a list of Waha's story sets across all curriculums and their associated data. When adding new story sets, any new sets must be added to this file.
+- [`specialIds.json`](./specialIds.json): Contains a list of miscellaneous set/lesson ids defined for convenience. For example, if wanting to determine if a lesson id is part of the disciple making course, you can use `specialIds.dmCourseLessonIds.includes(lessonId)`.
+- [`timings.json`](./timings.json): Contains data for the start times of each section (questions and scripture passages) of each Waha lesson. This is combined from the files in the [`timings`](./timings) directory and is used in the mobile app to skip to a specific section in the lesson and to highlight the text of a section when the audio reaches that point. This is updated automatically during [building](/scripts/build.ts).
+- [`topicsCurriculums.json`](./topicsCurriculums.json): Waha's various topics curriculums. Matched with sets found in `data/sets.json` to get a language's story sets.