@datagrok/hit-triage 1.0.1

package/.eslintignore

@@ -0,0 +1 @@
+src/**/*.d.ts*

package/.eslintrc.json

@@ -0,0 +1,44 @@
+{
+  "env": {
+    "browser": true,
+    "es2022": true
+  },
+  "extends": [
+    "google"
+  ],
+  "parser": "@typescript-eslint/parser",
+  "parserOptions": {
+    "ecmaVersion": 12,
+    "sourceType": "module"
+  },
+  "plugins": [
+    "@typescript-eslint"
+  ],
+  "rules": {
+    "no-unused-vars": "off",
+    "@typescript-eslint/no-unused-vars": ["warn", { "varsIgnorePattern": "^(_|ui$|grok$|DG$)", "argsIgnorePattern": "^_"}],
+    "indent": [
+      "error",
+      2
+    ],
+    "max-len": [
+      "error",
+      120
+    ],
+    "require-jsdoc": "off",
+    "spaced-comment": "off",
+    "linebreak-style": "off",
+    "curly": [
+      "error",
+      "multi-or-nest"
+    ],
+    "brace-style": [
+      "error",
+      "1tbs",
+      {
+        "allowSingleLine": true
+      }
+    ],
+    "block-spacing": 2
+  }
+}

package/CHANGELOG.md

@@ -0,0 +1,3 @@
+# HitTriage changelog
+
+## 0.0.5 (WIP)

package/README.md

@@ -0,0 +1,26 @@
+# HitTriage
+
+The HitTriage package is a powerful tool designed for molecule analysis and campaign management within the Datagrok environment. It consists of two applications: HitTriage and HitDesign. This README provides an overview of the package's functionalities, and subsequent readmes will dive deeper into each application's usage.
+
+## Features
+
+Common Workflow
+
+1. **Template Creation:**
+
+    Define a template specifying the data source for molecules, name, key, additional needed information and compute functions. This source can be a file upload or a query in any other package tagged with `HitTriageDataSource` tag.
+    The Compute functions are collected from any package with a tag `HitTriageFunction`.
+
+    ![template](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/template.png?raw=true)
+
+2. **Campaign Building**:
+
+    Create campaigns based on the template.
+    Provide a campaign name, select the data source, provide additional information and initiate the campaign.
+    During the campaign run, the specified compute functions are executed, and their results are appended to the dataframe. For example, you can compute molecular descriptors, toxicity risks, structural alerts and more.
+
+    ![template](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/campaign.png?raw=true)
+
+After running a campaign, you can submit the dataframe to any chosen function or query. or
+save the campaign for later use. Saved campaigns can be reloaded and run again by any user on the platform usign the link or the campaigns table on the first page.
+

package/README_HD.md

@@ -0,0 +1,114 @@
+# HitDesign
+
+HitDesign streamlines the process of designing new molecules and collaborating on molecular ideas.
+You can sketch molecules,
+calculate or predict chemical and biological properties, enrich data with information
+from the proprietary databases, filter based on these properties or substructure search, select hits, 
+move hits to different stages, save campaigns, and share your work with others.
+
+The application is built around templates and campaigns, providing a structured yet flexible approach to
+hit design process. In addition to the built-in functions for data ingestion, property calculation, and
+data submission, you can define functions specific to your company or use case, allowing for the
+seamless integration. 
+
+## Templates
+
+Templates in HitDesign contain essential configurations for conducting a campaign. Template configuration includes:
+
+- **Name** : Identifies the template.
+
+- **Campaign Prefix** : A code used as a prefix for campaign names (e.g., TMP-1, TMP-2).
+
+- **Additional fields** : Configure additional fields for the template, which will be prompted for input during campaign creation. These fields include name, type, and whether they are required or not. For example, additional field for a campaign can be a target protein name, Head scientist name, deadlile, etc.
+
+- **Stages** : Define stages for the campaign. Tiles view provides a versatile way to organize molecules in the campaign. Users can drag and drop molecules between stages. For example, stages can be used to organize molecules by their readiness for synthesis.
+
+- **Compute functions** : HitDesign aggregates compute functions tagged with `HitTriageFunction` from Datagrok packages. Users can select from these functions to perform calculations (e.g., mass, solubility, mutagenicity, partial charges, toxicity risks, etc.) on the dataset.
+Every time user changes given molecule or adds new molecule to a dataframe, compute functions are executed automatically for that row.
+
+- **Submit function** : Define custom submit functions (tagged with `HitTriageSubmitFunction`) to further process or save the filtered and computed dataset. This could include saving to a private database or additional calculations.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/template-HD.png?raw=true)
+
+## Campaigns
+
+Campaigns are built based on templates and encompass the actual hit design process.
+
+- **Creation** : Select a template, fill out additional information and data source, and start the campaign.
+
+- **Collaboration** : Campaigns are automatically shared. Users can copy and share URLs for seamless collaboration.
+
+- **Functionality**: Once a campaign starts, you can add extra calculated columns, add new rows to dataframe and sketch molecules, apply changes, fileter, save or submit the campaign.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/campaign-HD.png?raw=true)
+
+## Getting started
+
+Continue ongoing campaigns either directly by a link or by selecting it from the campaigns table.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/continue_campaign_HD.gif?raw=true)
+
+Create a new template by clicking on the `New Template` button in the `Templates` dropdown.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/create-template-HD.gif?raw=true)
+
+Start a new campaign by choosing a template and filling out the required information. Once a new campaign starts, an empty dataframe will be added, where you can add new molecules. Upon adding or changing molecules, all compute functions will be executed on that row and the results in coresponding columns will be updated.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/HD-start-campaign.gif?raw=true)
+
+After the campaign starts, users can sketch new molecules, filter, modify or add viewers to the campaign and then save them. Once saved, reloading the campaign will restore the saved state.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/HD-save-campaign.gif?raw=true)
+
+Hit design campaign consists of two views, a main design view and a tiles view. You can access the tiles view from the views list. Tiles view provides a versatile way to organize molecules in the campaign. Users can drag and drop molecules between stages, which were defined in the template.
+
+![hitDesignReadmeImg](https://github.com/datagrok-ai/public/blob/master/help/uploads/hittriage/HD-tiles.gif?raw=true)
+
+## Adding custom compute and submit functions
+
+HitDesign allows users to define custom compute and submit functions, and these functions can be written in any Datagrok package that is installed in the environment. 
+
+** Compute functions **
+
+Compute functions are used to calculate molecular properties. For example, mass, solubility, mutagenicity, partial charges, toxicity risks, etc. By default, Hit design will include compute functions from `Chem` package, which are molecular descriptors, Structural alerts, Toxicity risks and Chemical properties. Users can add additional compute functions by tagging them with `HitDesignFunction` tag and writing them in normal datagrok style. The First two inputs of these functions should be `Dataframe` `table` and `Column` `molecule`, and rest can be any other input. Function should perform a certain task, modify the dataframe in desired way and return the modified dataframe. For example, we can create a function that retrieves the `Chembl` mol registration number by smiles string:
+
+```
+//name: Chembl molregno
+//tags: HitTriageFunction
+//input: dataframe table [Input data table] {caption: Table}
+//input: column molecules {caption: Molecules; semType: Molecule}
+//output: dataframe result
+export async function chemblMolregno(table: DG.DataFrame, molecules: DG.Column): Promise<DG.DataFrame> {
+  const name = table.columns.getUnusedName('CHEMBL molregno');
+  table.columns.addNewInt(name);
+  for (let i = 0; i < molecules.length; i++) {
+    const smile = molecules.get(i);
+    if (!smile) {
+      table.set(name, i, null);
+      continue;
+    }
+    const canonical = grok.chem.convert(smile, DG.chem.Notation.Unknown, DG.chem.Notation.Smiles);
+    const resDf: DG.DataFrame = await grok.data.query('Chembl:ChemblMolregNoBySmiles', {smiles: canonical});
+    const res: number = resDf.getCol('molregno').toList()[0];
+    table.set(name, i, res);
+  }
+  return table;
+}
+```
+
+This function will go through every molecule in the dataframe, convert them to canonical smiles and call the query from Chembl database, that will retrieve the molregno number. The result will be added as a new column to the dataframe. If this function is defined in the `Chembl` package, after building and deploying it to stand, it will be automatically added to the compute functions list in HitDesign.
+
+** Submit functions **
+
+Submit functions are used to save or submit the filtered and computed dataset. This could include saving to a private database or additional calculations. Submit functions are defined in the same way as compute functions, but they are tagged with `HitTriageSubmitFunction` tag. The function should accept only two inputs, `Dataframe` `df` and `String` `molecules`, which are the resulting dataframe and name of molecules column respectively. For example, we can create a function that saves the filtered and computed dataset to a database:
+
+```
+//name: Sample File Submit
+//tags: HitTriageSubmitFunction
+//input: dataframe df [dataframe]
+//input: string molecules [molecules column name]
+export async function demoFileSubmit(df: DG.DataFrame, molecules: string): Promise<void> {
+    const smiles = df.getCol(molecules).toList();
+    myCustomDatabase.add(smiles) // template function which could for example save the smiles to a database
+}
+```

package/README_HT.md

@@ -0,0 +1,63 @@
+# HitTriage
+
+HitTriage is a dynamic application designed for chemists and biologists to streamline the process of selecting molecular hits. It facilitates collaboration by allowing users to load datasets, calculate molecular properties, filter based on these properties or substructure search, select hits, save campaigns, and share their work with others. The application is built around templates and campaigns, providing a structured yet flexible approach to hit triage process.
+
+## Templates
+
+Templates in HitTriage contain essential configurations for conducting a campaign. template configuration includes:
+
+- **Name** : Identifies the template.
+
+- **Campaign Prefix** : A code used as a prefix for campaign names (e.g., TMP-1, TMP-2).
+
+- **Data Ingestion Settings** : Controls how molecular data is ingested into the template. Users can choose between file upload or a query option. For the query, HitTriage searches for functions in any Datagrok package tagged with `HitTriageDataSource`. For example:
+
+    ```//name: Demo File Ingestion
+    //input: int numberOfMolecules [Molecules count]
+    //tags: HitTriageDataSource
+    //output: dataframe result
+    export async function demoFileIngest(numberOfMolecules: number): Promise<DG.DataFrame> {
+    const df = grok.data.demo.molecules(numberOfMolecules);
+    df.name = 'Variable Molecules number';
+    return df;
+    }
+    ```
+    The application will detect that the function requeires an input parameter and will prompt the user to provide it in campaigns form. The function must return a dataframe with a column containing molecules
+
+- **Additional fields** : Users can configure additional fields for the template, which will be prompted for input during campaign creation. These fields include name, type, and whether they are required or not. For example, additional field for a campaign can be a target protein name, Head scientist name, deadlile, etc.
+
+- **Compute functions** : HitTriage aggregates compute functions tagged with `HitTriageFunction` from Datagrok packages. Users can select from these functions to perform calculations (e.g., mass, solubility, mutagenicity, partial charges, toxicity risks, etc.) on the dataset.
+
+- **Submit function** : Users can define custom submit functions (tagged with `HitTriageSubmitFunction`) to further process or save the filtered and computed dataset. This could include saving to a private database or additional calculations.
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/template.png)
+
+## Campaigns
+
+Campaigns are built based on templates and encompass the actual hit triage process.
+
+- **Creation** : Users select a template, fill out additional information and data source, and start the campaign.
+
+- **Collaboration** : Campaigns are automatically shared. Users can copy and share URLs for seamless collaboration.
+
+- **Functionality**: Once a campaign starts, you can add extra calculated columns, apply changes, fileter, save or submit the campaign.
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/campaign.png)
+
+## Getting started
+
+Users can continue ongoing campaigns either directly by a link or by selecting it from the campaigns table.
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/HT_Continue_campaign.gif)
+
+Users can create a new template by clicking on the `New Template` button in the `Templates` dropdown.
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/HT_create_template.gif)
+
+Users can start a new campaign by choosing a template and filling out the required information. 
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/HT_create_campaign.gif)
+
+After the campaign starts, users can filter, modify or add viewers to the campaign and then save them. once saved, reloading the campaign will restore the saved state.
+
+![template](https://datagrok.ai/help/uploads/hittriage/files/images/HT_save_campaign.gif)

package/README_OLD.md

@@ -0,0 +1,130 @@
+# HitTriage
+
+HitTriage helps chemists assess the quality of hits and decide which compounds should make it to the next stage. It does
+it in a managed, reproducible manner, with the triage template consisting of separate steps.
+
+![hit-triage-workflow](images/hit-triage-workflow.gif)
+
+Technically, HitTriage is a [package](https://datagrok.ai/help/develop/#packages)
+for the [Datagrok](https://datagrok.ai) platform that contains the the `HitTriage` application. There are multiple ways
+to start it:
+
+* Open the `Apps` pane and double-click on the `HitTriage` app
+* Open it via the direct link: [https://public.datagrok.ai/apps/HitTriage](https://public.datagrok.ai/apps/HitTriage)
+
+There are two outcomes for each project:
+
+1. Compounds that proceed to the next steps
+2. Rules used to filter out compounds
+
+## Project Template
+
+Project template defines the structure associated with the following:
+
+* Data Ingestion
+  * Data query + pre-processing
+* Data Expansion
+  * Calculated Properties
+  * Data from External Sources
+* Property Filters
+* Outcome Report Format
+* Actions and Triggers (notifications, etc)
+
+The project template is structured and persisted in a way that allows capturing of decision and "replaying" the whole
+pipeline on demand, should any step change and new source data appear.
+
+## Data Access
+
+Datagrok's built-in [data query](../../help/access/access.md#data-query) framework is used as a foundation for data access.
+Out-of-the box, it provides the following capabilities that are relevant to this project:
+
+1. Ability to access any data source (db, files, web service) efficiently
+    1. Managed connection
+2. Parameterization
+3. Reproducibility
+4. Audit and data lineage
+
+## Data Expansion
+
+During the data expansion step, the system brings in additional information necessary to make a decision on the
+compound. Depending on the nature of this information, it would be either calculated, or retrieved from external
+sources.
+
+For calculated data, Datagrok's functions framework is used. It allows to define operations (such as computing RDKit
+descriptors) that would be executed on the set of molecules.
+
+For retrieving additional information from external sources, the system uses
+[parameterized queries](../../help/access/access.md#data-query).
+
+## Data Storage
+
+The data is stored in the Postgres database. RDKit cartridge is used for efficient searches. Templates are stored in the
+same database.
+
+## User Interactions
+
+Users would use the built-in Datagrok facilities for filtering that allow interactive filtering for both compounds and
+calculated properties. The UI would look like that:
+
+![hit-triage-filtering](images/hit-triage-filtering.png)
+
+Users will be able to filter out compounds manually, with an optional explanation.
+
+Once the filters are setup, a user clicks the "Submit" button, at which point both hits and decisions get stored in the
+database, a report is generated, and a notification is sent to the project team.
+
+## Capturing Results
+
+There are two kinds of results: hits and decisions. Both are stored in the database.
+
+## Advanced Analytics
+
+The following HitTriage-related advanced analytics functionality is available out-of-the-box:
+
+* Structure rendering (RDKit or OpenChemLib)
+* Substructure search (either server-side or client-side)
+* Interactive data exploration
+* Clustering
+* Property Calculators
+* Chemical Space Exploration
+* Dimensionality Reduction (t-SNE, SPE)
+* Multivariate Analysis
+
+In addition, the solution takes advantage of the built-in data augmentation system. Whenever a user clicks on a
+structure, the relevant information (assays, projects, dose-response curves, etc)
+gets shown in the context panel on the right.
+
+## Visualization and Reporting Outcomes
+
+* Built-in [viewers](../../help/visualize/viewers/viewers.md)
+* Export to Excel / PDF
+* Share outcomes as URLs
+
+## Security and Privileges
+
+Datagrok's built-in [privileges system](https://datagrok.ai/help/govern/security)
+is used for authentication and authorization within HitTriage. With some app-specific customizations, it allows to
+
+* Define groups and associate them with privileges for objects (HitTriage projects, templates, etc)
+  and corresponding actions (create, delete, submit, etc)
+
+## Notifications and Statuses
+
+In the context of HitTriage, the built-in notification system is used for the following:
+
+* Requests for action (review, sign-off, etc)
+* Requests for privileges and group memberships (routed to group admins)
+
+Each notification
+
+## External APIs, integration and extension points
+
+While the system fit for the purpose of hit triage, it is also designed in a way that allows to easily customize it in
+order to integrate with different systems and processes within enterprises. Here are some of the integration and
+extension points:
+
+* Postgres DB: could be directly accessed for integration purposes
+* Data Ingestion - ability to define company-specific data ingestion routines
+* Data Expansion - ability to define custom calculations (including Python/RDKit, R, etc)
+* Datagrok REST API - programmatically access exposed functions
+* Datagrok events: ability to customize app behavior (potentially with other packages)

package/css/hit-triage.css

@@ -0,0 +1,98 @@
+.grok-hit-triage-view .d4-grid {
+    width: initial !important;
+    height: initial !important;
+}
+
+.hit-triage-compute-dialog-pane-header {
+    display: flex;
+    align-items: center;
+    justify-content: end;
+}
+
+.hit-triage-compute-dialog-host>.d4-tab-host.d4-tab-vertical>.d4-tab-header-stripe>.d4-tab-header.hit-triage-compute-dialog-pane-header {
+    border: none;
+    border-left: 4px solid transparent;
+    border-radius: 1px;
+    padding: 4px 8px;
+    min-height: 28px;
+    justify-content: flex-end;
+    flex-direction: row-reverse;
+}
+
+.hit-triage-compute-dialog-host>.d4-tab-host.d4-tab-vertical>.d4-tab-header-stripe>.d4-tab-header.hit-triage-compute-dialog-pane-header.selected {
+    border-left: 4px solid var(--blue-1);
+    background-color: #f2f2f5;
+}
+
+.hit-triage-compute-dialog-host>.d4-tab-host.d4-tab-vertical>.d4-tab-header-stripe>.d4-tab-header.hit-triage-compute-dialog-pane-header>span {
+    padding: 0 4px;
+}
+
+table.hit-triage-table {
+    margin: 10px;
+    width: calc(100% - 20px);
+}
+
+.hit-triage-table-container {
+    max-width: 600px;
+    max-height: 500px;
+    overflow-y: scroll;
+}
+
+.hit-triage-error-div {
+    color: red;
+    text-align: start;
+}
+
+.mx-5 {
+    margin-left: 5px;
+    margin-right: 5px;
+}
+
+.hit-triage-new-template-functions-input > .ui-box {
+    height: unset !important;
+}
+
+.hit-triage-new-template-functions-input {
+    max-height: 200px;
+    max-width: unset !important;
+    overflow-y: scroll;
+}
+
+
+.hit-triage-new-template-functions-input > .d4-tab-host {
+    width: 100% !important;
+}
+
+.hit-triage-compute-dialog-host {
+    max-height: 550px;
+    max-width: 500px;
+}
+
+.hit-triage-compute-dialog-descriptors-group {
+    margin-left: 16px;
+}
+
+.oy-scroll {
+    overflow-y: scroll;
+}
+
+.d4-ribbon-name .ui-breadcrumbs {
+	overflow: hidden;
+    white-space: nowrap;
+    text-overflow: ellipsis;
+	flex-wrap: nowrap;
+    padding: 0;
+}
+
+.d4-ribbon-name .ui-breadcrumbs i {
+	padding: 0 6px;
+	font-size: 10px;
+	font-weight: bold;
+}
+
+.d4-ribbon-name .ui-breadcrumbs-element:first-child {
+	white-space: nowrap;
+    overflow: hidden;
+    text-overflow: ellipsis;
+}