PyReprism 0.0.2__tar.gz → 0.0.4__tar.gz

pyreprism-0.0.4/.flake8

@@ -0,0 +1,7 @@
+[flake8]
+show-source = True
+count = True
+statistics = True
+; max-line-length = 150
+ignore = E501
+exclude = .github,venv,build,dist,docs,.venv,src/tests,src/PyReprism/languages/__init__.py,__test__.py

pyreprism-0.0.4/.github/FUNDING.yaml

@@ -0,0 +1 @@
+github: unlv-evol

pyreprism-0.0.4/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,17 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the bug. For example, a specific commit in a repository, and/or a modification.
+
+**OS Version:**
+ Windows/MacOS/Linux

pyreprism-0.0.4/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,17 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Example: I often have the problem of [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Additional context**
+Add any other information or screenshots about the feature request here.

pyreprism-0.0.4/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,13 @@
+* **What kind of change does this PR introduce?** (Bug fix, feature, docs update, ...)
+
+
+
+* **What is the current behavior?** (You can also link to an open issue here)
+
+
+
+* **What is the new behavior (if this is a feature change)?**
+
+
+
+* **Other information**:

pyreprism-0.0.4/.github/workflows/ci.yml

@@ -0,0 +1,41 @@
+name: CI
+on:
+  push:
+    branches-ignore:
+      - main
+  pull_request:
+    branches-ignore:
+      - main
+
+jobs:
+  publish:
+    name: "Lint, test and upload to codecov"
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the code
+        uses: actions/checkout@v3
+
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: '3.x'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install flake8 pytest pytest-cov
+
+      - name: Lint with flake8
+        run: |
+          flake8 .
+
+      - name: Run tests with pytest
+        run: |
+          pytest --cov=src/ --cov-report=xml
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v4
+        with:
+          verbose: true
+        env:
+          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

pyreprism-0.0.4/.github/workflows/publish.yml

@@ -0,0 +1,38 @@
+name: Publish Python 🐍 distributions 📦 to PyPI
+on:
+  push:
+    tags:
+      - 'v*' 
+    branches:
+      - main 
+      - stable
+
+jobs:
+  publish:
+    name: "Build and upload to pypi"
+    runs-on: "ubuntu-latest"
+    # environment:
+    #   name: pypi
+    #   url: https://pypi.org/project/PyReprism/
+    # permissions:
+    #   id-token: write
+
+    steps:
+      - uses: actions/checkout@v3
+      - name: Install python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.11' 
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install hatchling twine build hatch-vcs
+      - name: Build app
+        run: |
+          python -m build
+      - name: Publish distribution 📦 to PyPI
+        if: startsWith(github.ref, 'refs/tags')
+        uses: pypa/gh-action-pypi-publish@release/v1
+        with:
+          password: ${{ secrets.PYPI_API_TOKEN }}
+        # run: twine upload dist/*

pyreprism-0.0.4/.readthedocs.yaml

@@ -0,0 +1,31 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+   os: ubuntu-22.04
+   tools:
+     python: "3.10"
+     # You can also specify other tool versions:
+     # nodejs: "16"
+     # rust: "1.55"
+     # golang: "1.17"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+   configuration: docs/conf.py
+
+# If using Sphinx, optionally build your docs in additional formats such as PDF
+# formats:
+#    - pdf
+
+# Optionally declare the Python requirements required to build your docs
+python:
+   install:
+   - requirements: docs/requirements.txt
+   - method: pip
+     path: .

pyreprism-0.0.4/CONTRIBUTING.md

@@ -0,0 +1,65 @@
+# PyReprism Contributing Guide
+
+Thank you for investing your time in contributing to this project!
+
+In this guide you will get an overview of the contribution workflow from opening an issue, creating a PR, reviewing, and merging the PR.
+
+## New contributor guide
+
+To get an overview of this project, to read the `README`:
+
+- [README](README.md)
+Here are some resources to help you get started with open source contributions:
+
+- [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/)
+- [Set up Git](https://docs.github.com/en/get-started/quickstart/set-up-git)
+- [Collaborating with pull requests](https://docs.github.com/en/github/collaborating-with-pull-requests)
+
+## Getting started
+
+### Issues
+
+#### Create a new issue
+
+If you spot a problem with any of our product, check the [existing issues](https://github.com/unlv-evol/PyReprism/issues) to establish if the problem has already been reported. If a related issue doesn't exist, you can open a new issue using a relevant [issue form](https://github.com/unlv-evol/PyReprism/issues/new/choose).
+
+#### Solve an issue
+
+Scan through the [existing issues](https://github.com/unlv-evol/PyReprism/issues) to find one that interests you. You can narrow down the search using `labels` as filters. If you find an issue to work on, you are welcome to open a PR with a fix.
+
+**Note** that you don't need to be assigned to this issue to work on it. Simply raise a PR that address it and add `closes #issue-number` in the PR description to [automatically link your PR to the issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword).
+
+### Make Changes
+
+#### Make changes locally
+
+1. [Fork the repository](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo#fork-an-example-repository).
+
+2. [Create a working branch](https://docs.github.com/en/issues/tracking-your-work-with-issues/creating-a-branch-for-an-issue) and start with your changes!
+
+## Style Guides
+We try our best to adhere to the style guide for Python which is the main languages for this project. Please ensure that you follow the style guides according.
+- [Python](https://google.github.io/styleguide/pyguide.html)
+
+### Commit your update
+
+Commit the changes, with a [good commit message](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html) once you are happy with them.
+
+Once your changes are ready, don't forget to test and self-review them to speed up the review process:zap:.
+
+### Pull Request
+
+When you're finished with the changes, create a pull request, also known as a PR.
+
+- Fill the "Ready for review" template so that we can review your PR. This template helps reviewers understand your changes as well as the purpose of your pull request.
+- Don't forget to [link PR to issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue) if you are solving one.
+- Enable the checkbox to [allow maintainer edits](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/allowing-changes-to-a-pull-request-branch-created-from-a-fork) so the branch can be updated for a merge.
+  Once you submit your PR, maintainers will review your proposal.
+- We may ask for changes to be made before a PR can be merged. You can apply suggested changes and/or make any other changes in your fork, then commit them to your branch.
+- As you update your PR and apply changes, mark each conversation as [resolved](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/commenting-on-a-pull-request#resolving-conversations).
+- If you run into any merge conflict issues, checkout this [git tutorial](https://github.com/skills/resolve-merge-conflicts) to help you resolve merge conflicts and other issues. 
+
+### Your PR is merged!
+
+Congratulations :tada: :tada: <br />
+The PyReprism team thanks you :sparkles:.

pyreprism-0.0.4/PKG-INFO

@@ -0,0 +1,98 @@
+Metadata-Version: 2.3
+Name: PyReprism
+Version: 0.0.4
+Summary: Framework for Source Code Preprocessing
+Project-URL: Homepage, https://github.com/unlv-evol/PyReprism
+Project-URL: Documentation, https://pyreprism.readthedocs.io
+Project-URL: Issues, https://github.com/unlv-evol/PyReprism/issues
+Author-email: UNLV EVOL LAB <ogenrwot@unlv.nevada.edu>
+License: MIT License
+        
+        Copyright (c) 2024 UNLV EVOL
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.6
+Provides-Extra: dev
+Requires-Dist: flake8; extra == 'dev'
+Requires-Dist: pytest; extra == 'dev'
+Requires-Dist: pytest-cov; extra == 'dev'
+Description-Content-Type: text/x-rst
+
+.. _intro_toplevel:
+
+==================
+Overview / Install
+==================
+
+PyReprism is a Python framework that helps researchers and developers the task of source code preprocessing. With PyReprism, you can easily match, extract, count, and remove comments, whitespaces, operators, numbers and other language specific constructs from over 150 programming languages and file extensions.
+
+
+Requirements
+============
+
+* `Python`_ 3.6 or newer
+* `Git`_
+
+.. _Python: https://www.python.org
+.. _Git: https://git-scm.com/
+
+Installing PyReprism
+====================
+
+Installing PyReprism is easily done using `pip`_. Assuming it is installed, just run the following from the command-line:
+
+.. _pip: https://pip.pypa.io/en/latest/installing.html
+
+.. sourcecode:: none
+
+    $ pip install PyReprism
+
+Source Code
+===========
+
+PyReprism's git repo is available on GitHub, which can be browsed at:
+
+ * https://github.com/unlv-evol/PyReprism.git
+
+and cloned using::
+
+    $ git clone https://github.com/unlv-evol/PyReprism.git
+    $ cd PyReprism
+
+Optionally (but suggested), make use of virtual environment. Therefore, before installing the requirements run::
+    
+    $ python3 -m venv venv
+    $ source venv/bin/activate
+
+Install the requirements::
+    
+    $ pip install -r requirements.txt
+
+and run the tests using pytest::
+
+    $ pytest

pyreprism-0.0.4/README.md

@@ -0,0 +1,96 @@
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+[![Downloads](https://static.pepy.tech/badge/pyreprism)](https://pepy.tech/project/pyreprism)
+[![FOSSA Status](https://app.fossa.com/api/projects/custom%2B46484%2Fgithub.com%2Funlv-evol%2FPyReprism.svg?type=shield&issueType=license)](https://app.fossa.com/projects/custom%2B46484%2Fgithub.com%2Funlv-evol%2FPyReprism?ref=badge_shield&issueType=license)
+[![FOSSA Status](https://app.fossa.com/api/projects/custom%2B46484%2Fgithub.com%2Funlv-evol%2FPyReprism.svg?type=shield&issueType=security)](https://app.fossa.com/projects/custom%2B46484%2Fgithub.com%2Funlv-evol%2FPyReprism?ref=badge_shield&issueType=security)
+[![Documentation Status](https://readthedocs.org/projects/pyreprism/badge/?version=latest)](https://pyreprism.readthedocs.io/en/latest/?badge=latest)
+![PyPI - Python Version](https://img.shields.io/pypi/pyversions/PyReprism)
+![CI](https://github.com/unlv-evol/PyReprism/actions/workflows/ci.yml/badge.svg)
+![Publish](https://github.com/unlv-evol/PyReprism/actions/workflows/publish.yml/badge.svg)
+[![codecov](https://codecov.io/gh/unlv-evol/PyReprism/graph/badge.svg?token=J2JV31837H)](https://codecov.io/gh/unlv-evol/PyReprism)
+![PyPI - Version](https://img.shields.io/pypi/v/pyreprism)
+![GitHub last commit (branch)](https://img.shields.io/github/last-commit/unlv-evol/pyreprism/main)
+[![SWH](https://archive.softwareheritage.org/badge/origin/https://pypi.org/project/PyReprism//)](https://archive.softwareheritage.org/browse/origin/?origin_url=https://pypi.org/project/PyReprism/)
+
+# PyReprism
+
+PyReprism is a Python framework that helps researchers and developers the task of source code preprocessing. With PyReprism, you can easily match, extract, count, and remove comments, whitespaces, operators, numbers and other language specific constructs from over **150 programming languages and file extensions**.
+
+## Install
+```shell
+pip install PyReprism
+```
+## Quick Usage
+Use case 1: Removing comments 
+```python
+
+from PyReprism.languages import Python
+# from PyReprism.languages import Java
+
+source = """
+# single line comment
+x = 5 + 6
+'''
+multiline
+comment
+'''
+print(x)
+"""
+
+source = Python.remove_comments(source)
+
+# expected output
+
+x = 5 + 6
+
+
+print(x)
+
+```
+
+Use case 2: Removing whitespaces 
+```python
+from PyReprism.utils.normalizer import Normalizer
+source = """
+
+x = 5 + 6
+
+
+print(x)
+
+"""
+
+
+source = Normalizer.remove_whitespaces(source)
+
+# expected output
+x=5+6
+print(x)
+```
+
+Read the [docs](https://pyreprism.readthedocs.io) for more usage examples. 
+
+NB: The beta versions of PyReprism is still unstable, but we are working 24/7 to ensure the tool is usable.
+
+## How to Contribute
+We invite you to help us build this tool and make it more extensive. Contribution is open to OSS community.
+
+```shell
+$ git clone https://github.com/unlv-evol/PyReprism.git
+$ cd PyReprism
+```
+**(Optional)** It is suggested to make use of virtualenv. Therefore, before installing the requirements run:
+
+```shell
+$ python3 -m venv venv
+$ source venv/bin/activate
+```
+
+Then, install the requirements:
+
+```shell
+$ pip install -r requirements.txt
+```
+For more information on how to contribute, read our [contributing guidelines](CONTRIBUTING.md).
+
+## Issues
+If you experience any issue, feel free to report it.

pyreprism-0.0.4/codecov.yml

@@ -0,0 +1,13 @@
+flag_management:
+  individual_flags:
+    - name: smart-tests
+      carryforward: true
+      carryforward_mode: "labels"
+      statuses:
+        - type: "project"
+        - type: "patch"
+
+cli:
+  plugins:
+    pycoverage:
+      report_type: "json"

pyreprism-0.0.4/dev-requirements.txt

@@ -0,0 +1,9 @@
+pytest
+flake8
+pytest-cov
+build
+twine
+hatchling
+hatch-vcs
+sphinx
+sphinx_rtd_theme

pyreprism-0.0.4/docs/intro.rst

@@ -0,0 +1,53 @@
+.. _intro_toplevel:
+
+==================
+Overview / Install
+==================
+
+PyReprism is a Python framework that helps researchers and developers the task of source code preprocessing. With PyReprism, you can easily match, extract, count, and remove comments, whitespaces, operators, numbers and other language specific constructs from over 150 programming languages and file extensions.
+
+
+Requirements
+============
+
+* `Python`_ 3.6 or newer
+* `Git`_
+
+.. _Python: https://www.python.org
+.. _Git: https://git-scm.com/
+
+Installing PyReprism
+====================
+
+Installing PyReprism is easily done using `pip`_. Assuming it is installed, just run the following from the command-line:
+
+.. _pip: https://pip.pypa.io/en/latest/installing.html
+
+.. sourcecode:: none
+
+    $ pip install PyReprism
+
+Source Code
+===========
+
+PyReprism's git repo is available on GitHub, which can be browsed at:
+
+ * https://github.com/unlv-evol/PyReprism.git
+
+and cloned using::
+
+    $ git clone https://github.com/unlv-evol/PyReprism.git
+    $ cd PyReprism
+
+Optionally (but suggested), make use of virtual environment. Therefore, before installing the requirements run::
+    
+    $ python3 -m venv venv
+    $ source venv/bin/activate
+
+Install the requirements::
+    
+    $ pip install -r requirements.txt
+
+and run the tests using pytest::
+
+    $ pytest

{pyreprism-0.0.2 → pyreprism-0.0.4}/pyproject.toml

@@ -1,28 +1,26 @@
 [build-system]
-requires = ["hatchling"]
+requires = ["hatchling", "hatch_vcs"]
 build-backend = "hatchling.build"
 
 [project]
 name = "PyReprism"
-version = "0.0.2"
+version = "0.0.4"
 authors = [
   { name= "UNLV EVOL LAB", email="ogenrwot@unlv.nevada.edu" },
 ]
+
 license={file="LICENSE" }
 python_requires = ">=3.5"
 description = "Framework for Source Code Preprocessing"
-readme = "README.md"
+readme = "docs/intro.rst"
 requires-python = ">=3.6"
 classifiers = [
     "Environment :: Console",
-    "Development Status :: 4 - Beta",
+    "Development Status :: 5 - Production/Stable",
     "Intended Audience :: Developers",
     "Intended Audience :: Science/Research",
-    "Programming Language :: Python :: 3.6",
-    "Programming Language :: Python :: 3.7",
-    "Programming Language :: Python :: 3.8",
-    "Programming Language :: Python :: 3.9",
-    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.12",
     "License :: OSI Approved :: MIT License",
     "Operating System :: OS Independent",
 ]
@@ -45,4 +43,5 @@ dev = [
 
 [project.urls]
 Homepage = "https://github.com/unlv-evol/PyReprism"
+Documentation = "https://pyreprism.readthedocs.io"
 Issues = "https://github.com/unlv-evol/PyReprism/issues"

pyreprism-0.0.4/src/PyReprism/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.0.4"

{pyreprism-0.0.2 → pyreprism-0.0.4}/src/PyReprism/languages/__init__.py

@@ -1,3 +1,4 @@
+__version__ = "0.0.3"
 from .python import *
 from .java import *
 from .c import *
@@ -114,7 +115,6 @@ from .pure import *
 from .q import *
 from .qore import * 
 from .r import * 
-from .r import * 
 from .reason import * 
 from .renpy import * 
 from .rest import * 

pyreprism-0.0.4/src/PyReprism/languages/abap.py

@@ -0,0 +1,50 @@
+import re
+from PyReprism.utils import extension
+
+
+class Abap:
+    def __init__(self):
+        pass
+
+    @staticmethod
+    def file_extension() -> str:
+        return extension.abap
+
+    @staticmethod
+    def keywords() -> list:
+        # keyword = 'SCIENTIFIC_WITH_LEADING_ZERO|SCALE_PRESERVING_SCIENTIFIC|RMC_COMMUNICATION_FAILURE|END-ENHANCEMENT-SECTION|MULTIPLY-CORRESPONDING|SUBTRACT-CORRESPONDING|VERIFICATION-MESSAGE|DIVIDE-CORRESPONDING|ENHANCEMENT-SECTION|CURRENCY_CONVERSION|RMC_SYSTEM_FAILURE|START-OF-SELECTION|MOVE-CORRESPONDING|RMC_INVALID_STATUS|CUSTOMER-FUNCTION|END-OF-DEFINITION|ENHANCEMENT-POINT|SYSTEM-EXCEPTIONS|ADD-CORRESPONDING|SCALE_PRESERVING|SELECTION-SCREEN|CURSOR-SELECTION|END-OF-SELECTION|LOAD-OF-PROGRAM|SCROLL-BOUNDARY|SELECTION-TABLE|EXCEPTION-TABLE|IMPLEMENTATIONS|PARAMETER-TABLE|RIGHT-JUSTIFIED|UNIT_CONVERSION|AUTHORITY-CHECK|LIST-PROCESSING|SIGN_AS_POSTFIX|COL_BACKGROUND|IMPLEMENTATION|INTERFACE-POOL|TRANSFORMATION|IDENTIFICATION|ENDENHANCEMENT|LINE-SELECTION|INITIALIZATION|LEFT-JUSTIFIED|SELECT-OPTIONS|SELECTION-SETS|COMMUNICATION|CORRESPONDING|DECIMAL_SHIFT|PRINT-CONTROL|VALUE-REQUEST|CHAIN-REQUEST|FUNCTION-POOL|FIELD-SYMBOLS|FUNCTIONALITY|INVERTED-DATE|SELECTION-SET|CLASS-METHODS|OUTPUT-LENGTH|CLASS-CODING|COL_NEGATIVE|ERRORMESSAGE|FIELD-GROUPS|HELP-REQUEST|NO-EXTENSION|NO-TOPOFPAGE|REDEFINITION|DISPLAY-MODE|ENDINTERFACE|EXIT-COMMAND|FIELD-SYMBOL|NO-SCROLLING|SHORTDUMP-ID|ACCESSPOLICY|CLASS-EVENTS|COL_POSITIVE|DECLARATIONS|ENHANCEMENTS|FILTER-TABLE|SWITCHSTATES|SYNTAX-CHECK|TRANSPORTING|ASYNCHRONOUS|SYNTAX-TRACE|TOKENIZATION|USER-COMMAND|WITH-HEADING|ABAP-SOURCE|BREAK-POINT|CHAIN-INPUT|COMPRESSION|FIXED-POINT|NEW-SECTION|NON-UNICODE|OCCURRENCES|RESPONSIBLE|SYSTEM-CALL|TRACE-TABLE|ABBREVIATED|CHAR-TO-HEX|END-OF-FILE|ENDFUNCTION|ENVIRONMENT|ASSOCIATION|COL_HEADING|EDITOR-CALL|END-OF-PAGE|ENGINEERING|IMPLEMENTED|INTENSIFIED|RADIOBUTTON|SYSTEM-EXIT|TOP-OF-PAGE|TRANSACTION|APPLICATION|CONCATENATE|DESTINATION|ENHANCEMENT|IMMEDIATELY|NO-GROUPING|PRECOMPILED|REPLACEMENT|TITLE-LINES|ACTIVATION|BYTE-ORDER|CLASS-POOL|CONNECTION|CONVERSION|DEFINITION|DEPARTMENT|EXPIRATION|INHERITING|MESSAGE-ID|NO-HEADING|PERFORMING|QUEUE-ONLY|RIGHTSPACE|SCIENTIFIC|STATUSINFO|STRUCTURES|SYNCPOINTS|WITH-TITLE|ATTRIBUTES|BOUNDARIES|CLASS-DATA|COL_NORMAL|DD\/MM\/YYYY|DESCENDING|INTERFACES|LINE-COUNT|MM\/DD\/YYYY|NON-UNIQUE|PRESERVING|SELECTIONS|STATEMENTS|SUBROUTINE|TRUNCATION|TYPE-POOLS|ARITHMETIC|BACKGROUND|ENDPROVIDE|EXCEPTIONS|IDENTIFIER|INDEX-LINE|OBLIGATORY|PARAMETERS|PERCENTAGE|PUSHBUTTON|RESOLUTION|COMPONENTS|DEALLOCATE|DISCONNECT|DUPLICATES|FIRST-LINE|HEAD-LINES|NO-DISPLAY|OCCURRENCE|RESPECTING|RETURNCODE|SUBMATCHES|TRACE-FILE|ASCENDING|BYPASSING|ENDMODULE|EXCEPTION|EXCLUDING|EXPORTING|INCREMENT|MATCHCODE|PARAMETER|PARTIALLY|PREFERRED|REFERENCE|REPLACING|RETURNING|SELECTION|SEPARATED|SPECIFIED|STATEMENT|TIMESTAMP|TYPE-POOL|ACCEPTING|APPENDAGE|ASSIGNING|COL_GROUP|COMPARING|CONSTANTS|DANGEROUS|IMPORTING|INSTANCES|LEFTSPACE|LOG-POINT|QUICKINFO|READ-ONLY|SCROLLING|SQLSCRIPT|STEP-LOOP|TOP-LINES|TRANSLATE|APPENDING|AUTHORITY|CHARACTER|COMPONENT|CONDITION|DIRECTORY|DUPLICATE|MESSAGING|RECEIVING|SUBSCREEN|ACCORDING|COL_TOTAL|END-LINES|ENDMETHOD|ENDSELECT|EXPANDING|EXTENSION|INCLUDING|INFOTYPES|INTERFACE|INTERVALS|LINE-SIZE|PF-STATUS|PROCEDURE|PROTECTED|REQUESTED|RESUMABLE|RIGHTPLUS|SAP-SPOOL|SECONDARY|STRUCTURE|SUBSTRING|TABLEVIEW|NUMOFCHAR|ADJACENT|ANALYSIS|ASSIGNED|BACKWARD|CHANNELS|CHECKBOX|CONTINUE|CRITICAL|DATAINFO|DD\/MM\/YY|DURATION|ENCODING|ENDCLASS|FUNCTION|LEFTPLUS|LINEFEED|MM\/DD\/YY|OVERFLOW|RECEIVED|SKIPPING|SORTABLE|STANDARD|SUBTRACT|SUPPRESS|TABSTRIP|TITLEBAR|TRUNCATE|UNASSIGN|WHENEVER|ANALYZER|COALESCE|COMMENTS|CONDENSE|DECIMALS|DEFERRED|ENDWHILE|EXPLICIT|KEYWORDS|MESSAGES|POSITION|PRIORITY|RECEIVER|RENAMING|TIMEZONE|TRAILING|ALLOCATE|CENTERED|CIRCULAR|CONTROLS|CURRENCY|DELETING|DESCRIBE|DISTANCE|ENDCATCH|EXPONENT|EXTENDED|GENERATE|IGNORING|INCLUDES|INTERNAL|MAJOR-ID|MODIFIER|NEW-LINE|OPTIONAL|PROPERTY|ROLLBACK|STARTING|SUPPLIED|ABSTRACT|CHANGING|CONTEXTS|CREATING|CUSTOMER|DATABASE|DAYLIGHT|DEFINING|DISTINCT|DIVISION|ENABLING|ENDCHAIN|ESCAPING|HARMLESS|IMPLICIT|INACTIVE|LANGUAGE|MINOR-ID|MULTIPLY|NEW-PAGE|NO-TITLE|POS_HIGH|SEPARATE|TEXTPOOL|TRANSFER|SELECTOR|DBMAXLEN|ITERATOR|SELECTOR|ARCHIVE|BIT-XOR|BYTE-CO|COLLECT|COMMENT|CURRENT|DEFAULT|DISPLAY|ENDFORM|EXTRACT|LEADING|LISTBOX|LOCATOR|MEMBERS|METHODS|NESTING|POS_LOW|PROCESS|PROVIDE|RAISING|RESERVE|SECONDS|SUMMARY|VISIBLE|BETWEEN|BIT-AND|BYTE-CS|CLEANUP|COMPUTE|CONTROL|CONVERT|DATASET|ENDCASE|FORWARD|HEADERS|HOTSPOT|INCLUDE|INVERSE|KEEPING|NO-ZERO|OBJECTS|OVERLAY|PADDING|PATTERN|PROGRAM|REFRESH|SECTION|SUMMING|TESTING|VERSION|WINDOWS|WITHOUT|BIT-NOT|BYTE-CA|BYTE-NA|CASTING|CONTEXT|COUNTRY|DYNAMIC|ENABLED|ENDLOOP|EXECUTE|FRIENDS|HANDLER|HEADING|INITIAL|\*-INPUT|LOGFILE|MAXIMUM|MINIMUM|NO-GAPS|NO-SIGN|PRAGMAS|PRIMARY|PRIVATE|REDUCED|REPLACE|REQUEST|RESULTS|UNICODE|WARNING|ALIASES|BYTE-CN|BYTE-NS|CALLING|COL_KEY|COLUMNS|CONNECT|ENDEXEC|ENTRIES|EXCLUDE|FILTERS|FURTHER|HELP-ID|LOGICAL|MAPPING|MESSAGE|NAMETAB|OPTIONS|PACKAGE|PERFORM|RECEIVE|STATICS|VARYING|BINDING|CHARLEN|GREATER|XSTRLEN|ACCEPT|APPEND|DETAIL|ELSEIF|ENDING|ENDTRY|FORMAT|FRAMES|GIVING|HASHED|HEADER|IMPORT|INSERT|MARGIN|MODULE|NATIVE|OBJECT|OFFSET|REMOTE|RESUME|SAVING|SIMPLE|SUBMIT|TABBED|TOKENS|UNIQUE|UNPACK|UPDATE|WINDOW|YELLOW|ACTUAL|ASPECT|CENTER|CURSOR|DELETE|DIALOG|DIVIDE|DURING|ERRORS|EVENTS|EXTEND|FILTER|HANDLE|HAVING|IGNORE|LITTLE|MEMORY|NO-GAP|OCCURS|OPTION|PERSON|PLACES|PUBLIC|REDUCE|REPORT|RESULT|SINGLE|SORTED|SWITCH|SYNTAX|TARGET|VALUES|WRITER|ASSERT|BLOCKS|BOUNDS|BUFFER|CHANGE|COLUMN|COMMIT|CONCAT|COPIES|CREATE|DDMMYY|DEFINE|ENDIAN|ESCAPE|EXPAND|KERNEL|LAYOUT|LEGACY|LEVELS|MMDDYY|NUMBER|OUTPUT|RANGES|READER|RETURN|SCREEN|SEARCH|SELECT|SHARED|SOURCE|STABLE|STATIC|SUBKEY|SUFFIX|TABLES|UNWIND|YYMMDD|ASSIGN|BACKUP|BEFORE|BINARY|BIT-OR|BLANKS|CLIENT|CODING|COMMON|DEMAND|DYNPRO|EXCEPT|EXISTS|EXPORT|FIELDS|GLOBAL|GROUPS|LENGTH|LOCALE|MEDIUM|METHOD|MODIFY|NESTED|OTHERS|REJECT|SCROLL|SUPPLY|SYMBOL|ENDFOR|STRLEN|ALIGN|BEGIN|BOUND|ENDAT|ENTRY|EVENT|FINAL|FLUSH|GRANT|INNER|SHORT|USING|WRITE|AFTER|BLACK|BLOCK|CLOCK|COLOR|COUNT|DUMMY|EMPTY|ENDDO|ENDON|GREEN|INDEX|INOUT|LEAVE|LEVEL|LINES|MODIF|ORDER|OUTER|RANGE|RESET|RETRY|RIGHT|SMART|SPLIT|STYLE|TABLE|THROW|UNDER|UNTIL|UPPER|UTF-8|WHERE|ALIAS|BLANK|CLEAR|CLOSE|EXACT|FETCH|FIRST|FOUND|GROUP|LLANG|LOCAL|OTHER|REGEX|SPOOL|TITLE|TYPES|VALID|WHILE|ALPHA|BOXED|CATCH|CHAIN|CHECK|CLASS|COVER|ENDIF|EQUIV|FIELD|FLOOR|FRAME|INPUT|LOWER|MATCH|NODES|PAGES|PRINT|RAISE|ROUND|SHIFT|SPACE|SPOTS|STAMP|STATE|TASKS|TIMES|TRMAC|ULINE|UNION|VALUE|WIDTH|EQUAL|LOG10|TRUNC|BLOB|CASE|CEIL|CLOB|COND|EXIT|FILE|GAPS|HOLD|INCL|INTO|KEEP|KEYS|LAST|LINE|LONG|LPAD|MAIL|MODE|OPEN|PINK|READ|ROWS|TEST|THEN|ZERO|AREA|BACK|BADI|BYTE|CAST|EDIT|EXEC|FAIL|FIND|FKEQ|FONT|FREE|GKEQ|HIDE|INIT|ITNO|LATE|LOOP|MAIN|MARK|MOVE|NEXT|NULL|RISK|ROLE|UNIT|WAIT|ZONE|BASE|CALL|CODE|DATA|DATE|FKGE|GKGE|HIGH|KIND|LEFT|LIST|MASK|MESH|NAME|NODE|PACK|PAGE|POOL|SEND|SIGN|SIZE|SOME|STOP|TASK|TEXT|TIME|USER|VARY|WITH|WORD|BLUE|CONV|COPY|DEEP|ELSE|FORM|FROM|HINT|ICON|JOIN|LIKE|LOAD|ONLY|PART|SCAN|SKIP|SORT|TYPE|UNIX|VIEW|WHEN|WORK|ACOS|ASIN|ATAN|COSH|EACH|FRAC|LESS|RTTI|SINH|SQRT|TANH|AVG|BIT|DIV|ISO|LET|OUT|PAD|SQL|ALL|CI_|CPI|END|LOB|LPI|MAX|MIN|NEW|OLE|RUN|SET|\?TO|YES|ABS|ADD|AND|BIG|FOR|HDB|JOB|LOW|NOT|SAP|TRY|VIA|XML|ANY|GET|IDS|KEY|MOD|OFF|PUT|RAW|RED|REF|SUM|TAB|XSD|CNT|COS|EXP|LOG|SIN|TAN|XOR|AT|CO|CP|DO|GT|ID|IF|NS|OR|BT|CA|CS|GE|NA|NB|EQ|IN|LT|NE|NO|OF|ON|PF|TO|AS|BY|CN|IS|LE|NP|UP|E|I|M|O|Z|C|X'.split('|')
+        # return keyword
+        pass
+
+    @staticmethod
+    def comment_regex():
+        pattern = re.compile(r'(?P<comment>^\*.*?$|".*?$|\(\*[\s\S]*?\*\)|\(\*.*?$|^.*?\*\))|(?P<noncomment>\'(\\.|[^\\\'])*\'|"(\\.|[^\\"])*"|.[^*"\'"]*)', re.DOTALL | re.MULTILINE)
+        return pattern
+
+    @staticmethod
+    def number_regex():
+        pattern = re.compile(r'\b\d+\b')
+        return pattern
+
+    @staticmethod
+    def operator_regex():
+        pattern = re.compile(r'(\s)(?:\*\*?|<[=>]?|>=?|\?=|[-+\/=])(?=\s)')
+        return pattern
+
+    @staticmethod
+    def keywords_regex():
+        return re.compile(r'\b(' + '|'.join(Abap.keywords()) + r')\b')
+
+    @staticmethod
+    def remove_comments(source_code: str) -> str:
+        result = []
+        for match in Abap.comment_regex().finditer(source_code):
+            if match.group('noncomment'):
+                result.append(match.group('noncomment'))
+        # if isList:
+        #     return result
+        return ''.join(result).strip()
+
+    @staticmethod
+    def remove_keywords(source: str):
+        return re.sub(re.compile(Abap.keywords_regex()), '', source)