llparse 0.1.0__tar.gz

llparse-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Vizonex
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

llparse-0.1.0/PKG-INFO

@@ -0,0 +1,129 @@
+Metadata-Version: 2.4
+Name: llparse
+Version: 0.1.0
+Summary: A Parody of llparse written for writing C Parsers with Python
+Author-email: Vizonex <VizonexBusiness@gmail.com>
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# pyllparse
+A python parody of the typescript library llparse.
+
+I take no credit for the orginal work done by indutny and I was originally very nervous about making 
+this python library that I made public... 
+
+Links to the original library 
+- https://llparse.org
+- https://github.com/nodejs/llparse
+
+Unlike the typescript library all 3 of llparse's libraries were combined in this version of the code for the sake of portability... 
+I ended up mentioning how I did this a while back and I also had a concept for a C parser as well but I just didn't like it and I ended up using this instead but I also learned typescript as a bonus and It was alot of fun for me. I don't plan to make this into a real pypi library yet but I also didn't want to take away from the magic of the original source code that I borrowed from...
+
+# Looking back on my dead work 2 years later
+I had this idea lay somewhat dormant for 2 years and now seeing that I wanted to write a new socks5 server parser writing it in python
+just made a little bit more sense to me, this way I don't have to stress over typescript or needing to relearn things I learned over 2 years ago. 
+I have now made this project public on pypi for anybody who wanted to dig into it and access the code
+or generate your own parsers for C.
+
+If your better at typescript. I reccommend sticking to the typescript version, no pressure. This library aims to not compete 
+with llparse but to act as an alternative for users who prefer python over typescript/node-js, simillar to puppeteer or pyppeteer. 
+Personally maybe we should make a rust crate for a rust version or at least someone will do it. 
+
+
+## Warnings
+- It should be safe enough to generate code and stress-test it but if something doesn't seem right to you, be sure to throw an issue and share a typescript version of something that works in typescript but not in python, this way something unintended by the typescript developers can be solved.
+- Some code may look unfinished or may need further internal refactoring/polishing and with the other things I want to go out and do
+such as contributions to aiohttp and it's other libraries, cyares, winloop, aiothreading, aiocallback & deprecated-params and I also have an irl Part-time Job, ideas and pull requests are welcomed without hesitation. I maintain other projects listed above to prevent myself from experiencing burnout.
+- Pytest testsuite has not been made yet. If this is a concern to you, please throw an issue on github. (Pytests and new workflows soon!)
+- Some code might be spaghetti-code (because It's been 2 years since I touched much of the code if not at all) 
+e.g. tests module should be moved away from the library into the github repo.
+- There's some research portions such as tools to help generate things for cython or tools to make mirrors of llhttp's code
+such as it's native c code (I was looking to see if it could be theoretically autogenerated). Feel free to use them but
+do it at your own risk. They maybe incompleted or not throughly stress-tested.
+
+## New Features
+- Throw me an issue if typescript llparse introduces something new that you want for me or another contributor to try and implement 
+just seeing llparse add new features is nothing but exciting to me.
+
+- If you want a feature that typescript llparse doesn't have, be sure to try making a pull request over there as well and not just here, 
+there's a good chance they will appericate you for helping over there too and your helping make llhttp better by doing so. :)
+
+
+# Why Did I Translate llparse to python?
+- I wanted to work with a langauge I was more familiar with
+- Better educate myself and others on how these great libraries like llhttp are made
+- Write faster C code that could do more than just a simple split function or a regex...
+- Make it easy for me or someone else to find a problem and solve it in typescript after testing it in python
+- Typescript takes 2 commands to run a script with node-js it while python only takes one cutting the time required tremendously...
+- The orginal project was MIT licensed.
+- I wanted to write my own C Parsering tool with llhttp styled callbacks of my own using a language I was the most comfortable with using.
+- I didn't like __Lemon Parser__ or __Yacc__ all that much and a good ide for handling them in Visual Studio Code with error checking to my knowlegde does not exist.
+- The closest thing I got to what I wanted was a project named __NMFU__ shorthand for no memory for you and even I had problems with writing things using that library...
+
+This was the Code that inspired me to try and make a new pyi writer branch for cython and if it wasn't for llhttp 
+existing as well as it's magical experience I would've never done what I did.
+
+Unlike llparse in typescript, this version has more integrated and experimental features like automatically building the api seen in llhttp and 
+I've added a few other things like the dot compiler from llparse_dot and I also made a brand new cython compiler 
+for it making easy and simple to make pxd files to port your projects to cython 
+
+
+
+## How to use
+```python
+# The good old http_parser was borrowed from llparse.org to demonstrate this for you :)
+from llparse import LLParse
+
+p = LLParse("http_parser")
+method = p.node("method")
+beforeUrl = p.node("before_url")
+urlSpan = p.span(p.code.span("on_url"))
+url = p.node("url")
+http = p.node("http")
+
+# Add custom uint8_t property to the state
+p.property("i8", "method")
+
+# Store method inside a custom property
+onMethod = p.invoke(p.code.store("method"), beforeUrl)
+
+# Invoke custom C function
+complete = p.invoke(
+    p.code.match("on_complete"),
+    {
+        #  Restart
+        0: method
+    },
+    p.error(4, "`on_complete` error"),
+)
+
+method.select(
+    {
+        "HEAD": 0,
+        "GET": 1,
+        "POST": 2,
+        "PUT": 3,
+        "DELETE": 4,
+        "OPTIONS": 5,
+        "CONNECT": 6,
+        "TRACE": 7,
+        "PATCH": 8,
+    },
+    onMethod,
+).otherwise(p.error(5, "Expected method"))
+
+beforeUrl.match(" ", beforeUrl).otherwise(urlSpan.start(url))
+
+url.peek(" ", urlSpan.end(http)).skipTo(url)
+
+http.match(" HTTP/1.1\r\n\r\n", complete).otherwise(
+    p.error(6, "Expected HTTP/1.1 and two newlines")
+)
+
+c = p.build(method)
+print(c.c)
+open("http_parser.c", "w").write(c.c)
+open("http_parser.h", "w").write(c.header)
+```

llparse-0.1.0/README.md

@@ -0,0 +1,119 @@
+# pyllparse
+A python parody of the typescript library llparse.
+
+I take no credit for the orginal work done by indutny and I was originally very nervous about making 
+this python library that I made public... 
+
+Links to the original library 
+- https://llparse.org
+- https://github.com/nodejs/llparse
+
+Unlike the typescript library all 3 of llparse's libraries were combined in this version of the code for the sake of portability... 
+I ended up mentioning how I did this a while back and I also had a concept for a C parser as well but I just didn't like it and I ended up using this instead but I also learned typescript as a bonus and It was alot of fun for me. I don't plan to make this into a real pypi library yet but I also didn't want to take away from the magic of the original source code that I borrowed from...
+
+# Looking back on my dead work 2 years later
+I had this idea lay somewhat dormant for 2 years and now seeing that I wanted to write a new socks5 server parser writing it in python
+just made a little bit more sense to me, this way I don't have to stress over typescript or needing to relearn things I learned over 2 years ago. 
+I have now made this project public on pypi for anybody who wanted to dig into it and access the code
+or generate your own parsers for C.
+
+If your better at typescript. I reccommend sticking to the typescript version, no pressure. This library aims to not compete 
+with llparse but to act as an alternative for users who prefer python over typescript/node-js, simillar to puppeteer or pyppeteer. 
+Personally maybe we should make a rust crate for a rust version or at least someone will do it. 
+
+
+## Warnings
+- It should be safe enough to generate code and stress-test it but if something doesn't seem right to you, be sure to throw an issue and share a typescript version of something that works in typescript but not in python, this way something unintended by the typescript developers can be solved.
+- Some code may look unfinished or may need further internal refactoring/polishing and with the other things I want to go out and do
+such as contributions to aiohttp and it's other libraries, cyares, winloop, aiothreading, aiocallback & deprecated-params and I also have an irl Part-time Job, ideas and pull requests are welcomed without hesitation. I maintain other projects listed above to prevent myself from experiencing burnout.
+- Pytest testsuite has not been made yet. If this is a concern to you, please throw an issue on github. (Pytests and new workflows soon!)
+- Some code might be spaghetti-code (because It's been 2 years since I touched much of the code if not at all) 
+e.g. tests module should be moved away from the library into the github repo.
+- There's some research portions such as tools to help generate things for cython or tools to make mirrors of llhttp's code
+such as it's native c code (I was looking to see if it could be theoretically autogenerated). Feel free to use them but
+do it at your own risk. They maybe incompleted or not throughly stress-tested.
+
+## New Features
+- Throw me an issue if typescript llparse introduces something new that you want for me or another contributor to try and implement 
+just seeing llparse add new features is nothing but exciting to me.
+
+- If you want a feature that typescript llparse doesn't have, be sure to try making a pull request over there as well and not just here, 
+there's a good chance they will appericate you for helping over there too and your helping make llhttp better by doing so. :)
+
+
+# Why Did I Translate llparse to python?
+- I wanted to work with a langauge I was more familiar with
+- Better educate myself and others on how these great libraries like llhttp are made
+- Write faster C code that could do more than just a simple split function or a regex...
+- Make it easy for me or someone else to find a problem and solve it in typescript after testing it in python
+- Typescript takes 2 commands to run a script with node-js it while python only takes one cutting the time required tremendously...
+- The orginal project was MIT licensed.
+- I wanted to write my own C Parsering tool with llhttp styled callbacks of my own using a language I was the most comfortable with using.
+- I didn't like __Lemon Parser__ or __Yacc__ all that much and a good ide for handling them in Visual Studio Code with error checking to my knowlegde does not exist.
+- The closest thing I got to what I wanted was a project named __NMFU__ shorthand for no memory for you and even I had problems with writing things using that library...
+
+This was the Code that inspired me to try and make a new pyi writer branch for cython and if it wasn't for llhttp 
+existing as well as it's magical experience I would've never done what I did.
+
+Unlike llparse in typescript, this version has more integrated and experimental features like automatically building the api seen in llhttp and 
+I've added a few other things like the dot compiler from llparse_dot and I also made a brand new cython compiler 
+for it making easy and simple to make pxd files to port your projects to cython 
+
+
+
+## How to use
+```python
+# The good old http_parser was borrowed from llparse.org to demonstrate this for you :)
+from llparse import LLParse
+
+p = LLParse("http_parser")
+method = p.node("method")
+beforeUrl = p.node("before_url")
+urlSpan = p.span(p.code.span("on_url"))
+url = p.node("url")
+http = p.node("http")
+
+# Add custom uint8_t property to the state
+p.property("i8", "method")
+
+# Store method inside a custom property
+onMethod = p.invoke(p.code.store("method"), beforeUrl)
+
+# Invoke custom C function
+complete = p.invoke(
+    p.code.match("on_complete"),
+    {
+        #  Restart
+        0: method
+    },
+    p.error(4, "`on_complete` error"),
+)
+
+method.select(
+    {
+        "HEAD": 0,
+        "GET": 1,
+        "POST": 2,
+        "PUT": 3,
+        "DELETE": 4,
+        "OPTIONS": 5,
+        "CONNECT": 6,
+        "TRACE": 7,
+        "PATCH": 8,
+    },
+    onMethod,
+).otherwise(p.error(5, "Expected method"))
+
+beforeUrl.match(" ", beforeUrl).otherwise(urlSpan.start(url))
+
+url.peek(" ", urlSpan.end(http)).skipTo(url)
+
+http.match(" HTTP/1.1\r\n\r\n", complete).otherwise(
+    p.error(6, "Expected HTTP/1.1 and two newlines")
+)
+
+c = p.build(method)
+print(c.c)
+open("http_parser.c", "w").write(c.c)
+open("http_parser.h", "w").write(c.header)
+```

llparse-0.1.0/llparse/C_compiler.py

@@ -0,0 +1,204 @@
+from typing import Optional
+
+from .compilator import Compilation, ICompilerOptions, Node
+from .constants import *
+from .frontend import IFrontendResult
+
+
+class CCompiler:
+    """The Final HeadPeice where the Main C-Code gets compiled to..."""
+
+    def __init__(
+        self, header: Optional[str] = None, debug: Optional[str] = None
+    ) -> None:
+        # NOTE Unlike in typescript llparse Containers are not Required since I'm using a different methoad to translate those parts...
+        self.options = ICompilerOptions(debug, header)
+
+    def compile(self, info: IFrontendResult):
+        compilation = Compilation(
+            info.prefix,
+            info.properties,
+            list(info.resumptionTargets),
+            options=self.options,
+        )
+
+        out: list[str] = []
+
+        out.append("#include <stdlib.h>")
+        out.append("#include <stdint.h>")
+        out.append("#include <string.h>")
+        out.append("")
+        # Seems LLParse was updated from /* UNREACHABLE */ abort(); to a Macro, Intresting...
+        out.append('#ifdef __SSE4_2__')
+        out.append(' #ifdef _MSC_VER')
+        out.append('  #include <nmmintrin.h>')
+        out.append(' #else  /* !_MSC_VER */')
+        out.append('  #include <x86intrin.h>')
+        out.append(' #endif  /* _MSC_VER */')
+        out.append('#endif  /* __SSE4_2__ */')
+        out.append('')
+
+        out.append('#ifdef __ARM_NEON__')
+        out.append(' #include <arm_neon.h>')
+        out.append('#endif  /* __ARM_NEON__ */')
+        out.append('')
+
+        out.append('#ifdef __wasm__')
+        out.append(' #include <wasm_simd128.h>')
+        out.append('#endif  /* __wasm__ */')
+        out.append('')
+
+        out.append('#ifdef _MSC_VER')
+        out.append(' #define ALIGN(n) _declspec(align(n))')
+        out.append(' #define UNREACHABLE __assume(0)')
+        out.append('#else  /* !_MSC_VER */')
+        out.append(' #define ALIGN(n) __attribute__((aligned(n)))')
+        out.append(' #define UNREACHABLE __builtin_unreachable()')
+        out.append('#endif  /* _MSC_VER */')
+
+        out.append("")
+        out.append(
+            f'#include "{self.options.header if self.options.header else info.prefix}.h"'
+        )
+        out.append("")
+        out.append(f"typedef int (*{info.prefix}__span_cb)(")
+        out.append(f"             {info.prefix}_t*, const char*, const char*);")
+        out.append("")
+
+        # Start Queuing span callbacks
+        # otherwise we will have nothing
+        # but mess which is not what we want - Vizonex
+        compilation.reserveSpans(info.spans)
+
+        rootState: Node = compilation.unwrapNode(info.root)
+        rootName = rootState.build(compilation)
+        # Bring in the rest of the variables...
+        compilation.buildGlobals(out)
+        out.append("")
+
+        out.append(f"int {info.prefix}_init({info.prefix}_t* {ARG_STATE}) " + "{")
+        out.append(f"  memset({ARG_STATE}, 0, sizeof(*{ARG_STATE}));")
+        out.append(f"  {ARG_STATE}->_current = (void*) (intptr_t) {rootName};")
+        out.append("  return 0;")
+        out.append("}")
+        out.append("")
+
+        # TODO (Vizonex) Make llparse_state_t's Name Optional and alterable incase mixed with 
+        # llhttp or another parser
+        out.append(f"static llparse_state_t {info.prefix}__run(")
+        out.append(f"    {info.prefix}_t* {ARG_STATE},")
+        out.append(f"    const unsigned char* {ARG_POS},")
+        out.append(f"    const unsigned char* {ARG_ENDPOS}) " + "{")
+        out.append(f"  int {VAR_MATCH};")
+        out.append(
+            "  switch ((llparse_state_t) (intptr_t) "
+            + f"{compilation.currentField()}) "
+            + "{"
+        )
+
+        # Now build resumption states... These are states what will have a 'case block' next to them...
+        # However I'm not refering to the characters those will be handles in thier inner switches,
+        # I'm talking about the major states...
+        tmp = []
+        compilation.buildResumptionStates(tmp)
+        compilation.indent(out, tmp, "    ")
+
+        # Final Resumption State... Very important!
+        out.append("    default:")
+        out.append("      UNREACHABLE;")
+        out.append("  }")
+
+        tmp = []
+        compilation.buildInternalStates(tmp)
+        compilation.indent(out, tmp, "  ")
+
+        out.append("}")
+        out.append("")
+
+        out.append(
+            f"int {info.prefix}_execute({info.prefix}_t* {ARG_STATE}, "
+            + f"const char* {ARG_POS}, const char* {ARG_ENDPOS}) "
+            + "{"
+        )
+        out.append("  llparse_state_t next;")
+        out.append("")
+
+        out.append("  /* check lingering errors */")
+        out.append(f"  if ({compilation.errorField()} != 0) " + "{")
+        out.append(f"    return {compilation.errorField()};")
+        out.append("  }")
+        out.append("")
+
+        tmp = []
+        self.restartSpans(compilation, info, tmp)
+        compilation.indent(out, tmp, "  ")
+        args = [
+            compilation.stateArg(),
+            f"(const unsigned char*) {compilation.posArg()}",
+            f"(const unsigned char*) {compilation.endPosArg()}",
+        ]
+        out.append(f"  next = {info.prefix}__run({(', ').join(args)});")
+        out.append(f"  if (next == {STATE_ERROR}) " + "{")
+        out.append(f"    return {compilation.errorField()};")
+        out.append("  }")
+        out.append(f"  {compilation.currentField()} = (void*) (intptr_t) next;")
+        out.append("")
+
+        tmp = []
+        self.executeSpans(compilation, info, tmp)
+        compilation.indent(out, tmp, "  ")
+
+        out.append("  return 0;")
+        out.append("}")
+
+        # JOIN ALL OF THEM!
+        return "\n".join(out)
+
+    def restartSpans(self, ctx: Compilation, info: IFrontendResult, out: list[str]):
+        if len(info.spans) == 0:
+            return
+
+        out.append("/* restart spans */")
+        for span in info.spans:
+            posField = ctx.spanPosField(span.index)
+
+            out.append(f"if ({posField} != NULL) " + "{")
+            out.append(f"  {posField} = (void*) {ctx.posArg()};")
+            out.append("}")
+        out.append("")
+
+    def executeSpans(self, ctx: Compilation, info: IFrontendResult, out: list[str]):
+        if not info.spans:
+            return
+
+        out.append("/* execute spans */")
+        for span in info.spans:
+            posField = ctx.spanPosField(span.index)
+
+            if len(span.callbacks) == 1:
+                cb = ctx.unwrapCode(span.callbacks[0], True)
+                callback = ctx.buildCode(cb)
+
+            else:
+                # TODO (Vizonex) Merge lines 139 & 140 together in a future update
+                callback = (
+                    f"({info.prefix}__span_cb)"
+                    + ctx.spanCbField(span.index)
+                    + f"({callback})"
+                )
+
+            args = [ctx.stateArg(), posField, f"(const char*) {ctx.endPosArg()}"]
+
+            out.append(f"if ({posField} != NULL) " + "{")
+            out.append("  int error;")
+            out.append("")
+            out.append(f"  error = {callback}({', '.join(args)});")
+
+            # TODO (Vizonex): Deduplicate when indutny updates his side so we can all make our changes accordingly...
+            out.append("  if (error != 0) {")
+            out.append(f"    {ctx.errorField()} = error;")
+            out.append(f"    {ctx.errorPosField()} = {ctx.endPosArg()};")
+            out.append("    return error;")
+            out.append("  }")
+            out.append("}")
+        out.append("")

llparse-0.1.0/llparse/__init__.py

@@ -0,0 +1,2 @@
+from .dot import Dot
+from .llparse import LLParse