pycparser 2.20__tar.gz → 2.22__tar.gz

{pycparser-2.20 → pycparser-2.22}/CHANGES

@@ -1,3 +1,15 @@
++ Starting with version 2.22, please use the GitHub UI
+  at https://github.com/eliben/pycparser/tags
+  to compare tags in order to find out what changed.
+
++ Version 2.21 (2021.11.06)
+
+  - Much improved support for C11 (multiple PRs)
+  - Support for parehthesized compount statements (#423)
+  - Support for modern Python versions (3.9 and 3.10)
+  - Fix support for structs with nested enums (#387)
+  - Multiple small bug fixes
+
 + Version 2.20 (2020.03.04)
 
   - #61: Fix slow backtracking when parsing strings.

{pycparser-2.20 → pycparser-2.22}/LICENSE

@@ -1,6 +1,6 @@
 pycparser -- A C parser in Python
 
-Copyright (c) 2008-2017, Eli Bendersky
+Copyright (c) 2008-2022, Eli Bendersky
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without modification,
@@ -11,7 +11,7 @@ are permitted provided that the following conditions are met:
 * Redistributions in binary form must reproduce the above copyright notice, 
   this list of conditions and the following disclaimer in the documentation 
   and/or other materials provided with the distribution.
-* Neither the name of Eli Bendersky nor the names of its contributors may 
+* Neither the name of the copyright holder nor the names of its contributors may 
   be used to endorse or promote products derived from this software without 
   specific prior written permission.
 

pycparser-2.22/PKG-INFO

@@ -0,0 +1,28 @@
+Metadata-Version: 2.1
+Name: pycparser
+Version: 2.22
+Summary: C parser in Python
+Home-page: https://github.com/eliben/pycparser
+Author: Eli Bendersky
+Author-email: eliben@gmail.com
+Maintainer: Eli Bendersky
+License: BSD-3-Clause
+Platform: Cross Platform
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: License :: OSI Approved :: BSD License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+License-File: LICENSE
+
+
+        pycparser is a complete parser of the C language, written in
+        pure Python using the PLY parsing library.
+        It parses C code into an AST and can serve as a front-end for
+        C compilers or analysis tools.
+    
+

{pycparser-2.20 → pycparser-2.22}/README.rst

@@ -1,9 +1,13 @@
 ===============
-pycparser v2.20
+pycparser v2.22
 ===============
 
-:Author: `Eli Bendersky <https://eli.thegreenplace.net/>`_
 
+.. image:: https://github.com/eliben/pycparser/workflows/pycparser-tests/badge.svg
+  :align: center
+  :target: https://github.com/eliben/pycparser/actions
+
+----
 
 .. contents::
     :backlinks: none
@@ -79,9 +83,7 @@ Installing
 Prerequisites
 -------------
 
-* **pycparser** was tested on Python 2.7, 3.4-3.6, on both Linux and
-  Windows. It should work on any later version (in both the 2.x and 3.x lines)
-  as well.
+* **pycparser** was tested with Python 3.8+ on Linux, macOS and Windows.
 
 * **pycparser** has no external dependencies. The only non-stdlib library it
   uses is PLY, which is bundled in ``pycparser/ply``. The current PLY version is
@@ -97,28 +99,10 @@ though.
 Installation process
 --------------------
 
-Installing **pycparser** is very simple. Once you download and unzip the
-package, you just have to execute the standard ``python setup.py install``. The
-setup script will then place the ``pycparser`` module into ``site-packages`` in
-your Python's installation library.
-
-Alternatively, since **pycparser** is listed in the `Python Package Index
-<https://pypi.org/project/pycparser/>`_ (PyPI), you can install it using your
-favorite Python packaging/distribution tool, for example with::
+The recommended way to install **pycparser** is with ``pip``::
 
     > pip install pycparser
 
-Known problems
---------------
-
-* Some users who've installed a new version of **pycparser** over an existing
-  version ran into a problem using the newly installed library. This has to do
-  with parse tables staying around as ``.pyc`` files from the older version. If
-  you see unexplained errors from **pycparser** after an upgrade, remove it (by
-  deleting the ``pycparser`` directory in your Python's ``site-packages``, or
-  wherever you installed it) and install again.
-
-
 Using
 =====
 
@@ -147,10 +131,10 @@ What about the standard C library headers?
 C code almost always ``#include``\s various header files from the standard C
 library, like ``stdio.h``. While (with some effort) **pycparser** can be made to
 parse the standard headers from any C compiler, it's much simpler to use the
-provided "fake" standard  includes in ``utils/fake_libc_include``. These are
-standard C header files that contain only the bare necessities to allow valid
-parsing of the files that use them. As a bonus, since they're minimal, it can
-significantly improve the performance of parsing large C files.
+provided "fake" standard includes for C11 in ``utils/fake_libc_include``. These
+are standard C header files that contain only the bare necessities to allow
+valid parsing of the files that use them. As a bonus, since they're minimal, it
+can significantly improve the performance of parsing large C files.
 
 The key point to understand here is that **pycparser** doesn't really care about
 the semantics of types. It only needs to know whether some token encountered in
@@ -195,7 +179,8 @@ There are a few points to keep in mind when modifying **pycparser**:
 
 * The code for **pycparser**'s AST nodes is automatically generated from a
   configuration file - ``_c_ast.cfg``, by ``_ast_gen.py``. If you modify the AST
-  configuration, make sure to re-generate the code.
+  configuration, make sure to re-generate the code. This can be done by running
+  the ``_build_tables.py`` script from the ``pycparser`` directory.
 * Make sure you understand the optimized mode of **pycparser** - for that you
   must read the docstring in the constructor of the ``CParser`` class. For
   development you should create the parser without optimizations, so that it
@@ -228,6 +213,8 @@ tests/:
 
 utils/fake_libc_include:
   Minimal standard C library include files that should allow to parse any C code.
+  Note that these headers now include C11 code, so they may not work when the
+  preprocessor is configured to an earlier C standard (like ``-std=c99``).
 
 utils/internal/:
   Internal utilities for my own use. You probably don't need them.
@@ -243,18 +230,3 @@ updating this list because Github does a much better job at tracking
 contributions.
 
 
-CI Status
-=========
-
-**pycparser** has automatic testing enabled through the convenient
-`Travis CI project <https://travis-ci.org>`_. Here is the latest build status:
-
-.. image:: https://travis-ci.org/eliben/pycparser.png?branch=master
-  :align: center
-  :target: https://travis-ci.org/eliben/pycparser
-
-AppVeyor also helps run tests on Windows:
-
-.. image:: https://ci.appveyor.com/api/projects/status/wrup68o5y8nuk1i9?svg=true
-  :align: center
-  :target: https://ci.appveyor.com/project/eliben/pycparser/

{pycparser-2.20 → pycparser-2.22}/examples/c-to-c.py

@@ -7,15 +7,13 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #------------------------------------------------------------------------------
-from __future__ import print_function
 import sys
 
 # This is not required if you've installed pycparser into
 # your site-packages/ with setup.py
-#
 sys.path.extend(['.', '..'])
 
-from pycparser import parse_file, c_parser, c_generator
+from pycparser import parse_file, c_generator
 
 
 def translate_to_c(filename):
@@ -26,36 +24,7 @@ def translate_to_c(filename):
     print(generator.visit(ast))
 
 
-def _zz_test_translate():
-    # internal use
-    src = r'''
-
-    void f(char * restrict joe){}
-
-int main(void)
-{
-    unsigned int long k = 4;
-    int p = - - k;
-    return 0;
-}
-'''
-    parser = c_parser.CParser()
-    ast = parser.parse(src)
-    ast.show()
-    generator = c_generator.CGenerator()
-
-    print(generator.visit(ast))
-
-    # tracing the generator for debugging
-    #~ import trace
-    #~ tr = trace.Trace(countcallers=1)
-    #~ tr.runfunc(generator.visit, ast)
-    #~ tr.results().write_results()
-
-
-#------------------------------------------------------------------------------
 if __name__ == "__main__":
-    #_zz_test_translate()
     if len(sys.argv) > 1:
         translate_to_c(sys.argv[1])
     else:

pycparser-2.22/examples/c_files/basic.c

@@ -0,0 +1,6 @@
+int foo() {}
+
+int main() {
+  foo();
+  return 0;
+}

{pycparser-2.20 → pycparser-2.22}/examples/c_files/memmgr.h

@@ -49,7 +49,7 @@
 //    minimize pool fragmentation in case of multiple allocations
 //    and deallocations, it is advisable to not allocate
 //    blocks that are too small.
-//    This flag sets the minimal ammount of quantas for
+//    This flag sets the minimal amount of quantas for
 //    an allocation. If the size of a ulong is 4 and you
 //    set this flag to 16, the minimal size of an allocation
 //    will be 4 * 2 * 16 = 128 bytes

pycparser-2.22/examples/c_files/pragmas.c

@@ -0,0 +1,15 @@
+int i;
+
+#pragma joe
+
+int main() {
+  int a = 1;
+#pragma inmain
+
+  for (int i = 0; i < 3; i++) {
+#pragma infor
+    a += i;
+  }
+
+  return a;
+}

{pycparser-2.20 → pycparser-2.22}/examples/c_files/year.c

@@ -7,8 +7,8 @@ void convert(int thousands, int hundreds, int tens, int ones)
 char *num[] = {"", "One", "Two", "Three", "Four", "Five", "Six",
 	       "Seven", "Eight", "Nine"};
 
-char *for_ten[] = {"", "", "Twenty", "Thirty", "Fourty", "Fifty", "Sixty",
-		   "Seventy", "Eighty", "Ninty"};
+char *for_ten[] = {"", "", "Twenty", "Thirty", "Forty", "Fifty", "Sixty",
+		   "Seventy", "Eighty", "Ninety"};
 
 char *af_ten[] = {"Ten", "Eleven", "Twelve", "Thirteen", "Fourteen",
 		  "Fifteen", "Sixteen", "Seventeen", "Eighteen", "Ninteen"};

{pycparser-2.20 → pycparser-2.22}/examples/c_json.py

@@ -34,8 +34,6 @@
 #         }
 #     }
 #------------------------------------------------------------------------------
-from __future__ import print_function
-
 import json
 import sys
 import re

pycparser-2.22/examples/construct_ast_from_scratch.py

@@ -0,0 +1,53 @@
+# -----------------------------------------------------------------
+# pycparser: construct_ast_from_scratch.py
+#
+# Tiny example of writing an AST from scratch to C code.
+#
+# Andre Ribeiro [https://github.com/Andree37]
+# License: BSD
+# -----------------------------------------------------------------
+import sys
+
+# This is not required if you've installed pycparser into
+# your site-packages/ with setup.py
+sys.path.extend(['.', '..'])
+
+from pycparser import c_ast, c_generator
+
+
+# target C code:
+# int main() {
+#     return 0;
+# }
+
+
+def empty_main_function_ast():
+    constant_zero = c_ast.Constant(type='int', value='0')
+    return_node = c_ast.Return(expr=constant_zero)
+    compound_node = c_ast.Compound(block_items=[return_node])
+    type_decl_node = c_ast.TypeDecl(declname='main', quals=[],
+                                    type=c_ast.IdentifierType(names=['int']),
+                                    align=[])
+    func_decl_node = c_ast.FuncDecl(args=c_ast.ParamList([]),
+                                    type=type_decl_node)
+    func_def_node = c_ast.Decl(name='main', quals=[], storage=[], funcspec=[],
+                               type=func_decl_node, init=None,
+                               bitsize=None, align=[])
+    main_func_node = c_ast.FuncDef(decl=func_def_node, param_decls=None,
+                                   body=compound_node)
+
+    return main_func_node
+
+
+def generate_c_code(my_ast):
+    generator = c_generator.CGenerator()
+    return generator.visit(my_ast)
+
+
+if __name__ == '__main__':
+    main_function_ast = empty_main_function_ast()
+    print("|----------------------------------------|")
+    main_function_ast.show(offset=2)
+    print("|----------------------------------------|")
+    main_c_code = generate_c_code(main_function_ast)
+    print("C code: \n%s" % main_c_code)

{pycparser-2.20 → pycparser-2.22}/examples/dump_ast.py

@@ -6,7 +6,6 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-from __future__ import print_function
 import argparse
 import sys
 
@@ -14,11 +13,14 @@ import sys
 # your site-packages/ with setup.py
 sys.path.extend(['.', '..'])
 
-from pycparser import c_parser, c_ast, parse_file
+from pycparser import parse_file
 
 if __name__ == "__main__":
     argparser = argparse.ArgumentParser('Dump AST')
-    argparser.add_argument('filename', help='name of file to parse')
+    argparser.add_argument('filename',
+                           default='examples/c_files/basic.c',
+                           nargs='?',
+                           help='name of file to parse')
     argparser.add_argument('--coord', help='show coordinates in the dump',
                            action='store_true')
     args = argparser.parse_args()

{pycparser-2.20 → pycparser-2.22}/examples/explore_ast.py

@@ -12,7 +12,6 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-from __future__ import print_function
 import sys
 
 # This is not required if you've installed pycparser into
@@ -20,7 +19,7 @@ import sys
 #
 sys.path.extend(['.', '..'])
 
-from pycparser import c_parser, c_ast
+from pycparser import c_parser
 
 # This is some C source to parse. Note that pycparser must begin
 # at the top level of the C file, i.e. with either declarations

{pycparser-2.20 → pycparser-2.22}/examples/func_calls.py

@@ -7,15 +7,13 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-from __future__ import print_function
 import sys
 
 # This is not required if you've installed pycparser into
 # your site-packages/ with setup.py
 sys.path.extend(['.', '..'])
 
-from pycparser import c_parser, c_ast, parse_file
-
+from pycparser import c_ast, parse_file
 
 # A visitor with some state information (the funcname it's looking for)
 class FuncCallVisitor(c_ast.NodeVisitor):
@@ -41,7 +39,7 @@ if __name__ == "__main__":
         filename = sys.argv[1]
         func = sys.argv[2]
     else:
-        filename = 'examples/c_files/hash.c'
-        func = 'malloc'
+        filename = 'examples/c_files/basic.c'
+        func = 'foo'
 
     show_func_calls(filename, func)

{pycparser-2.20 → pycparser-2.22}/examples/func_defs.py

@@ -10,14 +10,13 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-from __future__ import print_function
 import sys
 
 # This is not required if you've installed pycparser into
 # your site-packages/ with setup.py
 sys.path.extend(['.', '..'])
 
-from pycparser import c_parser, c_ast, parse_file
+from pycparser import c_ast, parse_file
 
 
 # A simple visitor for FuncDef nodes that prints the names and

pycparser-2.22/examples/func_defs_add_param.py

@@ -0,0 +1,60 @@
+#-----------------------------------------------------------------
+# pycparser: func_defs_add_param.py
+#
+# Example of rewriting AST nodes to add parameters to function
+# definitions. Adds an "int _hidden" to every function.
+#
+# Eli Bendersky [https://eli.thegreenplace.net/]
+# License: BSD
+#-----------------------------------------------------------------
+import sys
+sys.path.extend(['.', '..'])
+
+from pycparser import c_parser, c_ast, c_generator
+
+text = r"""
+void foo(int a, int b) {
+}
+
+void bar() {
+}
+"""
+
+
+class ParamAdder(c_ast.NodeVisitor):
+    def visit_FuncDecl(self, node):
+        ty = c_ast.TypeDecl(declname='_hidden',
+                            quals=[],
+                            align=[],
+                            type=c_ast.IdentifierType(['int']))
+        newdecl = c_ast.Decl(
+                    name='_hidden',
+                    quals=[],
+                    align=[],
+                    storage=[],
+                    funcspec=[],
+                    type=ty,
+                    init=None,
+                    bitsize=None,
+                    coord=node.coord)
+        if node.args:
+            node.args.params.append(newdecl)
+        else:
+            node.args = c_ast.ParamList(params=[newdecl])
+
+
+if __name__ == '__main__':
+    parser = c_parser.CParser()
+    ast = parser.parse(text)
+    print("AST before change:")
+    ast.show(offset=2)
+
+    v = ParamAdder()
+    v.visit(ast)
+
+    print("\nAST after change:")
+    ast.show(offset=2)
+
+    print("\nCode after change:")
+    generator = c_generator.CGenerator()
+    print(generator.visit(ast))

{pycparser-2.20 → pycparser-2.22}/examples/rewrite_ast.py

@@ -6,9 +6,9 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-from __future__ import print_function
 import sys
 
+sys.path.extend(['.', '..'])
 from pycparser import c_parser
 
 text = r"""
@@ -18,14 +18,15 @@ void func(void)
 }
 """
 
-parser = c_parser.CParser()
-ast = parser.parse(text)
-print("Before:")
-ast.show(offset=2)
+if __name__ == '__main__':
+    parser = c_parser.CParser()
+    ast = parser.parse(text)
+    print("Before:")
+    ast.show(offset=2)
 
-assign = ast.ext[0].body.block_items[0]
-assign.lvalue.name = "y"
-assign.rvalue.value = 2
+    assign = ast.ext[0].body.block_items[0]
+    assign.lvalue.name = "y"
+    assign.rvalue.value = 2
 
-print("After:")
-ast.show(offset=2)
+    print("After:")
+    ast.show(offset=2)

pycparser-2.22/examples/serialize_ast.py

@@ -0,0 +1,34 @@
+#-----------------------------------------------------------------
+# pycparser: serialize_ast.py
+#
+# Simple example of serializing AST
+#
+# Hart Chu [https://github.com/CtheSky]
+# Eli Bendersky [https://eli.thegreenplace.net/]
+# License: BSD
+#-----------------------------------------------------------------
+import pickle
+import sys
+
+sys.path.extend(['.', '..'])
+from pycparser import c_parser
+
+text = r"""
+void func(void)
+{
+  x = 1;
+}
+"""
+
+if __name__ == '__main__':
+    parser = c_parser.CParser()
+    ast = parser.parse(text)
+    dump_filename = 'ast.pickle'
+
+    with open(dump_filename, 'wb') as f:
+        pickle.dump(ast, f, protocol=pickle.HIGHEST_PROTOCOL)
+
+    # Deserialize.
+    with open(dump_filename, 'rb') as f:
+        ast = pickle.load(f)
+        ast.show()

{pycparser-2.20 → pycparser-2.22}/pycparser/__init__.py

@@ -8,7 +8,7 @@
 # License: BSD
 #-----------------------------------------------------------------
 __all__ = ['c_lexer', 'c_parser', 'c_ast']
-__version__ = '2.20'
+__version__ = '2.22'
 
 import io
 from subprocess import check_output
@@ -49,7 +49,7 @@ def preprocess_file(filename, cpp_path='cpp', cpp_args=''):
 
 
 def parse_file(filename, use_cpp=False, cpp_path='cpp', cpp_args='',
-               parser=None):
+               parser=None, encoding=None):
     """ Parse a C file using pycparser.
 
         filename:
@@ -71,6 +71,9 @@ def parse_file(filename, use_cpp=False, cpp_path='cpp', cpp_args='',
             r'-I../utils/fake_libc_include'
             If several arguments are required, pass a list of strings.
 
+        encoding:
+            Encoding to use for the file to parse
+
         parser:
             Optional parser object to be used instead of the default CParser
 
@@ -82,7 +85,7 @@ def parse_file(filename, use_cpp=False, cpp_path='cpp', cpp_args='',
     if use_cpp:
         text = preprocess_file(filename, cpp_path, cpp_args)
     else:
-        with io.open(filename) as f:
+        with io.open(filename, encoding=encoding) as f:
             text = f.read()
 
     if parser is None:

{pycparser-2.20 → pycparser-2.22}/pycparser/_ast_gen.py

@@ -10,7 +10,6 @@
 # Eli Bendersky [https://eli.thegreenplace.net/]
 # License: BSD
 #-----------------------------------------------------------------
-import pprint
 from string import Template
 
 
@@ -86,7 +85,6 @@ class NodeCfg(object):
         src = self._gen_init()
         src += '\n' + self._gen_children()
         src += '\n' + self._gen_iter()
-
         src += '\n' + self._gen_attr_names()
         return src
 
@@ -196,7 +194,7 @@ def _repr(obj):
     if isinstance(obj, list):
         return '[' + (',\n '.join((_repr(e).replace('\n', '\n ') for e in obj))) + '\n]'
     else:
-        return repr(obj) 
+        return repr(obj)
 
 class Node(object):
     __slots__ = ()
@@ -206,19 +204,19 @@ class Node(object):
         """ Generates a python representation of the current node
         """
         result = self.__class__.__name__ + '('
-        
+
         indent = ''
         separator = ''
         for name in self.__slots__[:-2]:
             result += separator
             result += indent
             result += name + '=' + (_repr(getattr(self, name)).replace('\n', '\n  ' + (' ' * (len(name) + len(self.__class__.__name__)))))
-            
+
             separator = ','
             indent = '\n ' + (' ' * len(self.__class__.__name__))
-        
+
         result += indent + ')'
-        
+
         return result
 
     def children(self):

{pycparser-2.20 → pycparser-2.22}/pycparser/_build_tables.py

@@ -13,6 +13,7 @@
 # Insert '.' and '..' as first entries to the search path for modules.
 # Restricted environments like embeddable python do not include the
 # current working directory on startup.
+import importlib
 import sys
 sys.path[0:0] = ['.', '..']
 
@@ -32,6 +33,8 @@ c_parser.CParser(
 
 # Load to compile into .pyc
 #
+importlib.invalidate_caches()
+
 import lextab
 import yacctab
 import c_ast