puppet-lint 2.3.6 → 2.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 299ce7a01574b87c32ac76798f691800ef27fc2c
-  data.tar.gz: 4f3f73a44040dbfc0d6f3b2bfb6535b000d3bdf1
+SHA256:
+  metadata.gz: b67dfbcf1aff9be2449b52eb8ff66db18cf025ad69ce6dee4f3e8a7ae4155980
+  data.tar.gz: 2eec8d5d37d1deeaaf9e26ca6ad24e7f841e137bb9de99e61736675862f6bffc
 SHA512:
-  metadata.gz: 920ee6f31d30184faf6c288902f53387bba45df38976a854c7e837647520e8162e140303fad43a6b7de70be4c1b3efd1cede9fc2f93b377e0773442f66fe239d
-  data.tar.gz: ab9bb35509a161c70c73b4880b737e39416884d6709c166df95817b3fd6f6efce5329ca856f50644705907c776b631c61f658ecfa2bae1c7972e79310fc45cb4
+  metadata.gz: 80e81796c7aa00827859e45f6586105bbeb1c16c5a813e3130e8ff1fa11272c012dd86563d21172862a00852f49c0b0df2015881c83e15b62e208e117c85e938
+  data.tar.gz: f21481a188185489015c3fa7dff8bd3d00e5eec505a547abc4c7633805c41e275f311eec37f5c4a93899ea77fb43a530ba25588fdbb0f78dc043f904d6716c1b

data/.rubocop_todo.yml

@@ -45,7 +45,7 @@ Metrics/BlockNesting:
 # Offense count: 2
 # Configuration parameters: CountComments.
 Metrics/ClassLength:
-  Max: 383
+  Max: 384
 
 # Offense count: 20
 Metrics/CyclomaticComplexity:
@@ -64,7 +64,7 @@ Metrics/MethodLength:
 
 # Offense count: 18
 Metrics/PerceivedComplexity:
-  Max: 25
+  Max: 28
 
 # Offense count: 1
 # Cop supports --auto-correct.

data/.travis.yml

@@ -1,7 +1,7 @@
 ---
 language: ruby
+dist: trusty
 sudo: false
-before_install: gem update --system && gem install bundler
 branches:
   only:
     - master
@@ -15,10 +15,12 @@ rvm:
   - 1.9.3
   - 2.0.0
   - 2.1.10
-  - 2.2.7
-  - 2.3.4
-  - 2.4.1
+  - 2.2.10
+  - 2.3.8
+  - 2.4.7
+  - 2.5.6
+  - 2.6.4
 matrix:
     include:
-        - rvm: 2.4.1
+        - rvm: 2.6.4
           env: CHECK=rubocop

data/CHANGELOG.md

@@ -1,5 +1,58 @@
 # Change Log
 
+## [2.4.0](https://github.com/rodjek/puppet-lint/tree/2.4.0) (2019-10-08)
+[Full Changelog](https://github.com/rodjek/puppet-lint/compare/2.3.6...2.4.0)
+
+**Fixed bugs:**
+
+- Command line options do not override options from config files [\#879](https://github.com/rodjek/puppet-lint/issues/879)
+- Fix for variables\_not\_enclosed incorrectly handles variables followed by a dash [\#836](https://github.com/rodjek/puppet-lint/issues/836)
+- Syntax error on Pattern data type [\#833](https://github.com/rodjek/puppet-lint/issues/833)
+- Error with puppet-lint --fix: NoMethodError: undefined method `next\_token' for nil:NilClass [\#831](https://github.com/rodjek/puppet-lint/issues/831)
+- TypeError: no implicit conversion of nil into String [\#830](https://github.com/rodjek/puppet-lint/issues/830)
+- Selector with 'default' case disables check for missing default in outer case statement [\#829](https://github.com/rodjek/puppet-lint/issues/829)
+- undefined method `next\_token' for nil:NilClass [\#824](https://github.com/rodjek/puppet-lint/issues/824)
+- NoMethodError: undefined method `next\_token' for nil:NilClass [\#790](https://github.com/rodjek/puppet-lint/issues/790)
+- Puppet-lint --fix silently removes necessary $ inside double quoted strings [\#773](https://github.com/rodjek/puppet-lint/issues/773)
+- It looks like puppet-lint has encountered an error that it doesn't know how to handle [\#768](https://github.com/rodjek/puppet-lint/issues/768)
+- puppet-lint lexer string interpolation needs to be updated to match PUP-5887 changes [\#747](https://github.com/rodjek/puppet-lint/issues/747)
+- Syntax error causes 'Whoops!' [\#740](https://github.com/rodjek/puppet-lint/issues/740)
+- "quoted boolean value found" in hash value should not raise a warning. [\#474](https://github.com/rodjek/puppet-lint/issues/474)
+
+**Closed issues:**
+
+- Error when running puppet-lint [\#862](https://github.com/rodjek/puppet-lint/issues/862)
+- puppet-lint crashes with mispelled namespace seperators [\#853](https://github.com/rodjek/puppet-lint/issues/853)
+- NoMethodError: undefined method `prev\_token' for nil:NilClass [\#845](https://github.com/rodjek/puppet-lint/issues/845)
+- Lint incorrectly errors on quoted bool [\#844](https://github.com/rodjek/puppet-lint/issues/844)
+- Type\[\]\] raises NoMethodError [\#843](https://github.com/rodjek/puppet-lint/issues/843)
+- Whoops! It looks like puppet-lint has encountered an error that it doesn't know how to handle. [\#842](https://github.com/rodjek/puppet-lint/issues/842)
+- Whoops! It looks like puppet-lint has encountered an error that it doesn't [\#838](https://github.com/rodjek/puppet-lint/issues/838)
+- Incorrectly wrapped hash variable inside double quotes [\#826](https://github.com/rodjek/puppet-lint/issues/826)
+- Test puppet-lint against Ruby 2.5.x [\#818](https://github.com/rodjek/puppet-lint/issues/818)
+- nested ensure misdetected as not coming first. [\#410](https://github.com/rodjek/puppet-lint/issues/410)
+
+**Merged pull requests:**
+
+- Handle unenclosed variables followed by dashes when fixing [\#881](https://github.com/rodjek/puppet-lint/pull/881) ([rodjek](https://github.com/rodjek))
+- Let command line args override config from files [\#880](https://github.com/rodjek/puppet-lint/pull/880) ([usev6](https://github.com/usev6))
+- Ignore hash keys when checking resource parameter order [\#877](https://github.com/rodjek/puppet-lint/pull/877) ([rodjek](https://github.com/rodjek))
+- Only look for 'default' at first level of 'case' statement [\#876](https://github.com/rodjek/puppet-lint/pull/876) ([usev6](https://github.com/usev6))
+- Report syntax error on unbalanced braces [\#875](https://github.com/rodjek/puppet-lint/pull/875) ([rodjek](https://github.com/rodjek))
+- Include hash/array references when enclosing variables [\#874](https://github.com/rodjek/puppet-lint/pull/874) ([rodjek](https://github.com/rodjek))
+- Disable quoted\_booleans check by default [\#873](https://github.com/rodjek/puppet-lint/pull/873) ([rodjek](https://github.com/rodjek))
+- Test against Ruby 2.5 & 2.6 [\#872](https://github.com/rodjek/puppet-lint/pull/872) ([rodjek](https://github.com/rodjek))
+- README - Add GitHub Actions action [\#868](https://github.com/rodjek/puppet-lint/pull/868) ([ScottBrenner](https://github.com/ScottBrenner))
+- Update TravisCI config to use trusty image [\#867](https://github.com/rodjek/puppet-lint/pull/867) ([rodjek](https://github.com/rodjek))
+- Use the default travis rubygems & bundler [\#860](https://github.com/rodjek/puppet-lint/pull/860) ([rodjek](https://github.com/rodjek))
+- Add `Sensitive` to the list of KNOWN\_TOKEN TYPES [\#858](https://github.com/rodjek/puppet-lint/pull/858) ([alexjfisher](https://github.com/alexjfisher))
+- Avoid internal error for typoed namespace [\#855](https://github.com/rodjek/puppet-lint/pull/855) ([usev6](https://github.com/usev6))
+- Use lookahead assertion for matching function name [\#854](https://github.com/rodjek/puppet-lint/pull/854) ([usev6](https://github.com/usev6))
+- Resource: fix nested ensure error. [\#848](https://github.com/rodjek/puppet-lint/pull/848) ([keur](https://github.com/keur))
+- Rewrite double quoted string handling for nested interpolation [\#846](https://github.com/rodjek/puppet-lint/pull/846) ([rodjek](https://github.com/rodjek))
+- Allow for spaces in the heredoc tag [\#841](https://github.com/rodjek/puppet-lint/pull/841) ([jarretlavallee](https://github.com/jarretlavallee))
+- Recognizes multiline regexes [\#835](https://github.com/rodjek/puppet-lint/pull/835) ([jcbollinger](https://github.com/jcbollinger))
+
 ## [2.3.6](https://github.com/rodjek/puppet-lint/tree/2.3.6) (2018-07-09)
 [Full Changelog](https://github.com/rodjek/puppet-lint/compare/2.3.5...2.3.6)
 

data/README.md

@@ -120,6 +120,18 @@ Or to specify a whitelist of allowed checks, include a line like:
 --only-checks=trailing_whitespace,hard_tabs,duplicate_params,double_quoted_strings,unquoted_file_mode,only_variable_string,variables_not_enclosed,single_quote_string_with_variables,variable_contains_dash,ensure_not_symlink_target,unquoted_resource_title,relative_classname_inclusion,file_mode,resource_reference_without_title_capital,leading_zero,arrow_alignment,variable_is_lowercase,ensure_first_param,resource_reference_without_whitespace,file_ensure,trailing_comma,leading_zero
 ```
 
+Please note that there is an important difference between reading options from the command line and reading options from a configuration file: In the former case the shell interprets one level of quotes. That does not happen in the latter case. So, it would make sense to quote some configuration values on the command line, like so:
+
+```
+$ puppet-lint --ignore-paths 'modules/stdlib/*' modules/
+```
+
+When reading from a configuration file those quotes would be passed on to the option parser -- probably not giving the expected result. Instead the line should read
+
+```
+--ignore-paths=modules/stdlib/*
+```
+
 ## Testing with Puppet Lint as a Rake task
 
 To test your entire Puppet manifest directory, add `require 'puppet-lint/tasks/puppet-lint'` to your Rakefile and then run:
@@ -195,6 +207,12 @@ You can also disable checks when running Puppet Lint through the supplied Rake t
   PuppetLint.configuration.pattern = "modules"
   ```
 
+## Testing with Puppet Lint as a GitHub Action
+
+There is a GitHub Actions action available to get linter feedback in workflows:
+
+* [puppet-lint-action](https://github.com/marketplace/actions/puppet-lint-action)
+
 ## Options
 
 See `puppet-lint --help` for a full list of command line options and checks.

data/appveyor.yml

@@ -13,6 +13,8 @@ environment:
     - RUBY_VERSION: 22
     - RUBY_VERSION: 23-x64
     - RUBY_VERSION: 24-x64
+    - RUBY_VERSION: 25-x64
+    - RUBY_VERSION: 26-x64
 
 matrix:
   allow_failures:

data/lib/puppet-lint/data.rb

@@ -201,6 +201,9 @@ class PuppetLint::Data
       lbrace_idx = tokens[0..index].rindex do |token|
         token.type == :LBRACE && token.prev_code_token.type != :QMARK
       end
+
+      raise PuppetLint::SyntaxError, tokens[index] if lbrace_idx.nil?
+
       tokens[lbrace_idx].prev_code_token
     end
 
@@ -212,9 +215,21 @@ class PuppetLint::Data
     #
     # Returns an Array of Token objects.
     def find_resource_param_tokens(resource_tokens)
-      resource_tokens.select do |token|
-        token.type == :NAME && token.next_code_token.type == :FARROW
+      param_tokens = []
+
+      iter_token = resource_tokens.first.prev_token
+
+      until iter_token.nil?
+        iter_token = iter_token.next_token_of(:NAME)
+
+        break unless resource_tokens.include?(iter_token)
+
+        if iter_token && iter_token.next_code_token.type == :FARROW
+          param_tokens << iter_token
+        end
       end
+
+      param_tokens
     end
 
     # Internal: Calculate the positions of all class definitions within the

data/lib/puppet-lint/lexer.rb

@@ -2,8 +2,9 @@
 
 require 'pp'
 require 'strscan'
-require 'puppet-lint/lexer/token'
 require 'set'
+require 'puppet-lint/lexer/token'
+require 'puppet-lint/lexer/string_slurper'
 
 class PuppetLint
   # Internal: A generic error thrown by the lexer when it encounters something
@@ -28,11 +29,15 @@ class PuppetLint
       @column = column
       @reason = reason
     end
+
+    def to_s
+      "PuppetLint::LexerError: Line:#{line_no} Column: #{column} Reason: #{reason}"
+    end
   end
 
   # Internal: The puppet-lint lexer. Converts your manifest into its tokenised
   # form.
-  class Lexer # rubocop:disable Metrics/ClassLength
+  class Lexer
     def initialize
       @line_no = 1
       @column = 1
@@ -101,16 +106,27 @@ class PuppetLint
       :LPAREN  => true,
     }.freeze
 
+    # Internal: some commonly used regular expressions
+    # \t == tab
+    # \v == vertical tab
+    # \f == form feed
+    # \p{Zs} == ASCII + Unicode non-linebreaking whitespace
+    WHITESPACE_RE = RUBY_VERSION == '1.8.7' ? %r{[\t\v\f ]} : %r{[\t\v\f\p{Zs}]}
+
+    LINE_END_RE = %r{(?:\r\n|\r|\n)}
+
+    NAME_RE = %r{\A((?:(?:::)?[_a-z0-9][-\w]*)(?:::[a-z0-9][-\w]*)*)}
+
     # Internal: An Array of Arrays containing tokens that can be described by
     # a single regular expression.  Each sub-Array contains 2 elements, the
     # name of the token as a Symbol and a regular expression describing the
     # value of the token.
-    NAME_RE = %r{\A(((::)?[_a-z0-9][-\w]*)(::[a-z0-9][-\w]*)*)}
     KNOWN_TOKENS = [
-      [:TYPE, %r{\A(Integer|Float|Boolean|Regexp|String|Array|Hash|Resource|Class|Collection|Scalar|Numeric|CatalogEntry|Data|Tuple|Struct|Optional|NotUndef|Variant|Enum|Pattern|Any|Callable|Type|Runtime|Undef|Default)\b}],
+      [:WHITESPACE, %r{\A(#{WHITESPACE_RE}+)}],
+      [:TYPE, %r{\A(Integer|Float|Boolean|Regexp|String|Array|Hash|Resource|Class|Collection|Scalar|Numeric|CatalogEntry|Data|Tuple|Struct|Optional|NotUndef|Variant|Enum|Pattern|Any|Callable|Type|Runtime|Undef|Default|Sensitive)\b}], # rubocop:disable Metrics/LineLength
       [:CLASSREF, %r{\A(((::){0,1}[A-Z][-\w]*)+)}],
       [:NUMBER, %r{\A\b((?:0[xX][0-9A-Fa-f]+|0?\d+(?:\.\d+)?(?:[eE]-?\d+)?))\b}],
-      [:FUNCTION_NAME, %r{#{NAME_RE}\(}],
+      [:FUNCTION_NAME, %r{#{NAME_RE}(?=\()}],
       [:NAME, NAME_RE],
       [:LBRACK, %r{\A(\[)}],
       [:RBRACK, %r{\A(\])}],
@@ -166,14 +182,6 @@ class PuppetLint
       :INDENT        => true,
     }.freeze
 
-    # \t == tab
-    # \v == vertical tab
-    # \f == form feed
-    # \p{Zs} == ASCII + Unicode non-linebreaking whitespace
-    WHITESPACE_RE = RUBY_VERSION == '1.8.7' ? %r{[\t\v\f ]} : %r{[\t\v\f\p{Zs}]}
-
-    LINE_END_RE = %r{(?:\r\n|\r|\n)}
-
     # Internal: Access the internal token storage.
     #
     # Returns an Array of PuppetLint::Lexer::Toxen objects.
@@ -200,13 +208,12 @@ class PuppetLint
           value = chunk[regex, 1]
           next if value.nil?
 
-          length = value.size
+          i += value.size
           tokens << if type == :NAME && KEYWORDS.include?(value)
                       new_token(value.upcase.to_sym, value)
                     else
                       new_token(type, value)
                     end
-          i += length
           found = true
           break
         end
@@ -215,7 +222,12 @@ class PuppetLint
 
         if var_name = chunk[%r{\A\$((::)?(\w+(-\w+)*::)*\w+(-\w+)*(\[.+?\])*)}, 1]
           length = var_name.size + 1
-          tokens << new_token(:VARIABLE, var_name)
+          opts = if chunk.start_with?('$')
+                   { :raw => "$#{var_name}" }
+                 else
+                   {}
+                 end
+          tokens << new_token(:VARIABLE, var_name, opts)
 
         elsif chunk =~ %r{\A'.*?'}m
           str_content = StringScanner.new(code[i + 1..-1]).scan_until(%r{(\A|[^\\])(\\\\)*'}m)
@@ -223,12 +235,16 @@ class PuppetLint
           tokens << new_token(:SSTRING, str_content[0..-2])
 
         elsif chunk.start_with?('"')
-          str_contents = slurp_string(code[i + 1..-1])
-          lines_parsed = code[0..i].split(LINE_END_RE)
-          interpolate_string(str_contents, lines_parsed.count, lines_parsed.last.length)
-          length = str_contents.size + 1
+          slurper = PuppetLint::Lexer::StringSlurper.new(code[i + 1..-1])
+          begin
+            string_segments = slurper.parse
+            process_string_segments(string_segments)
+            length = slurper.consumed_bytes + 1
+          rescue PuppetLint::Lexer::StringSlurper::UnterminatedStringError
+            raise PuppetLint::LexerError, @line_no, @column, 'unterminated string'
+          end
 
-        elsif heredoc_name = chunk[%r{\A@\(("?.+?"?(:.+?)?(/.*?)?)\)}, 1]
+        elsif heredoc_name = chunk[%r{\A@\(("?.+?"?(:.+?)?#{WHITESPACE_RE}*(/.*?)?)\)}, 1]
           heredoc_queue << heredoc_name
           tokens << new_token(:HEREDOC_OPEN, heredoc_name)
           length = heredoc_name.size + 3
@@ -251,7 +267,7 @@ class PuppetLint
           mlcomment.gsub!(%r{^ *\*}, '')
           tokens << new_token(:MLCOMMENT, mlcomment, :raw => mlcomment_raw)
 
-        elsif chunk.match(%r{\A/.*?/}) && possible_regex?
+        elsif chunk.match(%r{\A/.*?/}m) && possible_regex?
           str_content = StringScanner.new(code[i + 1..-1]).scan_until(%r{(\A|[^\\])(\\\\)*/}m)
           length = str_content.size + 1
           tokens << new_token(:REGEX, str_content[0..-2])
@@ -267,29 +283,16 @@ class PuppetLint
             length += indent.size
           else
             heredoc_tag = heredoc_queue.shift
-            heredoc_name = heredoc_tag[%r{\A"?(.+?)"?(:.+?)?(/.*)?\Z}, 1]
-            str_contents = StringScanner.new(code[(i + length)..-1]).scan_until(%r{\|?\s*-?\s*#{heredoc_name}})
-            interpolate_heredoc(str_contents, heredoc_tag)
-            length += str_contents.size
+            slurper = PuppetLint::Lexer::StringSlurper.new(code[i + length..-1])
+            heredoc_segments = slurper.parse_heredoc(heredoc_tag)
+            process_heredoc_segments(heredoc_segments)
+            length += slurper.consumed_bytes
           end
 
-        elsif whitespace = chunk[%r{\A(#{WHITESPACE_RE}+)}, 1]
-          length = whitespace.size
-          tokens << new_token(:WHITESPACE, whitespace)
-
         elsif eol = chunk[%r{\A(#{LINE_END_RE})}, 1]
           length = eol.size
           tokens << new_token(:NEWLINE, eol)
 
-          unless heredoc_queue.empty?
-            heredoc_tag = heredoc_queue.shift
-            heredoc_name = heredoc_tag[%r{\A"?(.+?)"?(:.+?)?(/.*)?\Z}, 1]
-            str_contents = StringScanner.new(code[(i + length)..-1]).scan_until(%r{\|?\s*-?\s*#{heredoc_name}})
-            _ = code[0..(i + length)].split(LINE_END_RE)
-            interpolate_heredoc(str_contents, heredoc_tag)
-            length += str_contents.size
-          end
-
         elsif chunk.start_with?('/')
           length = 1
           tokens << new_token(:DIV, '/')
@@ -308,22 +311,6 @@ class PuppetLint
       tokens
     end
 
-    def slurp_string(string)
-      dq_str_regexp = %r{(\$\{|(\A|[^\\])(\\\\)*")}m
-      scanner = StringScanner.new(string)
-      contents = scanner.scan_until(dq_str_regexp)
-
-      if scanner.matched.nil?
-        raise LexerError.new(@line_no, @column, 'Double quoted string missing closing quote')
-      end
-
-      until scanner.matched.end_with?('"')
-        contents += scanner.scan_until(%r{\}}m)
-        contents += scanner.scan_until(dq_str_regexp)
-      end
-      contents
-    end
-
     # Internal: Given the tokens already processed, determine if the next token
     # could be a regular expression.
     #
@@ -403,167 +390,66 @@ class PuppetLint
       token
     end
 
-    # Internal: Split a string on multiple terminators, excluding escaped
-    # terminators.
-    #
-    # string      - The String to be split.
-    # terminators - The String of terminators that the String should be split
-    #               on.
-    #
-    # Returns an Array consisting of two Strings, the String up to the first
-    # terminator and the terminator that was found.
-    def get_string_segment(string, terminators)
-      str = string.scan_until(%r{([^\\]|^|[^\\])([\\]{2})*[#{terminators}]+})
-      begin
-        [str[0..-2], str[-1, 1]]
-      rescue
-        [nil, nil]
+    def process_string_segments(segments)
+      return if segments.empty?
+
+      if segments.length == 1
+        tokens << new_token(:STRING, segments[0][1])
+        return
       end
-    end
 
-    # Internal: Tokenise the contents of a double quoted string.
-    #
-    # string - The String to be tokenised.
-    # line   - The Integer line number of the start of the passed string.
-    # column - The Integer column number of the start of the passed string.
-    #
-    # Returns nothing.
-    def interpolate_string(string, line, column)
-      ss = StringScanner.new(string)
-      first = true
-      value, terminator = get_string_segment(ss, '"$')
-      until value.nil?
-        if terminator == '"'
-          if first
-            tokens << new_token(:STRING, value, :line => line, :column => column)
-            first = false
-          else
-            token_column = column + (ss.pos - value.size)
-            tokens << new_token(:DQPOST, value, :line => line, :column => token_column)
-            line += value.scan(LINE_END_RE).size
-            @column = column + ss.pos + 1
-            @line_no = line
+      pre_segment = segments.delete_at(0)
+      post_segment = segments.delete_at(-1)
+
+      tokens << new_token(:DQPRE, pre_segment[1])
+      segments.each do |segment|
+        case segment[0]
+        when :INTERP
+          lexer = PuppetLint::Lexer.new
+          lexer.tokenise(segment[1])
+          lexer.tokens.each_with_index do |t, i|
+            type = i.zero? && t.type == :NAME ? :VARIABLE : t.type
+            tokens << new_token(type, t.value, :raw => t.raw)
           end
+        when :UNENC_VAR
+          tokens << new_token(:UNENC_VARIABLE, segment[1].gsub(%r{\A\$}, ''))
         else
-          if first
-            tokens << new_token(:DQPRE, value, :line => line, :column => column)
-            first = false
-          else
-            token_column = column + (ss.pos - value.size)
-            tokens << new_token(:DQMID, value, :line => line, :column => token_column)
-            line += value.scan(LINE_END_RE).size
-          end
-          if ss.scan(%r{\{}).nil?
-            var_name = ss.scan(%r{(::)?(\w+(-\w+)*::)*\w+(-\w+)*})
-            if var_name.nil?
-              token_column = column + ss.pos - 1
-              tokens << new_token(:DQMID, '$', :line => line, :column => token_column)
-            else
-              token_column = column + (ss.pos - var_name.size)
-              tokens << new_token(:UNENC_VARIABLE, var_name, :line => line, :column => token_column)
-            end
-          else
-            line += value.scan(LINE_END_RE).size
-            contents = ss.scan_until(%r{\}})[0..-2]
-            raw = contents.dup
-            if contents.match(%r{\A(::)?([\w-]+::)*[\w-]+(\[.+?\])*}) && !contents.match(%r{\A\w+\(})
-              contents = "$#{contents}"
-            end
-            lexer = PuppetLint::Lexer.new
-            lexer.tokenise(contents)
-            lexer.tokens.each do |token|
-              tok_col = column + token.column + (ss.pos - contents.size - 1)
-              tok_line = token.line + line - 1
-              tokens << new_token(token.type, token.value, :line => tok_line, :column => tok_col)
-            end
-            if lexer.tokens.length == 1 && lexer.tokens[0].type == :VARIABLE
-              tokens.last.raw = raw
-            end
-          end
+          tokens << new_token(:DQMID, segment[1])
         end
-        value, terminator = get_string_segment(ss, '"$')
       end
+      tokens << new_token(:DQPOST, post_segment[1])
     end
 
-    # Internal: Tokenise the contents of a heredoc.
-    #
-    # string - The String to be tokenised.
-    # name   - The String name/endtext of the heredoc.
-    #
-    # Returns nothing.
-    def interpolate_heredoc(string, name)
-      ss = StringScanner.new(string)
-      eos_text = name[%r{\A"?(.+?)"?(:.+?)?(/.*)?\Z}, 1]
-      first = true
-      interpolate = name.start_with?('"')
-      value, terminator = get_heredoc_segment(ss, eos_text, interpolate)
-      until value.nil?
-        if terminator =~ %r{\A\|?\s*-?\s*#{Regexp.escape(eos_text)}}
-          if first
-            tokens << new_token(:HEREDOC, value, :raw => "#{value}#{terminator}")
-            first = false
-          else
-            tokens << new_token(:HEREDOC_POST, value, :raw => "#{value}#{terminator}")
+    def process_heredoc_segments(segments)
+      return if segments.empty?
+
+      end_tag = segments.delete_at(-1)
+
+      if segments.length == 1
+        tokens << new_token(:HEREDOC, segments[0][1], :raw => "#{segments[0][1]}#{end_tag[1]}")
+        return
+      end
+
+      pre_segment = segments.delete_at(0)
+      post_segment = segments.delete_at(-1)
+
+      tokens << new_token(:HEREDOC_PRE, pre_segment[1])
+      segments.each do |segment|
+        case segment[0]
+        when :INTERP
+          lexer = PuppetLint::Lexer.new
+          lexer.tokenise(segment[1])
+          lexer.tokens.each_with_index do |t, i|
+            type = i.zero? && t.type == :NAME ? :VARIABLE : t.type
+            tokens << new_token(type, t.value, :raw => t.raw)
           end
+        when :UNENC_VAR
+          tokens << new_token(:UNENC_VARIABLE, segment[1].gsub(%r{\A\$}, ''))
         else
-          if first
-            tokens << new_token(:HEREDOC_PRE, value)
-            first = false
-          else
-            tokens << new_token(:HEREDOC_MID, value)
-          end
-          if ss.scan(%r{\{}).nil?
-            var_name = ss.scan(%r{(::)?(\w+(-\w+)*::)*\w+(-\w+)*})
-            tokens << if var_name.nil?
-                        new_token(:HEREDOC_MID, '$')
-                      else
-                        new_token(:UNENC_VARIABLE, var_name)
-                      end
-          else
-            contents = ss.scan_until(%r{\}})[0..-2]
-            raw = contents.dup
-            if contents.match(%r{\A(::)?([\w-]+::)*[\w-]|(\[.+?\])*}) && !contents.match(%r{\A\w+\(})
-              contents = "$#{contents}" unless contents.start_with?('$')
-            end
-
-            lexer = PuppetLint::Lexer.new
-            lexer.tokenise(contents)
-            lexer.tokens.each do |token|
-              tokens << new_token(token.type, token.value)
-            end
-            if lexer.tokens.length == 1 && lexer.tokens[0].type == :VARIABLE
-              tokens.last.raw = raw
-            end
-          end
+          tokens << new_token(:HEREDOC_MID, segment[1])
         end
-        value, terminator = get_heredoc_segment(ss, eos_text, interpolate)
-      end
-    end
-
-    # Internal: Splits a heredoc String into segments if it is to be
-    # interpolated.
-    #
-    # string      - The String heredoc.
-    # eos_text    - The String endtext for the heredoc.
-    # interpolate - A Boolean that specifies whether this heredoc can contain
-    #               interpolated values (defaults to True).
-    #
-    # Returns an Array consisting of two Strings, the String up to the first
-    # terminator and the terminator that was found.
-    def get_heredoc_segment(string, eos_text, interpolate = true)
-      regexp = if interpolate
-                 %r{(([^\\]|^|[^\\])([\\]{2})*[$]+|\|?\s*-?#{Regexp.escape(eos_text)})}
-               else
-                 %r{\|?\s*-?#{Regexp.escape(eos_text)}}
-               end
-
-      str = string.scan_until(regexp)
-      begin
-        str =~ %r{\A(.*?)([$]+|\|?\s*-?#{Regexp.escape(eos_text)})\Z}m
-        [Regexp.last_match(1), Regexp.last_match(2)]
-      rescue
-        [nil, nil]
       end
+      tokens << new_token(:HEREDOC_POST, post_segment[1], :raw => "#{post_segment[1]}#{end_tag[1]}")
     end
   end
 end