arachni 1.1 → 1.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8f24a93602ee05ee80f03367f6076200362efff6
-  data.tar.gz: 0e6c3ee901415342be5661d62bf41925cc6b36f2
+  metadata.gz: b7b04bebb490a564d756ded63b03da429d3f96cb
+  data.tar.gz: d8b634d80fb6db78621fb574b6c52954da0cb201
 SHA512:
-  metadata.gz: ab0b21531f7be52bbe924d1a58f58de719222b72df67431c22898d519dd66bd47f297166b083a68add14cf64ee64ea3497c66350f3c68ec9da0eeda85a56751c
-  data.tar.gz: 54e4aff6b4437087c2f32919c2eb669648fc88e4fb42c397ec27a64058741af717bd750eb1a0c5d4f1cefa46cfb44750049e024a57c4538409b9686daee17b54
+  metadata.gz: 84acf5244c5bb7e1c3dd05e1fe85f27087bbfa447c45dea559a4496fa7947bac87c75dd3c7e5550be5f1c87ed42ca69ffb825b62d3e59a1c789ac55148d8985b
+  data.tar.gz: c2c0d02d9194befc03fc37d1b38a036f4ef448336f3c6b3e8ff5ac23140f4fc035e09b698a347a5c47738ec5c6aa25b42692e4979b951756920175fa6e587db9

data/CHANGELOG.md

@@ -1,5 +1,162 @@
 # ChangeLog
 
+## 1.2 _(July 16, 2015)_
+
+- Switched to Arachni Public Source License v1.0.
+- `UI`
+    - `CLI::Framework`
+        - Fixed timeout enforcement.
+        - `OptionParser`
+            - Added `--browser-cluster-wait-for-element`.
+    - `Output`
+        - `#error_log_fd` -- Catch `Errno` system errors (like `Too many open files`)
+            to avoid crashing.
+- `OptionGroups`
+    - `HTTP`
+        - `#request_queue_size` -- Lowered from `500` to `100`.
+    - `BrowserCluster`
+        - `#wait_for_elements` -- Wait for element matching `CSS` to appear when
+            visiting a page whose URL matches the `PATTERN`.
+        - `#job_timeout` -- Increased from 15 to 25 seconds.
+- `Framework`
+    - `#pause` -- Pause is now near instant.
+    - `#audit` -- Substantially simplified and optimized the consumption of URL
+        and page queues.
+    - `#audit_page` -- Application of DOM metadata now happens asynchronously
+        and uses the `BrowserCluster` instead of an independent `Browser`.
+- `HTTP`
+    - `Client`
+        - Updated cookie setting from `OptionGroups::HTTP#cookies` `Hash`.
+        - Trigger garbage collections before and after performing the queued
+            requests to prevent large RAM spikes.
+        - `Dynamic404Handler`
+            - Account for cases where the server returns intermittent errors
+                that can lead to signature corruption and possibly false positives.
+            - Updated training scenarios for cases where `~` are ignored.
+            - Disable platform fingerprinting during the gathering of signatures.
+    - `Request`
+        - Ignore proxy-related traffic (`CONNECT`) when capturing raw traffic data.
+        - Added `#fingerprint` option to enable/disable platform fingerprinting
+            on a per request basis.
+        - `#response_max_size` -- In addition to setting the `maxfilesize` for
+            the `Typhoeus::Request`, stream bodies and manually abort if the
+            buffer exceeds the limit -- covers cases where no `Content-Type`
+            is set.
+    - `Headers`
+        - Merge values of headers with identical normalized names (i.e.
+            `set-cookie` and `Set-Cookie` in the same response).
+        - Cache header name canonicalization.
+    - `ProxyServer`
+        - Cache header name canonicalization.
+        - SSL interceptor now automatically generates certificate/key pairs
+            based on Arachni CA.
+- `Page`
+    - `#has_script?` -- Detect using the body instead of the parsed document.
+- `Parser`
+    - Optimized to avoid HTML parsing if it contains no indication of elements.
+    - `#headers` -- Updated to include headers from the HTTP request in addition
+        to common ones.
+    - `Extractors` -- Optimized to avoid HTML parsing if it contains no
+        indication of elements.
+- `Element`
+    - Cleaned up per-element input value encoding.
+    - Enforce a `MAX_SIZE` on acceptable values during parsing.
+    - Optimized to avoid HTML parsing if it contains no indication of elements.
+    - `Server`
+        - `#log_remote_file_if_exists?` -- Flag issues as untrusted at that point
+        if possible, instead of at the end of the scan.
+        - `#remote_file_exist?` -- Disable platform fingerprinting when dealing
+            with a dynamic handler.
+    - `Capabilities`
+        - `Inputtable` -- Added cache for `#inputtable_id` calculation.
+        - `Analyzable`
+            - `Taint` -- Added match cache based on signatures and haystacks.
+            - `Timeout` -- Override user audit options that don't play nice with this technique.
+- `Check::Auditor`
+    - `#log_remote_file` -- Assign `HTTP::Response#status_line` as proof.
+- `Issue`
+    - `#signature` -- Store `Regexp` source instead of converting it to String.
+- `Browser`
+    - Updated to extract and whitelist CDNs from response bodies.
+    - `#cookies` -- Normalize cookies with quoted values since Watir doesn't take
+        care of that bit.
+    - `Javascript`
+        - `#inject` -- Inject `TaintTracer` and `DOMMonitor` update calls in
+            requested JS assets.
+        - `TaintTracer`
+            - Limited data and execution flow sinks to a max size of 50 entries.
+            - Don't trace functions known to cause issues:
+                - Anonymous functions.
+                - `lodash()`
+        - `DOMMonitor`
+            - Keep track of `jQuery` delegated events.
+- `Support`
+    - `Cache`
+        - `RandomReplacement` -- Removed extra key `Array`.
+    - `Signature` -- Cache token generation.
+- Checks -- Added `Issue#proof` to as many issues as possible.
+    - Active
+        - `xss`
+            - When the case involves payloads landing in `textarea`s, break out of
+                them to prevent possible FPs.
+            - Added double-encoded payloads.
+        - `xss_dom_inputs`
+            - Don't perform redundant audits.
+            - Don't process custom events.
+            - Updated to handle cases where a button needs to be clicked after
+                filling in the inputs.
+            - Added progress messages.
+        - `unvalidated_redirect`
+            - Escalated severity to 'High'.
+            - Only perform straight payload injections.
+        - `unvalidated_redirect_dom`
+            - Escalated severity to 'High'.
+        - `path_traversal`, `file_inclusion`, `os_cmd_injection`, `xxe`
+            - Updated `/etc/passwd` content matching pattern.
+    - Passive
+        - Added
+            - `common_admin_intefaces` -- By Brendan Coles.
+        - `backdoors`, `backup_directories`, `backup_files`, `common_files`,
+            `directory_listing`
+            - Added MVC frameworks as exempt platforms since they do their own routing.
+- Plugins
+    - Added
+        - `restrict_to_dom_state` -- Restricts the audit to a single page's DOM
+            state, based on a URL fragment.
+        - `metrics` -- Captures metrics about multiple aspects of the scan and
+            the web application.
+    - `autologin` -- Updated to fail gracefully in cases of an invisible form DOM elements.
+    - `login_script` -- Added support for Javascript login scripts.
+    - `proxy`
+        - Updated to show JSON and XML inputs in the inspection page.
+        - Added output message with instructions for server that use SSL.
+    - `vector_feed` -- Updated to support XML and JSON elements.
+- Reporters
+    - `xml`
+        - Fixed bug causing vector `affected_input_name` to be blank.
+- Fingerprinters -- Optimized across the board to prefer less resource intensive checks.
+    - Frameworks
+        - Rack -- Expanded signatures.
+    - Languages
+        - JSP renamed to Java and expanded signatures.
+        - PHP -- Expanded signatures.
+        - Python -- Expanded signatures.
+    - Servers
+        - Tomcat -- Expanded signatures.
+    - Added
+        - Frameworks
+            - Django
+            - Rails
+            - ASP.NET MVC
+            - CakePHP
+            - JSF
+            - CherryPy
+        - Servers
+            - Gunicorn
+- Path extractors
+    - Added
+        - `data_url` -- Extracts paths from `data-url` attributes of `a` tags.
+
 ## 1.1 _(May 1, 2015)_
 
 - `gemspec` -- Require Ruby >= 2.0.0.
@@ -96,6 +253,8 @@
     - Added `.full_and_absolute_url?`.
 - `Browser`
     - Updated to extract JSON and XML input vectors from HTTP requests.
+    - `#cookies` -- Normalize cookies with quoted values since Watir doesn't take
+        care of that bit.
     - `#shutdown` -- Fixed Selenium exceptions on dead browser process.
     - `#to_page` -- Apply DOM metadata to page elements.
     - `#spawn_phantomjs` -- Enabled `--disk-cache` option for `phantomjs`.

data/LICENSE.md

@@ -2,210 +2,140 @@
 
 Copyright 2010-2015 [Tasos Laskos](mailto:tasos.laskos@arachni-scanner.com).
 
-The Arachni Framework (henceforth referred to simply as "Arachni") is dual-licensed.
+```
+                    Arachni Public Source License
+                        Version 1.0, June 2015
 
-Cases that include **commercialization** of Arachni require a commercial,
-non-free license. Otherwise, the system can be used under the terms of
-Apache License v2.0, found at the bottom of this document.
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 
-Cases of **commercialization** are:
+1. Definitions
 
-* Using Arachni to provide commercial managed/Software-as-a-Service services.
-* Distributing Arachni as a commercial product or as part of one.
+    "License" shall mean the terms and conditions for use, reproduction,
+    and distribution as defined by Sections 1 through 9 of this document.
 
-Cases which **don't** require a commercial license, and thus fall under
-the terms of Apache License v2.0, include (but are not limited to):
+    "Licensor" shall mean the copyright owner or entity authorized by
+    the copyright owner that is granting the License.
 
-* Penetration testers (or penetration testing organizations) using Arachni as
-    part of their assessment toolkit.
-    * So long as that doesn't conflict with the **commercialization** clause.
-* Using Arachni to test your own systems.
-* Any non-commercial use of Arachni.
+    "Legal Entity" shall mean the union of the acting entity and all
+    other entities that control, are controlled by, or are under common
+    control with that entity. For the purposes of this definition,
+    "control" means (i) the power, direct or indirect, to cause the
+    direction or management of such entity, whether by contract or
+    otherwise, or (ii) ownership of fifty percent (50%) or more of the
+    outstanding shares, or (iii) beneficial ownership of such entity.
 
-If you need to acquire a commercial license or are unsure about whether you
-need to acquire a commercial license, please get in touch, we'll be happy to
-clarify things for you and work with you to accommodate your requirements.
+    "You" (or "Your") shall mean an individual or Legal Entity
+    exercising permissions granted by this License.
 
-You can use the [licensing contact form](http://www.arachni-scanner.com/license)
-to contact us about these matters.
+    "Source" form shall mean the preferred form for making modifications,
+    including but not limited to software source code, documentation
+    source, and configuration files.
 
-Code contributions will be accepted under the Apache License v2.0.
+    "Object" form shall mean any form resulting from mechanical
+    transformation or translation of a Source form, including but
+    not limited to compiled object code, generated documentation,
+    and conversions to other media types.
 
-## Apache License v2.0
+    "Work" shall mean the work of authorship, whether in Source or
+    Object form, made available under the License, as indicated by a
+    copyright notice that is included in or attached to the work.
 
-```
-                                 Apache License
-                           Version 2.0, January 2004
-                        http://www.apache.org/licenses/
-
-   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
-   1. Definitions.
-
-      "License" shall mean the terms and conditions for use, reproduction,
-      and distribution as defined by Sections 1 through 9 of this document.
-
-      "Licensor" shall mean the copyright owner or entity authorized by
-      the copyright owner that is granting the License.
-
-      "Legal Entity" shall mean the union of the acting entity and all
-      other entities that control, are controlled by, or are under common
-      control with that entity. For the purposes of this definition,
-      "control" means (i) the power, direct or indirect, to cause the
-      direction or management of such entity, whether by contract or
-      otherwise, or (ii) ownership of fifty percent (50%) or more of the
-      outstanding shares, or (iii) beneficial ownership of such entity.
-
-      "You" (or "Your") shall mean an individual or Legal Entity
-      exercising permissions granted by this License.
-
-      "Source" form shall mean the preferred form for making modifications,
-      including but not limited to software source code, documentation
-      source, and configuration files.
-
-      "Object" form shall mean any form resulting from mechanical
-      transformation or translation of a Source form, including but
-      not limited to compiled object code, generated documentation,
-      and conversions to other media types.
-
-      "Work" shall mean the work of authorship, whether in Source or
-      Object form, made available under the License, as indicated by a
-      copyright notice that is included in or attached to the work
-      (an example is provided in the Appendix below).
-
-      "Derivative Works" shall mean any work, whether in Source or Object
-      form, that is based on (or derived from) the Work and for which the
-      editorial revisions, annotations, elaborations, or other modifications
-      represent, as a whole, an original work of authorship. For the purposes
-      of this License, Derivative Works shall not include works that remain
-      separable from, or merely link (or bind by name) to the interfaces of,
-      the Work and Derivative Works thereof.
-
-      "Contribution" shall mean any work of authorship, including
-      the original version of the Work and any modifications or additions
-      to that Work or Derivative Works thereof, that is intentionally
-      submitted to Licensor for inclusion in the Work by the copyright owner
-      or by an individual or Legal Entity authorized to submit on behalf of
-      the copyright owner. For the purposes of this definition, "submitted"
-      means any form of electronic, verbal, or written communication sent
-      to the Licensor or its representatives, including but not limited to
-      communication on electronic mailing lists, source code control systems,
-      and issue tracking systems that are managed by, or on behalf of, the
-      Licensor for the purpose of discussing and improving the Work, but
-      excluding communication that is conspicuously marked or otherwise
-      designated in writing by the copyright owner as "Not a Contribution."
-
-      "Contributor" shall mean Licensor and any individual or Legal Entity
-      on behalf of whom a Contribution has been received by Licensor and
-      subsequently incorporated within the Work.
-
-   2. Grant of Copyright License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      copyright license to reproduce, prepare Derivative Works of,
-      publicly display, publicly perform, sublicense, and distribute the
-      Work and such Derivative Works in Source or Object form.
-
-   3. Grant of Patent License. Subject to the terms and conditions of
-      this License, each Contributor hereby grants to You a perpetual,
-      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
-      (except as stated in this section) patent license to make, have made,
-      use, offer to sell, sell, import, and otherwise transfer the Work,
-      where such license applies only to those patent claims licensable
-      by such Contributor that are necessarily infringed by their
-      Contribution(s) alone or by combination of their Contribution(s)
-      with the Work to which such Contribution(s) was submitted. If You
-      institute patent litigation against any entity (including a
-      cross-claim or counterclaim in a lawsuit) alleging that the Work
-      or a Contribution incorporated within the Work constitutes direct
-      or contributory patent infringement, then any patent licenses
-      granted to You under this License for that Work shall terminate
-      as of the date such litigation is filed.
-
-   4. Redistribution. You may reproduce and distribute copies of the
-      Work or Derivative Works thereof in any medium, with or without
-      modifications, and in Source or Object form, provided that You
-      meet the following conditions:
-
-      (a) You must give any other recipients of the Work or
-          Derivative Works a copy of this License; and
-
-      (b) You must cause any modified files to carry prominent notices
-          stating that You changed the files; and
-
-      (c) You must retain, in the Source form of any Derivative Works
-          that You distribute, all copyright, patent, trademark, and
-          attribution notices from the Source form of the Work,
-          excluding those notices that do not pertain to any part of
-          the Derivative Works; and
-
-      (d) If the Work includes a "NOTICE" text file as part of its
-          distribution, then any Derivative Works that You distribute must
-          include a readable copy of the attribution notices contained
-          within such NOTICE file, excluding those notices that do not
-          pertain to any part of the Derivative Works, in at least one
-          of the following places: within a NOTICE text file distributed
-          as part of the Derivative Works; within the Source form or
-          documentation, if provided along with the Derivative Works; or,
-          within a display generated by the Derivative Works, if and
-          wherever such third-party notices normally appear. The contents
-          of the NOTICE file are for informational purposes only and
-          do not modify the License. You may add Your own attribution
-          notices within Derivative Works that You distribute, alongside
-          or as an addendum to the NOTICE text from the Work, provided
-          that such additional attribution notices cannot be construed
-          as modifying the License.
-
-      You may add Your own copyright statement to Your modifications and
-      may provide additional or different license terms and conditions
-      for use, reproduction, or distribution of Your modifications, or
-      for any such Derivative Works as a whole, provided Your use,
-      reproduction, and distribution of the Work otherwise complies with
-      the conditions stated in this License.
-
-   5. Submission of Contributions. Unless You explicitly state otherwise,
-      any Contribution intentionally submitted for inclusion in the Work
-      by You to the Licensor shall be under the terms and conditions of
-      this License, without any additional terms or conditions.
-      Notwithstanding the above, nothing herein shall supersede or modify
-      the terms of any separate license agreement you may have executed
-      with Licensor regarding such Contributions.
-
-   6. Trademarks. This License does not grant permission to use the trade
-      names, trademarks, service marks, or product names of the Licensor,
-      except as required for reasonable and customary use in describing the
-      origin of the Work and reproducing the content of the NOTICE file.
-
-   7. Disclaimer of Warranty. Unless required by applicable law or
-      agreed to in writing, Licensor provides the Work (and each
-      Contributor provides its Contributions) on an "AS IS" BASIS,
-      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
-      implied, including, without limitation, any warranties or conditions
-      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
-      PARTICULAR PURPOSE. You are solely responsible for determining the
-      appropriateness of using or redistributing the Work and assume any
-      risks associated with Your exercise of permissions under this License.
-
-   8. Limitation of Liability. In no event and under no legal theory,
-      whether in tort (including negligence), contract, or otherwise,
-      unless required by applicable law (such as deliberate and grossly
-      negligent acts) or agreed to in writing, shall any Contributor be
-      liable to You for damages, including any direct, indirect, special,
-      incidental, or consequential damages of any character arising as a
-      result of this License or out of the use or inability to use the
-      Work (including but not limited to damages for loss of goodwill,
-      work stoppage, computer failure or malfunction, or any and all
-      other commercial damages or losses), even if such Contributor
-      has been advised of the possibility of such damages.
-
-   9. Accepting Warranty or Additional Liability. While redistributing
-      the Work or Derivative Works thereof, You may choose to offer,
-      and charge a fee for, acceptance of support, warranty, indemnity,
-      or other liability obligations and/or rights consistent with this
-      License. However, in accepting such obligations, You may act only
-      on Your own behalf and on Your sole responsibility, not on behalf
-      of any other Contributor, and only if You agree to indemnify,
-      defend, and hold each Contributor harmless for any liability
-      incurred by, or claims asserted against, such Contributor by reason
-      of your accepting any such warranty or additional liability.
+    "Contribution" shall mean any work of authorship, including
+    the original version of the Work and any modifications or additions
+    to that Work, that is intentionally submitted to Licensor for inclusion in
+    the Work by the copyright owner or by an individual or Legal Entity
+    authorized to submit on behalf of the copyright owner. For the purposes of
+    this definition, "submitted" means any form of electronic, verbal, or
+    written communication sent to the Licensor or its representatives, including
+    but not limited to communication on electronic mailing lists, source code
+    control systems, and issue tracking systems that are managed by, or on
+    behalf of, the Licensor for the purpose of discussing and improving the Work,
+    but excluding communication that is conspicuously marked or otherwise
+    designated in writing by the copyright owner as "Not a Contribution."
+
+    "Contributor" shall mean Licensor and any individual or Legal Entity
+    on behalf of whom a Contribution has been received by Licensor and
+    subsequently incorporated within the Work.
+
+    "Commercialization" shall mean intention to use this software for commercial
+    advantage or monetary compensation.
+
+        Cases of commercialization include but are not limited to:
+
+            1. Use of the Work to provide commercial managed/Software-as-a-Service services.
+            2. Distribution of the Work as a commercial product or as part of one.
+            3. Use or distribution of the Work as a value added service/product.
+
+        Exempt cases:
+
+            1. Penetration testers (or penetration testing organizations) using
+                this Work as part of their manual assessment toolkit.
+            2. Using this Work to assess the security of Your own systems.
+
+2. Basic Permissions
+
+Use of the Work is permitted free of charge, provided that said use does not
+involve Commercialization.
+
+Any use of the Work, in whole or in part, involving Commercialization, is
+strictly prohibited without the prior written consent of Licensor.
+
+Should You require a license that allows for Commercialization, please contact
+Licensor at:
+    license@arachni-scanner.com
+
+In cases of uncertainty, clarifications can be provided by Licensor on a
+case-by-case basis, please contact:
+    license@arachni-scanner.com
+
+3. Redistribution
+
+Redistribution is permitted under the following conditions:
+
+1. Unmodified License is provided with the Work.
+2. Unmodified Copyright notices are provided with the Work.
+3. Does not conflict with Section 2.
+
+4. Copying
+
+Copying is permitted so long as it does not conflict with Section 3.
+
+5. Modification
+
+Modification is permitted so long as it does not conflict with Section 3.
+
+6. Submission of Contributions
+
+Upon submission, Contributor grants to Licensor a perpetual, worldwide,
+non-exclusive, no-charge, royalty-free, irrevocable copyright and patent license
+to reproduce, publicly display, publicly perform, sublicense, distribute, use,
+offer to sell, sell, import, and otherwise transfer the Contribution in Source
+or Object form.
+
+7. Trademarks
+
+This License does not grant permission to use the trade names, trademarks, service
+marks, or product names of the Licensor.
+
+8. Disclaimer of Warranty
+
+Unless required by applicable law or agreed to in writing, Licensor provides the
+Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT
+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without
+limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY,
+or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the
+appropriateness of using or redistributing the Work and assume any risks associated
+with Your exercise of permissions under this License.
+
+9. Limitation of Liability
+
+In no event and under no legal theory, whether in tort (including negligence),
+contract, or otherwise, unless required by applicable law (such as deliberate
+and grossly negligent acts) or agreed to in writing, shall any Contributor be
+liable to You for damages, including any direct, indirect, special, incidental,
+or consequential damages of any character arising as a result of this License or
+out of the use or inability to use the Work (including but not limited to damages
+for loss of goodwill, work stoppage, computer failure or malfunction, or any and
+all other commercial damages or losses), even if such Contributor has been advised
+of the possibility of such damages.
 ```