ceedling 1.2.1 → 1.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/GIT_COMMIT_SHA +1 -1
- data/README.md +11 -9
- data/Rakefile +2 -1
- data/assets/features/ceedling +1 -1
- data/assets/fixtures/test_source_file_duplicate_basenames/test_calc_typo.c +14 -0
- data/assets/fixtures/tests_with_duplicate_test_names/test_foo.c +21 -0
- data/assets/fixtures/tests_with_partials_coverage/src/private_macro.c +7 -0
- data/assets/fixtures/tests_with_partials_coverage/src/private_macro.h +5 -0
- data/assets/fixtures/tests_with_partials_coverage/src/tally.c +13 -0
- data/assets/fixtures/tests_with_partials_coverage/src/tally.h +4 -0
- data/assets/fixtures/tests_with_partials_coverage/test/test_private_macro.c +10 -0
- data/assets/fixtures/tests_with_partials_coverage/test/test_tally.c +9 -0
- data/bin/actionator.rb +6 -12
- data/bin/app_cfg.rb +14 -1
- data/bin/ceedling +13 -3
- data/bin/cli.rb +25 -6
- data/bin/cli_handler.rb +113 -73
- data/bin/cli_helper.rb +51 -78
- data/bin/installation_selector.rb +81 -0
- data/bin/launchinator.rb +69 -0
- data/bin/mixin_resolvinator.rb +26 -36
- data/bin/objects.yml +13 -0
- data/bin/path_validator.rb +25 -29
- data/bin/versionator.rb +47 -48
- data/ceedling.gemspec +19 -15
- data/docs/BreakingChanges.md +86 -1
- data/docs/Changelog.md +145 -3
- data/docs/KnownIssues.md +179 -6
- data/docs/ReleaseNotes.md +13 -0
- data/examples/wondrous_forest/README.md +29 -1
- data/examples/wondrous_forest/src/ForestUnits.h +23 -0
- data/examples/wondrous_forest/src/RainGauge.c +59 -0
- data/examples/wondrous_forest/src/RainGauge.h +39 -0
- data/examples/wondrous_forest/src/RainTipIsr.c +17 -0
- data/examples/wondrous_forest/src/RainTipIsr.h +14 -0
- data/examples/wondrous_forest/test/TestRainGauge.c +126 -0
- data/lib/ceedling/build_log_writer.rb +33 -0
- data/lib/ceedling/c_extractor/README.md +15 -7
- data/lib/ceedling/c_extractor/c_extractor.rb +25 -14
- data/lib/ceedling/c_extractor/c_extractor_code_text.rb +84 -54
- data/lib/ceedling/c_extractor/c_extractor_constants.rb +18 -7
- data/lib/ceedling/c_extractor/c_extractor_declarations.rb +319 -283
- data/lib/ceedling/c_extractor/c_extractor_declarator.rb +285 -0
- data/lib/ceedling/c_extractor/c_extractor_defined_type.rb +111 -0
- data/lib/ceedling/c_extractor/c_extractor_definitions.rb +174 -121
- data/lib/ceedling/c_extractor/c_extractor_functions.rb +191 -162
- data/lib/ceedling/c_extractor/c_extractor_preprocessing.rb +46 -55
- data/lib/ceedling/c_extractor/c_extractor_statement.rb +30 -0
- data/lib/ceedling/c_extractor/c_extractor_types.rb +63 -41
- data/lib/ceedling/c_extractor/c_module_edits.rb +48 -0
- data/lib/ceedling/c_identifier.rb +21 -0
- data/lib/ceedling/ceedling.h +8 -0
- data/lib/ceedling/config/build_root_policy.rb +70 -0
- data/lib/ceedling/config/configurator.rb +18 -19
- data/lib/ceedling/config/configurator_builder.rb +75 -28
- data/lib/ceedling/config/configurator_paths_validator.rb +179 -0
- data/lib/ceedling/config/configurator_plugins.rb +1 -1
- data/lib/ceedling/config/configurator_setup.rb +44 -33
- data/lib/ceedling/config/configurator_validator.rb +1 -110
- data/lib/ceedling/constants.rb +5 -1
- data/lib/ceedling/defaults.rb +35 -5
- data/lib/ceedling/dependencies/CACHE_SCHEMA.md +118 -0
- data/lib/ceedling/dependencies/README.md +18 -16
- data/lib/ceedling/dependencies/dependency_cache_store.rb +43 -21
- data/lib/ceedling/dependencies/dependency_hasher.rb +35 -1
- data/lib/ceedling/dependencies/dependency_tracker.rb +9 -3
- data/lib/ceedling/dependencies/dependinator.rb +13 -1
- data/lib/ceedling/diagnostics/DOCTOR.md +142 -0
- data/lib/ceedling/diagnostics/README.md +5 -0
- data/lib/ceedling/diagnostics/build_step_doctor.rb +110 -0
- data/lib/ceedling/diagnostics/duplicate_symbol_specialist.rb +108 -0
- data/lib/ceedling/diagnostics/guidance_phrases.rb +30 -0
- data/lib/ceedling/diagnostics/header_collision_guidance.rb +56 -0
- data/lib/ceedling/diagnostics/header_collision_specialist.rb +152 -0
- data/lib/ceedling/diagnostics/include_depth_specialist.rb +132 -0
- data/lib/ceedling/diagnostics/missing_symbol_specialist.rb +111 -0
- data/lib/ceedling/diagnostics/partial_name_specialist.rb +134 -0
- data/lib/ceedling/diagnostics/toolchain_output.rb +44 -0
- data/lib/ceedling/filename_extension.rb +7 -1
- data/lib/ceedling/filesystem/FILE_TREE.md +253 -0
- data/lib/ceedling/filesystem/PATH_HANDLING.md +360 -0
- data/lib/ceedling/filesystem/README.md +46 -0
- data/lib/ceedling/filesystem/build_roots.rb +105 -0
- data/lib/ceedling/filesystem/build_subpath.rb +187 -0
- data/lib/ceedling/filesystem/destination_collision.rb +25 -0
- data/lib/ceedling/{file_finder.rb → filesystem/file_finder.rb} +177 -156
- data/lib/ceedling/filesystem/file_finder_helper.rb +154 -0
- data/lib/ceedling/filesystem/file_path_collection_utils.rb +183 -0
- data/lib/ceedling/{file_path_utils.rb → filesystem/file_path_utils.rb} +200 -167
- data/lib/ceedling/{file_system_wrapper.rb → filesystem/file_system_wrapper.rb} +6 -5
- data/lib/ceedling/filesystem/file_tree.rb +241 -0
- data/lib/ceedling/{file_wrapper.rb → filesystem/file_wrapper.rb} +149 -65
- data/lib/ceedling/filesystem/path_exclusions.rb +30 -0
- data/lib/ceedling/{path_matcher.rb → filesystem/path_matcher.rb} +48 -23
- data/lib/ceedling/{path_mirror.rb → filesystem/path_mirror.rb} +19 -18
- data/lib/ceedling/filesystem/path_spellings.rb +55 -0
- data/lib/ceedling/filesystem/project_paths.rb +184 -0
- data/lib/ceedling/filesystem/test_build_layout.rb +114 -0
- data/lib/ceedling/filesystem/win_long_paths.rb +153 -0
- data/lib/ceedling/filesystem/win_path_limit.rb +83 -0
- data/lib/ceedling/filesystem/win_path_prober.rb +52 -0
- data/lib/ceedling/filesystem/working_dir.rb +40 -0
- data/lib/ceedling/generators/generator.rb +32 -70
- data/lib/ceedling/generators/generator_helper.rb +0 -44
- data/lib/ceedling/generators/generator_mocks.rb +5 -0
- data/lib/ceedling/generators/generator_partials.rb +306 -244
- data/lib/ceedling/generators/generator_test_results_backtrace.rb +7 -7
- data/lib/ceedling/generators/generator_test_runner.rb +5 -8
- data/lib/ceedling/generators/partial_header_writer.rb +95 -0
- data/lib/ceedling/generators/partial_source_writer.rb +137 -0
- data/lib/ceedling/handoff.rb +121 -0
- data/lib/ceedling/includes/header_search.rb +225 -0
- data/lib/ceedling/includes/include_factory.rb +9 -12
- data/lib/ceedling/includes/include_pathinator.rb +112 -33
- data/lib/ceedling/includes/includes.rb +150 -38
- data/lib/ceedling/installation.rb +92 -0
- data/lib/ceedling/library_resolver.rb +99 -0
- data/lib/ceedling/objects.yml +112 -6
- data/lib/ceedling/partials/README.md +110 -16
- data/lib/ceedling/partials/carry_plan.rb +71 -0
- data/lib/ceedling/partials/definition_head.rb +74 -0
- data/lib/ceedling/partials/include_groups.rb +37 -0
- data/lib/ceedling/partials/interface_extras.rb +25 -0
- data/lib/ceedling/partials/partializer.rb +603 -124
- data/lib/ceedling/partials/partializer_config.rb +8 -1
- data/lib/ceedling/partials/partializer_helper.rb +851 -145
- data/lib/ceedling/partials/partializer_reconciler.rb +174 -0
- data/lib/ceedling/partials/partializer_utils.rb +89 -109
- data/lib/ceedling/partials/partials.rb +153 -10
- data/lib/ceedling/partials/type_macros.rb +126 -0
- data/lib/ceedling/plugins/report_log_writer_plugin.rb +3 -5
- data/lib/ceedling/preprocess/README.md +43 -33
- data/lib/ceedling/preprocess/preprocessinator.rb +100 -186
- data/lib/ceedling/preprocess/preprocessinator_code_finder.rb +65 -72
- data/lib/ceedling/preprocess/preprocessinator_file_assembler.rb +32 -76
- data/lib/ceedling/preprocess/preprocessinator_includes_handler.rb +1 -1
- data/lib/ceedling/preprocess/preprocessinator_line_marker_includes_extractor.rb +1 -1
- data/lib/ceedling/preprocess/preprocessinator_reconstructor.rb +51 -14
- data/lib/ceedling/rake_app/rakefile_component_resolver.rb +10 -17
- data/lib/ceedling/rakefile.rb +15 -1
- data/lib/ceedling/rakefiles/base/tasks_base.rake +1 -1
- data/lib/ceedling/rakefiles/release/rules_release.rake +4 -7
- data/lib/ceedling/release_invoker/release_build_executor.rb +33 -44
- data/lib/ceedling/release_invoker/release_build_planner.rb +14 -6
- data/lib/ceedling/reportinator.rb +3 -4
- data/lib/ceedling/setupinator.rb +104 -87
- data/lib/ceedling/test_context_extractor.rb +5 -1
- data/lib/ceedling/test_invoker/HEADER_COLLISIONS.md +21 -7
- data/lib/ceedling/test_invoker/PIPELINE.md +62 -19
- data/lib/ceedling/test_invoker/README.md +1 -1
- data/lib/ceedling/test_invoker/derived_search_path_notices.rb +45 -0
- data/lib/ceedling/test_invoker/mock_locator.rb +175 -10
- data/lib/ceedling/test_invoker/partial_files_preprocessinator.rb +181 -0
- data/lib/ceedling/test_invoker/partials_manager.rb +271 -302
- data/lib/ceedling/test_invoker/standin_headerinator.rb +19 -17
- data/lib/ceedling/test_invoker/test_bare_includes_extractinator.rb +26 -3
- data/lib/ceedling/test_invoker/test_build_executor.rb +358 -234
- data/lib/ceedling/test_invoker/test_build_planner.rb +46 -33
- data/lib/ceedling/test_invoker/test_build_setup.rb +43 -18
- data/lib/ceedling/test_invoker/test_build_validations.rb +35 -47
- data/lib/ceedling/test_invoker/test_derived_search_paths.rb +189 -0
- data/lib/ceedling/test_invoker/test_file_preprocessinator.rb +2 -3
- data/lib/ceedling/test_invoker/test_includes_reconcilinator.rb +1 -1
- data/lib/ceedling/test_invoker/test_invoker.rb +6 -1
- data/lib/ceedling/test_invoker/test_invoker_types.rb +28 -20
- data/lib/ceedling/test_invoker/test_pipeline_helpers.rb +6 -4
- data/lib/ceedling/test_invoker/test_pipeline_manager.rb +25 -8
- data/lib/ceedling/test_invoker/test_runner_generatinator.rb +31 -3
- data/lib/ceedling/test_invoker/test_search_paths.rb +30 -8
- data/lib/ceedling/test_invoker/test_source_file_directive_resolver.rb +40 -20
- data/lib/ceedling/test_invoker/test_substitutions.rb +72 -0
- data/lib/ceedling/tools/shell_quoting.rb +45 -0
- data/lib/ceedling/{tool_executor.rb → tools/tool_executor.rb} +9 -3
- data/lib/ceedling/tools/tool_quoting.rb +79 -0
- data/lib/ceedling/{tool_validator.rb → tools/tool_validator.rb} +18 -7
- data/lib/ceedling/yaml_wrapper.rb +3 -2
- data/lib/version.rb +1 -1
- data/plugins/bullseye/bullseye.rake +1 -3
- data/plugins/bullseye/config/defaults_bullseye.rb +1 -1
- data/plugins/bullseye/lib/bullseye.rb +24 -9
- data/plugins/command_hooks/lib/command_hooks.rb +1 -1
- data/plugins/cppcheck/config/defaults_cppcheck.rb +1 -1
- data/plugins/dependencies/config/defaults.yml +6 -6
- data/plugins/dependencies/dependencies.rake +4 -1
- data/plugins/dependencies/lib/dependencies.rb +18 -8
- data/plugins/dependencies/lib/dependency_spec.rb +17 -5
- data/plugins/fff/lib/fff.rb +79 -46
- data/plugins/fff/lib/fff_mock_generator.rb +12 -9
- data/plugins/fff/spec/fff_cmock_wrapper_spec.rb +262 -0
- data/plugins/fff/spec/fff_mock_header_generator_spec.rb +9 -0
- data/plugins/fff/spec/fff_mock_source_generator_spec.rb +20 -0
- data/plugins/gcov/config/defaults_gcov.rb +1 -1
- data/plugins/gcov/lib/console_reportinator.rb +28 -36
- data/plugins/gcov/lib/gcov.rb +9 -2
- data/plugins/gcov/lib/gcovr_reportinator.rb +8 -40
- data/plugins/gcov/lib/gcovr_thresholds.rb +74 -0
- data/plugins/gcov/lib/reportgenerator_reportinator.rb +3 -2
- data/plugins/module_generator/lib/module_generator.rb +3 -3
- data/plugins/report_tests_teamcity_stdout/lib/report_tests_teamcity_stdout.rb +53 -80
- data/plugins/sanitize/lib/sanitize_records.rb +3 -6
- data/plugins/valgrind/lib/valgrind.rb +14 -7
- data/site-local/404.html +2 -2
- data/site-local/assets/javascripts/{bundle.c21346df.min.js → bundle.a5f9d56c.min.js} +1 -1
- data/site-local/assets/javascripts/{bundle.c21346df.min.js.map → bundle.a5f9d56c.min.js.map} +2 -2
- data/site-local/configuration/environment-vars.html +2 -2
- data/site-local/configuration/global-collections.html +3 -3
- data/site-local/configuration/index.html +2 -2
- data/site-local/configuration/loading.html +2 -2
- data/site-local/configuration/mixins.html +2 -2
- data/site-local/configuration/parallel-builds.html +2 -2
- data/site-local/configuration/project-file.html +2 -2
- data/site-local/configuration/reference/cexception.html +2 -2
- data/site-local/configuration/reference/cmock.html +7 -3
- data/site-local/configuration/reference/defines.html +2 -2
- data/site-local/configuration/reference/environment.html +2 -2
- data/site-local/configuration/reference/extension.html +2 -2
- data/site-local/configuration/reference/files.html +65 -12
- data/site-local/configuration/reference/flags.html +2 -2
- data/site-local/configuration/reference/index.html +2 -2
- data/site-local/configuration/reference/libraries.html +23 -5
- data/site-local/configuration/reference/mixins.html +2 -2
- data/site-local/configuration/reference/partials.html +84 -2
- data/site-local/configuration/reference/paths.html +36 -8
- data/site-local/configuration/reference/plugins.html +2 -2
- data/site-local/configuration/reference/project.html +264 -12
- data/site-local/configuration/reference/release-build.html +4 -2
- data/site-local/configuration/reference/test-build.html +2 -2
- data/site-local/configuration/reference/test-runner.html +2 -2
- data/site-local/configuration/reference/tools.html +58 -4
- data/site-local/configuration/reference/unity.html +2 -2
- data/site-local/configuration/which-ceedling.html +379 -19
- data/site-local/development/index.html +2 -2
- data/site-local/development/plugins/configuration.html +2 -2
- data/site-local/development/plugins/index.html +2 -2
- data/site-local/development/plugins/plugin-subclass.html +5 -4
- data/site-local/development/plugins/rake-tasks.html +2 -2
- data/site-local/development/releases.html +7 -3
- data/site-local/development/workflow.html +50 -45
- data/site-local/getting-started/builds.html +4 -2
- data/site-local/getting-started/command-line.html +58 -6
- data/site-local/getting-started/example-projects/cipher-quest.html +2 -2
- data/site-local/getting-started/example-projects/index.html +2 -2
- data/site-local/getting-started/example-projects/temp-sensor.html +2 -2
- data/site-local/getting-started/example-projects/wondrous-forest.html +10 -2
- data/site-local/getting-started/index.html +2 -2
- data/site-local/getting-started/installation.html +2 -2
- data/site-local/getting-started/quick-start.html +2 -2
- data/site-local/help.html +2 -2
- data/site-local/index.html +2 -2
- data/site-local/overview/build-system.html +2 -2
- data/site-local/overview/index.html +2 -2
- data/site-local/overview/test-environments.html +2 -2
- data/site-local/overview/tools-and-frameworks.html +2 -2
- data/site-local/plugins/beep.html +2 -2
- data/site-local/plugins/bullseye/index.html +45 -38
- data/site-local/plugins/bullseye/licensing.html +2 -2
- data/site-local/plugins/command-hooks.html +4 -4
- data/site-local/plugins/compile-commands-json-db.html +2 -2
- data/site-local/plugins/cppcheck.html +2 -2
- data/site-local/plugins/dependencies.html +8 -5
- data/site-local/plugins/fff.html +58 -13
- data/site-local/plugins/gcov/examples.html +2 -2
- data/site-local/plugins/gcov/gcovr.html +23 -14
- data/site-local/plugins/gcov/index.html +2 -2
- data/site-local/plugins/gcov/overview.html +2 -2
- data/site-local/plugins/gcov/reportgenerator.html +18 -14
- data/site-local/plugins/gcov/reporting.html +2 -2
- data/site-local/plugins/gcov/setup.html +4 -3
- data/site-local/plugins/gcov/tool-versions.html +2 -2
- data/site-local/plugins/gcov/troubleshooting.html +2 -2
- data/site-local/plugins/index.html +2 -2
- data/site-local/plugins/module-generator.html +2 -2
- data/site-local/plugins/report-build-warnings-log.html +2 -2
- data/site-local/plugins/report-tests-gtestlike-stdout.html +2 -2
- data/site-local/plugins/report-tests-ide-stdout.html +2 -2
- data/site-local/plugins/report-tests-log-factory.html +2 -2
- data/site-local/plugins/report-tests-pretty-stdout.html +2 -2
- data/site-local/plugins/report-tests-raw-output-log.html +5 -3
- data/site-local/plugins/report-tests-teamcity-stdout.html +8 -5
- data/site-local/plugins/sanitize.html +9 -8
- data/site-local/plugins/valgrind.html +10 -9
- data/site-local/project/index.html +2 -2
- data/site-local/project/release-history.html +2 -2
- data/site-local/project/sbom.html +2 -2
- data/site-local/project/upgrade.html +5 -2
- data/site-local/reference/build-directives.html +2 -2
- data/site-local/reference/command-line.html +11 -4
- data/site-local/reference/environment-vars.html +2 -2
- data/site-local/reference/gcov-plugin.html +2 -2
- data/site-local/reference/global-collections.html +12 -12
- data/site-local/reference/index.html +2 -2
- data/site-local/reference/partials-macros.html +47 -2
- data/site-local/reference/project-configuration.html +2 -2
- data/site-local/sitemap.xml.gz +0 -0
- data/site-local/snapshot/examples/cipher_quest/index.html +2 -2
- data/site-local/snapshot/examples/temp_sensor/index.html +2 -2
- data/site-local/snapshot/examples/wondrous_forest/index.html +72 -15
- data/site-local/snapshot/plugins/bullseye/config/defaults_bullseye.rb +1 -1
- data/site-local/snapshot/plugins/gcov/config/defaults_gcov.rb +1 -1
- data/site-local/testing-guide/build-directives.html +2 -2
- data/site-local/testing-guide/conventions.html +113 -15
- data/site-local/testing-guide/frameworks.html +2 -2
- data/site-local/testing-guide/index.html +2 -2
- data/site-local/testing-guide/partials/configuration.html +31 -4
- data/site-local/testing-guide/partials/conventions.html +509 -8
- data/site-local/testing-guide/partials/directives.html +4 -3
- data/site-local/testing-guide/partials/example.html +2 -2
- data/site-local/testing-guide/partials/index.html +2 -2
- data/site-local/testing-guide/partials/overview.html +30 -12
- data/site-local/testing-guide/partials/variables.html +169 -4
- data/site-local/testing-guide/test-cases.html +2 -2
- data/site-local/testing-guide/test-sample.html +2 -2
- data/site-local/testing-guide/test-suite-anatomy.html +2 -2
- data/spec/integration/actionator_file_operations_spec.rb +91 -1
- data/spec/integration/build_step_doctor_spec.rb +270 -0
- data/spec/integration/c_extractor_composition_spec.rb +254 -4
- data/spec/integration/compile_commands_json_db_spec.rb +1 -1
- data/spec/integration/configurator_validation_spec.rb +1 -1
- data/spec/integration/dependencies/dependency_tracker_cache_hardening_spec.rb +26 -0
- data/spec/integration/dependencies/dependency_tracker_core_spec.rb +22 -0
- data/spec/integration/includes_extraction_spec.rb +27 -14
- data/spec/integration/launcher_handoff_spec.rb +297 -0
- data/spec/integration/mixin_loading_spec.rb +1 -1
- data/spec/integration/module_generator_spec.rb +3 -3
- data/spec/integration/nested_test_artifacts_spec.rb +99 -0
- data/spec/integration/partial_files_preprocessing_spec.rb +128 -0
- data/spec/integration/partials_carry_spec.rb +520 -0
- data/spec/integration/partials_declarations_spec.rb +655 -0
- data/spec/integration/partials_fallback_preprocessing_spec.rb +65 -0
- data/spec/integration/partials_function_pointers_spec.rb +277 -0
- data/spec/integration/partials_header_composition_spec.rb +268 -35
- data/spec/integration/partials_macro_decorators_spec.rb +267 -0
- data/spec/integration/partials_module_resolution_spec.rb +33 -23
- data/spec/integration/partials_symbol_linkage_spec.rb +13 -0
- data/spec/integration/path_intake_scenarios_spec.rb +101 -0
- data/spec/integration/path_length_reporting_spec.rb +64 -0
- data/spec/integration/path_scenarios_spec.rb +399 -0
- data/spec/integration/quote_include_isolator_spec.rb +57 -1
- data/spec/integration/report_build_warnings_log_spec.rb +1 -1
- data/spec/integration/report_tests_log_factory_spec.rb +1 -1
- data/spec/integration/test_build_parallel_barrier_spec.rb +1 -1
- data/spec/integration/test_executable_manifest_spec.rb +1 -1
- data/spec/integration/test_includes_extractinator_spec.rb +69 -3
- data/spec/integration/test_naming_spec.rb +201 -0
- data/spec/integration/test_source_file_checks_spec.rb +168 -0
- data/spec/integration/tool_executor_spec.rb +1 -1
- data/spec/integration/tool_validator_spec.rb +3 -3
- data/spec/integration/upgrade_docs_spec.rb +84 -0
- data/spec/integration/vendor_copy_transient_io_retry_spec.rb +74 -5
- data/spec/integration/win_long_paths_probe_spec.rb +46 -0
- data/spec/integration/win_path_limit_spec.rb +53 -0
- data/spec/integration/windows_registry_query_tool_spec.rb +41 -0
- data/spec/support/file_tree_helper.rb +88 -0
- data/spec/support/integration/configurator_integration_helper.rb +14 -9
- data/spec/support/integration/dependency_tracker_integration_helper.rb +1 -1
- data/spec/support/integration/partial_files_harness.rb +181 -0
- data/spec/support/integration/partials_generation_integration_helper.rb +239 -143
- data/spec/support/integration/path_scenario_harness.rb +175 -0
- data/spec/support/integration/spec_integration_helper.rb +9 -4
- data/spec/support/integration/test_includes_harness.rb +69 -37
- data/spec/support/integration/tool_executor_integration_helper.rb +14 -2
- data/spec/support/spec_helper.rb +2 -0
- data/spec/support/system/spec_system_helper.rb +7 -6
- data/spec/support/system/system_context.rb +21 -3
- data/spec/support/working_dir_tracking.rb +49 -0
- data/spec/system/build_output_mirroring_spec.rb +2 -2
- data/spec/system/bullseye_deployment_spec.rb +2 -0
- data/spec/system/conditional_mock_includes_spec.rb +1 -1
- data/spec/system/delta_builds_spec.rb +163 -23
- data/spec/system/encoding_stress_spec.rb +4 -4
- data/spec/system/example_wondrous_forest_spec.rb +2 -2
- data/spec/system/gcov_deployment_spec.rb +3 -0
- data/spec/system/header_include_path_validation_spec.rb +4 -4
- data/spec/system/header_search_paths_spec.rb +1083 -0
- data/spec/system/launcher_handoff_spec.rb +82 -0
- data/spec/system/mock_include_folders_spec.rb +212 -0
- data/spec/system/mockable_header_reassembly_spec.rb +184 -0
- data/spec/system/parent_directory_reference_spec.rb +5 -3
- data/spec/system/partials_carry_spec.rb +581 -0
- data/spec/system/partials_delta_builds_spec.rb +280 -0
- data/spec/system/partials_function_pointers_spec.rb +106 -0
- data/spec/system/partials_macro_decorators_spec.rb +127 -0
- data/spec/system/partials_module_disambiguation_spec.rb +125 -0
- data/spec/system/path_handling_probes_spec.rb +196 -0
- data/spec/system/pinned_thread_count_spec.rb +4 -4
- data/spec/system/preprocessing_fallback_conditionals_spec.rb +1 -1
- data/spec/system/preprocessing_parameterized_tests_spec.rb +1 -1
- data/spec/system/rake_release_task_path_disambiguation_spec.rb +6 -6
- data/spec/system/rake_test_task_path_disambiguation_spec.rb +85 -0
- data/spec/system/short_build_paths_spec.rb +87 -0
- data/spec/system/sibling_header_collision_spec.rb +30 -6
- data/spec/system/special_path_characters_spec.rb +86 -0
- data/spec/system/support/bullseye_common_test_cases.rb +42 -0
- data/spec/system/support/common_test_cases.rb +23 -21
- data/spec/system/support/gcov_common_test_cases.rb +37 -9
- data/spec/system/support/gcov_partials_test_cases.rb +48 -0
- data/spec/system/support/sanitize_common_test_cases.rb +2 -2
- data/spec/system/support/valgrind_common_test_cases.rb +12 -12
- data/spec/system/test_build_layout_spec.rb +209 -0
- data/spec/system/test_pipeline_stop_points_spec.rb +4 -4
- data/spec/system/test_runner_system_include_spec.rb +1 -1
- data/spec/system/test_source_file_directive_spec.rb +64 -5
- data/spec/units/bin/actionator_spec.rb +14 -17
- data/spec/units/bin/app_cfg_spec.rb +37 -0
- data/spec/units/bin/cli_handler_spec.rb +282 -6
- data/spec/units/bin/cli_helper_spec.rb +116 -45
- data/spec/units/bin/installation_selector_spec.rb +114 -0
- data/spec/units/bin/launchinator_spec.rb +171 -0
- data/spec/units/bin/path_validator_spec.rb +10 -0
- data/spec/units/bin/versionator_spec.rb +38 -6
- data/spec/units/build_log_writer_spec.rb +57 -0
- data/spec/units/c_extractor/c_extractor_code_text_spec.rb +67 -10
- data/spec/units/c_extractor/c_extractor_declarations_spec.rb +501 -4
- data/spec/units/c_extractor/c_extractor_declarator_spec.rb +245 -0
- data/spec/units/c_extractor/c_extractor_defined_type_spec.rb +45 -0
- data/spec/units/c_extractor/c_extractor_definitions_spec.rb +163 -3
- data/spec/units/c_extractor/c_extractor_functions_spec.rb +462 -21
- data/spec/units/c_extractor/c_extractor_preprocessing_spec.rb +39 -0
- data/spec/units/c_extractor/c_extractor_spec.rb +4 -4
- data/spec/units/c_extractor/c_extractor_types_spec.rb +163 -0
- data/spec/units/c_identifier_spec.rb +35 -0
- data/spec/units/config/build_root_policy_spec.rb +61 -0
- data/spec/units/config/configurator_builder_spec.rb +61 -6
- data/spec/units/config/configurator_paths_validator_spec.rb +292 -0
- data/spec/units/config/configurator_setup_spec.rb +80 -68
- data/spec/units/config/configurator_spec.rb +54 -6
- data/spec/units/config/configurator_validator_spec.rb +1 -172
- data/spec/units/dependencies/dependency_cache_store_spec.rb +98 -2
- data/spec/units/dependencies/dependency_debug_tree_spec.rb +1 -1
- data/spec/units/dependencies/dependency_hasher_spec.rb +42 -1
- data/spec/units/dependencies/dependency_path_normalizer_spec.rb +1 -1
- data/spec/units/dependencies/dependency_tracker_spec.rb +28 -1
- data/spec/units/dependencies/dependinator_spec.rb +17 -2
- data/spec/units/diagnostics/build_step_doctor_spec.rb +170 -0
- data/spec/units/diagnostics/duplicate_symbol_specialist_spec.rb +117 -0
- data/spec/units/diagnostics/header_collision_specialist_spec.rb +267 -0
- data/spec/units/diagnostics/include_depth_specialist_spec.rb +137 -0
- data/spec/units/diagnostics/missing_symbol_specialist_spec.rb +154 -0
- data/spec/units/diagnostics/partial_name_specialist_spec.rb +129 -0
- data/spec/units/diagnostics/toolchain_output_spec.rb +86 -0
- data/spec/units/encodinator_spec.rb +3 -4
- data/spec/units/filename_extension_spec.rb +11 -0
- data/spec/units/filesystem/build_roots_spec.rb +131 -0
- data/spec/units/filesystem/build_subpath_spec.rb +235 -0
- data/spec/units/filesystem/destination_collision_spec.rb +30 -0
- data/spec/units/{file_finder_helper_spec.rb → filesystem/file_finder_helper_spec.rb} +126 -1
- data/spec/units/filesystem/file_finder_spec.rb +845 -0
- data/spec/units/{file_path_collection_utils_spec.rb → filesystem/file_path_collection_utils_spec.rb} +181 -1
- data/spec/units/{file_path_utils_spec.rb → filesystem/file_path_utils_spec.rb} +435 -185
- data/spec/units/filesystem/file_system_wrapper_spec.rb +33 -0
- data/spec/units/filesystem/file_tree_spec.rb +511 -0
- data/spec/units/filesystem/file_tree_windows_spec.rb +78 -0
- data/spec/units/{file_wrapper_spec.rb → filesystem/file_wrapper_spec.rb} +286 -18
- data/spec/units/{path_matcher_spec.rb → filesystem/path_matcher_spec.rb} +48 -1
- data/spec/units/{path_mirror_spec.rb → filesystem/path_mirror_spec.rb} +32 -4
- data/spec/units/filesystem/project_paths_spec.rb +147 -0
- data/spec/units/filesystem/test_build_layout_spec.rb +216 -0
- data/spec/units/filesystem/win_long_paths_spec.rb +204 -0
- data/spec/units/filesystem/win_path_prober_spec.rb +158 -0
- data/spec/units/filesystem/working_dir_spec.rb +71 -0
- data/spec/units/generators/generator_helper_spec.rb +0 -60
- data/spec/units/generators/generator_mocks_spec.rb +13 -0
- data/spec/units/generators/generator_partials_spec.rb +656 -55
- data/spec/units/generators/generator_spec.rb +38 -6
- data/spec/units/generators/generator_test_results_backtrace_spec.rb +50 -46
- data/spec/units/generators/generator_test_results_spec.rb +1 -1
- data/spec/units/generators/generator_test_runner_spec.rb +9 -5
- data/spec/units/generators/partial_source_writer_spec.rb +129 -0
- data/spec/units/handoff_spec.rb +205 -0
- data/spec/units/includes/header_search_spec.rb +406 -0
- data/spec/units/includes/include_factory_spec.rb +23 -6
- data/spec/units/includes/include_pathinator_spec.rb +118 -8
- data/spec/units/includes/include_spec.rb +14 -0
- data/spec/units/includes/includes_spec.rb +176 -19
- data/spec/units/installation_spec.rb +228 -0
- data/spec/units/library_resolver_spec.rb +157 -0
- data/spec/units/loginator_spec.rb +31 -0
- data/spec/units/mock_locator_spec.rb +275 -6
- data/spec/units/partials/include_groups_spec.rb +42 -0
- data/spec/units/partials/partializer_config_spec.rb +32 -0
- data/spec/units/partials/partializer_helper_carry_spec.rb +661 -0
- data/spec/units/partials/partializer_helper_interface_extras_spec.rb +396 -0
- data/spec/units/partials/partializer_helper_spec.rb +529 -277
- data/spec/units/partials/partializer_reconciler_spec.rb +661 -0
- data/spec/units/partials/partializer_spec.rb +1579 -270
- data/spec/units/partials/partializer_utils_spec.rb +269 -232
- data/spec/units/partials/partials_rename_spec.rb +160 -0
- data/spec/units/partials_manager_spec.rb +561 -251
- data/spec/units/plugins/bullseye_spec.rb +49 -2
- data/spec/units/plugins/console_reportinator_spec.rb +73 -26
- data/spec/units/plugins/dependencies_spec.rb +33 -7
- data/spec/units/plugins/dependency_spec_spec.rb +17 -0
- data/spec/units/plugins/gcov_spec.rb +28 -3
- data/spec/units/plugins/gcovr_reportinator_spec.rb +19 -47
- data/spec/units/plugins/gcovr_thresholds_spec.rb +95 -0
- data/spec/units/plugins/module_generator_spec.rb +5 -5
- data/spec/units/plugins/report_build_warnings_log_spec.rb +3 -3
- data/spec/units/plugins/report_log_writer_plugin_spec.rb +10 -1
- data/spec/units/plugins/report_tests_raw_output_log_spec.rb +13 -2
- data/spec/units/plugins/report_tests_teamcity_stdout_spec.rb +77 -3
- data/spec/units/plugins/reportgenerator_reportinator_spec.rb +52 -4
- data/spec/units/plugins/sanitize_records_spec.rb +3 -1
- data/spec/units/plugins/sanitize_spec.rb +1 -1
- data/spec/units/plugins/valgrind_spec.rb +23 -8
- data/spec/units/preprocess/preprocessinator_code_finder_spec.rb +73 -10
- data/spec/units/preprocess/preprocessinator_comment_stripper_spec.rb +1 -1
- data/spec/units/preprocess/preprocessinator_file_assembler_spec.rb +125 -37
- data/spec/units/preprocess/preprocessinator_includes_handler_spec.rb +15 -0
- data/spec/units/preprocess/preprocessinator_reconstructor_spec.rb +197 -0
- data/spec/units/preprocess/preprocessinator_spec.rb +153 -95
- data/spec/units/rake_app/rakefile_component_resolver_spec.rb +21 -0
- data/spec/units/release_build_executor_spec.rb +53 -14
- data/spec/units/release_build_planner_spec.rb +15 -3
- data/spec/units/reportinator_spec.rb +5 -5
- data/spec/units/setupinator_spec.rb +51 -0
- data/spec/units/standin_headerinator_spec.rb +55 -9
- data/spec/units/test_build_executor_spec.rb +842 -166
- data/spec/units/test_build_planner_spec.rb +205 -15
- data/spec/units/test_build_setup_spec.rb +179 -30
- data/spec/units/test_context_extractor_spec.rb +36 -2
- data/spec/units/test_derived_search_paths_spec.rb +259 -0
- data/spec/units/test_file_preprocessinator_spec.rb +17 -1
- data/spec/units/test_includes_extractinator_spec.rb +61 -9
- data/spec/units/test_invoker/objects_wiring_spec.rb +24 -4
- data/spec/units/test_invoker/partial_files_preprocessinator_spec.rb +264 -0
- data/spec/units/test_invoker/test_substitutions_spec.rb +122 -0
- data/spec/units/test_invoker_spec.rb +14 -1
- data/spec/units/test_invoker_types_spec.rb +8 -7
- data/spec/units/test_pipeline_manager_spec.rb +54 -13
- data/spec/units/test_runner_generatinator_spec.rb +93 -1
- data/spec/units/test_search_paths_spec.rb +73 -5
- data/spec/units/test_source_file_directive_resolver_spec.rb +38 -7
- data/spec/units/tools/default_tools_quoting_spec.rb +117 -0
- data/spec/units/tools/shell_quoting_spec.rb +76 -0
- data/spec/units/{tool_executor_helper_spec.rb → tools/tool_executor_helper_spec.rb} +1 -1
- data/spec/units/{tool_executor_spec.rb → tools/tool_executor_spec.rb} +27 -1
- data/spec/units/{tool_validator_spec.rb → tools/tool_validator_spec.rb} +110 -1
- data/spec/units/windows_registry_query_tool_spec.rb +53 -0
- data/spec/units/yaml_wrapper_spec.rb +11 -0
- metadata +265 -36
- data/lib/ceedling/file_finder_helper.rb +0 -127
- data/lib/ceedling/file_path_collection_utils.rb +0 -164
- data/spec/units/file_finder_spec.rb +0 -495
- /data/lib/ceedling/{tool_executor_helper.rb → tools/tool_executor_helper.rb} +0 -0
|
@@ -0,0 +1,253 @@
|
|
|
1
|
+
# File Trees
|
|
2
|
+
|
|
3
|
+
Ceedling copies trees of files whose types it must not misjudge. `FileTree` walks and copies those trees without `lstat`. It lists each directory with `Dir.children` and classifies each entry with `File.stat`. It brings a copy up to date file by file, writes each file whole, and replaces any entry of the wrong type. This is best effort. It removes the dependence on `lstat`, and trusts `stat` in its place, which no filesystem guarantees.
|
|
4
|
+
|
|
5
|
+
This document is for developers and coding agents changing code that walks or copies trees. It begins with orientation and ends with detail. The orientation explains the problem, where Ceedling copies trees, the rules the code follows, and the invariants a change must keep. Later sections trace each rule to its methods, then cover limits, platforms, and testing.
|
|
6
|
+
|
|
7
|
+
Statements of fact carry an evidence label where the evidence is not the code itself.
|
|
8
|
+
|
|
9
|
+
| Label | Meaning |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| **[verified]** | Run and observed, as the testing section describes |
|
|
12
|
+
| **[sourced]** | Taken from vendor documentation, not reproduced here |
|
|
13
|
+
| **[inferred]** | Reasoned from sources, not confirmed |
|
|
14
|
+
| **[unverified]** | A hypothesis awaiting a run on the platform in question |
|
|
15
|
+
|
|
16
|
+
## Orientation
|
|
17
|
+
|
|
18
|
+
### The problem
|
|
19
|
+
|
|
20
|
+
`stat` and `lstat` both report a path's type. `stat` follows a link and reports its target. `lstat` reports the link itself. Code that must not follow links, such as a recursive copy or a recursive delete, classifies entries with `lstat`. Ruby's `FileUtils.cp_r` and `FileUtils.rm_rf` do.
|
|
21
|
+
|
|
22
|
+
On some filesystems `lstat` reports a regular file as a directory while `stat` reports it correctly. Windows reparse points are the known case. A reparse point is a file or directory carrying a tag that a filesystem filter interprets. Some tags make an entry a link to another name, such as a symbolic link or a junction. These are name surrogates. Others mark a transparent storage optimization, such as Data Deduplication (`0x80000013`) or Windows Overlay Filter compression (`0x80000017`). These are not links, and reading such a file returns its contents as usual **[sourced]**. Ruby on Windows can report a non-link reparse point as a directory from `lstat` **[inferred]**. A file on a deduplicated network share has been observed to do so, `stat` reporting it as a file **[sourced]**.
|
|
23
|
+
|
|
24
|
+
A copy that classifies by `lstat` then makes a directory where the file belongs, and fails reading the file as a directory **[verified]**. The stray directory remains, and every later copy collides with it. Retrying cannot help, because the misreport and the leftover both persist.
|
|
25
|
+
|
|
26
|
+
A glob is no safer a walk. `Dir.glob( '**/*' )` lists a linked directory without descending into it **[verified]**. It reads `[`, `{`, `*`, and `?` in a pattern as syntax. How it decides an entry's type on Windows is unknown **[unverified]**.
|
|
27
|
+
|
|
28
|
+
### Where Ceedling copies trees
|
|
29
|
+
|
|
30
|
+
| Copy | Path through the code | When |
|
|
31
|
+
| --- | --- | --- |
|
|
32
|
+
| Unity, CMock, and CException into `build/vendor/` | `ConfiguratorSetup#vendor_frameworks_and_support_files` → `FileWrapper#sync_tree` → `FileTree.sync` | Before build tasks, for a test or release build only |
|
|
33
|
+
| `ceedling.h` and the gdb backtrace script | `ConfiguratorSetup#vendor_test_support_files` → `FileWrapper#sync_file` → `FileTree.sync_file` | Before test build tasks, when Partials or gdb backtraces are on |
|
|
34
|
+
| Ceedling into a project's `vendor/ceedling/`, examples, and docs | `Actionator#copy_directory` → `FileWrapper#tree_files` → `FileTree.each_file` | `ceedling new --local`, `upgrade`, `example`, and `--docs` |
|
|
35
|
+
|
|
36
|
+
### The rules
|
|
37
|
+
|
|
38
|
+
1. **Classify by `stat`.** Every source entry is classified with `File.stat`. A destination entry is checked with `File.directory?`, `File.file?`, and `File.exist?`, which follow links as `stat` does.
|
|
39
|
+
2. **List with `Dir.children`.** A directory's names are read as names, never as a pattern, and sorted.
|
|
40
|
+
3. **Follow links.** A link's target is copied, so a copy holds only regular files and directories. A link loop, a dangling link, a special file such as a socket, or an entry that cannot be read or listed raises a `CeedlingException` naming it.
|
|
41
|
+
4. **Write whole.** A file is written to a temporary name beside its destination and renamed over it. A reader sees the old file or the new one, never part of either.
|
|
42
|
+
5. **Write only what differs.** A destination file matching its source in size and content is left alone, and keeps its modification time.
|
|
43
|
+
6. **Replace the wrong type.** A directory where a file belongs, or a file or dangling link where a directory belongs, is removed and logged before the copy goes on.
|
|
44
|
+
7. **Retry only what clears.** A file's whole sync, from its type check to its write, is retried after each of a series of delays when it fails with `EACCES`, `EBUSY`, `EPERM`, or `EINTR`. Any other error fails at once.
|
|
45
|
+
|
|
46
|
+
### Code organization
|
|
47
|
+
|
|
48
|
+
| File | Module or class | Role |
|
|
49
|
+
| --- | --- | --- |
|
|
50
|
+
| `file_tree.rb` | `FileTree` | The walk, the sync, healing, and retries. A pure module, used by `lib/` and `bin/` alike |
|
|
51
|
+
| `file_wrapper.rb` | `FileWrapper` | `#sync_tree`, `#sync_file`, and `#tree_files` give `FileTree` Ceedling's publish step, retry delays, and logging |
|
|
52
|
+
| `../config/configurator_setup.rb` | `ConfiguratorSetup` | Choosing which frameworks a build vendors, and when |
|
|
53
|
+
| `../../../bin/actionator.rb` | `Actionator` | Copying trees for the command line, with its own status lines and skip rules |
|
|
54
|
+
|
|
55
|
+
### Invariants
|
|
56
|
+
|
|
57
|
+
Every change to this code must keep these invariants. Specs pin each one.
|
|
58
|
+
|
|
59
|
+
- No walk classifies by `lstat`. A copy uses it only on the destination: `FileUtils.rm_r` on a directory with contents, and `File.symlink?` to find a link.
|
|
60
|
+
- A copy holds only regular files and directories.
|
|
61
|
+
- A file that matches its source is never written.
|
|
62
|
+
- A destination file is never visible half-written.
|
|
63
|
+
- A walk ends. A link loop raises rather than recursing.
|
|
64
|
+
- An error that retrying cannot clear surfaces at once.
|
|
65
|
+
|
|
66
|
+
## Walking: `FileTree.each_file` and `.each_entry`
|
|
67
|
+
|
|
68
|
+
`each_entry( root, skip: )` yields each directory and regular file below `root`, a directory ahead of its contents, as its path relative to `root`, its full path, and whether it is a directory. `each_file( root, skip: )` yields only the files, as relative and full paths. Without a block, it returns an Enumerator. `skip`, when given, is a callable given each entry's name. An entry it accepts is neither yielded nor entered, nor even stat'd.
|
|
69
|
+
|
|
70
|
+
Each directory's names come from `Dir.children` and are sorted. A directory that cannot be listed raises "could not be listed". The walk stats each entry. A directory is descended, a regular file is yielded, and anything else raises a `CeedlingException` naming the entry and its type. An entry `stat` cannot read raises "could not be read".
|
|
71
|
+
|
|
72
|
+
`ENOENT` or `ELOOP` from `stat` means a link points nowhere, and raises "is a dangling or unresolvable link". Before descending, the walk resolves the directory with `File.realpath` and compares it to the real paths of the directories above it. A match raises "Link loop". A directory whose real path cannot be resolved stands for itself, so the walk goes on without that safeguard rather than stopping. A root that is not a directory raises at once.
|
|
73
|
+
|
|
74
|
+
## Copying: `FileTree.sync` and `.sync_file`
|
|
75
|
+
|
|
76
|
+
`sync( source, destination, publish:, retry_delays:, sleep:, log: )` brings every file below `source` up to date below `destination`. It returns a `Result` counting files `copied`, `unchanged`, and `healed`. `sync_file( source, destination_dir, … )` does the same for one regular file.
|
|
77
|
+
|
|
78
|
+
`publish` is a callable given a source and a destination path. It does the writing, so the caller decides how a file is written and checked. `retry_delays` lists seconds to wait between attempts. `sleep` and `log` are seams for specs and for Ceedling's logging. `log` receives a message, a `Verbosity` level, and a `LogLabels` value.
|
|
79
|
+
|
|
80
|
+
### Directories
|
|
81
|
+
|
|
82
|
+
A destination directory that exists is kept. A file or a dangling link in its place is removed, counted as healed, and logged. Then `FileUtils.mkdir_p` makes the directory, so empty source directories appear in the copy too.
|
|
83
|
+
|
|
84
|
+
### Files
|
|
85
|
+
|
|
86
|
+
A destination directory in a file's place is removed. A link to a directory is unlinked, `Dir.rmdir` removes an empty directory, and `FileUtils.rm_r` one with contents. Any error removing it reaches the retries. Another run may heal the same entry first. `ENOENT` means it removed the directory, and `ENOTDIR` that it already published the file there. This run then counts no heal, and publishes over that file rather than deleting it, so a reader always finds one. A destination file of the same size and the same bytes, by `FileUtils.compare_file`, is counted unchanged and left alone. Otherwise `publish` writes the file.
|
|
87
|
+
|
|
88
|
+
### Retries
|
|
89
|
+
|
|
90
|
+
A file's sync raising one of `TRANSIENT_ERRORS` at any step waits the next delay and tries the file again. A scanner or sync client can hold the destination open as well as the file being written. Once the delays are spent, the failure is logged and raised as a `CeedlingException` naming the file and the number of retries. Any other error propagates at once, unchanged. Its message names the path.
|
|
91
|
+
|
|
92
|
+
### Log messages
|
|
93
|
+
|
|
94
|
+
| Event | Verbosity, label | Message |
|
|
95
|
+
| --- | --- | --- |
|
|
96
|
+
| A directory replaced by a file | COMPLAIN, NOTICE | `Replaced a directory where the file '<path>' belongs` |
|
|
97
|
+
| A file replaced by a directory | COMPLAIN, NOTICE | `Replaced a file where the directory '<path>' belongs` |
|
|
98
|
+
| A retry | DEBUG | `Retrying copy to '<path>' after a transient error: <message>` |
|
|
99
|
+
| Retries spent | ERRORS | `Failed to copy to '<path>' after <n> retries ⏩️ <message>`, then advice to delete `build/` or reinstall the gem |
|
|
100
|
+
|
|
101
|
+
## FileWrapper's part
|
|
102
|
+
|
|
103
|
+
`#sync_tree` and `#sync_file` pass `FileTree` four things:
|
|
104
|
+
- `#replace_file` as the publish step;
|
|
105
|
+
- `TRANSIENT_IO_RETRY_DELAYS`, four delays totalling 0.37 s;
|
|
106
|
+
- `sleep`;
|
|
107
|
+
- the Loginator.
|
|
108
|
+
|
|
109
|
+
`#replace_file` copies the source to `.<name>.<pid>.<thread id>.tmp` beside the destination and renames it into place. It checks that temporary path's length against the platform's limit first. It makes the copy owner-writable, and a read-only destination file too, since Windows refuses to rename over a read-only file. A destination link is replaced, and the file it names keeps its mode. Execute bits carry over from the source.
|
|
110
|
+
|
|
111
|
+
`#tree_files( dir, skip: )` returns every regular file below `dir` as a full path, sorted as one list. Sorting the full paths puts `a.c` ahead of `a/b.c`, the same order a sorted glob listing gives.
|
|
112
|
+
|
|
113
|
+
## Actionator's part
|
|
114
|
+
|
|
115
|
+
`#copy_directory` copies each file `FileWrapper#tree_files` lists, in that order. It passes `skip: Actionator::HIDDEN`, so entries named with a leading dot are never listed or entered, and nothing inside a hidden directory can stop a copy. It leaves out the junk files its exclude pattern matches too. It compares, forces, and reports each file as it always does, so its status lines match Thor's.
|
|
116
|
+
|
|
117
|
+
## Limits
|
|
118
|
+
|
|
119
|
+
- **`stat` is trusted.** The design assumes `stat` reports a type correctly wherever `lstat` does not. That is the observation behind it, not a guarantee.
|
|
120
|
+
- **`lstat` remains on the destination.** `FileUtils.rm_r` classifies by `lstat` when it removes a destination directory with contents, and never follows a link out of it. `File.symlink?` asks `lstat` whether a destination entry is a link. Both work on the destination, which Ceedling creates, not on a source a filesystem may misreport.
|
|
121
|
+
- **Special files raise.** A socket, a FIFO, or a device in a source tree stops the copy.
|
|
122
|
+
- **Extra files stay.** A copy never removes destination files its source lacks.
|
|
123
|
+
- **Every file is read.** Comparing a file reads both copies on every build. Vendored trees hold a few hundred small files, so this costs little. Comparing by modification time would cost less, but would trust timestamps.
|
|
124
|
+
- **Timestamps.** A written file takes the time of writing. A skipped file keeps its own. Source times are not preserved, and nothing in Ceedling depends on them.
|
|
125
|
+
- **Concurrent builds.** Each file is renamed into place whole, so two builds sharing one `build/vendor/` never read a partial file. A heal never deletes a file another build already published, and publishes over it instead. A spec simulates that race. No spec runs two processes.
|
|
126
|
+
|
|
127
|
+
## Platforms
|
|
128
|
+
|
|
129
|
+
- **Windows.**
|
|
130
|
+
- Reparse points are the case this design exists for.
|
|
131
|
+
- A junction is a link to a directory. Ruby reports it as one, `File.symlink?` true and `lstat` a link, and the walk descends it **[verified]**. Windows makes junctions only on local NTFS volumes, never on a UNC share **[verified]**.
|
|
132
|
+
- `File.rename` replaces an existing file. Windows refuses to replace a read-only one, though Ruby may clear the attribute itself **[inferred]**. `#replace_file` clears it first either way.
|
|
133
|
+
- Names cannot hold `*` or `?`.
|
|
134
|
+
- Paths stop near 260 characters. The temporary name adds about 20 characters beyond the destination's, and `check_path_length` warns as a path nears the limit.
|
|
135
|
+
- **macOS and Linux.**
|
|
136
|
+
- Symlinks are followed.
|
|
137
|
+
- `rename` over a read-only file succeeds, since a directory's permissions govern it.
|
|
138
|
+
- On Linux, `Dir.glob` makes no stat-family calls of its own, reading types from the directory instead **[verified]**.
|
|
139
|
+
- **FAT filesystems.**
|
|
140
|
+
- FAT has no links, refuses `*` and `?` in names, and keeps time to 2 s.
|
|
141
|
+
- The specs skip what FAT cannot hold.
|
|
142
|
+
|
|
143
|
+
## Testing and coverage
|
|
144
|
+
|
|
145
|
+
### Approach
|
|
146
|
+
|
|
147
|
+
The specs test the shape of the problem rather than the infrastructure that produces it. Real network shares and real Data Deduplication are too heavy for CI, and unnecessary once the failure is expressed as `lstat` misreporting a type.
|
|
148
|
+
|
|
149
|
+
### The stand-in
|
|
150
|
+
|
|
151
|
+
`spec/support/file_tree_helper.rb` defines a `LyingStat`, a stat result that claims a directory. `lie_about_lstat( pattern )` stubs `File.lstat` through RSpec to return one for each path matching `pattern`. `FileUtils` classifies entries by calling `File.lstat` from Ruby, so the stub reaches it **[verified]**. RSpec removes the stub after each example. A global patch of `File` would leak between examples.
|
|
152
|
+
|
|
153
|
+
Three kinds of example stub `File.lstat`. The control and the survival examples use the stand-in, and the guard a stricter stub.
|
|
154
|
+
|
|
155
|
+
1. **The control.** `FileUtils.cp_r` fails under the stand-in, making a directory where the file belongs **[verified]**. This shows the stand-in reproduces the failure. If a later Ruby stops failing here, review the control rather than deleting it.
|
|
156
|
+
2. **Survival.** `FileTree.sync` copies every file as a file while every entry lies, and `FileTree.each_file` lists every file **[verified]**.
|
|
157
|
+
3. **The guard.** With `File.lstat` stubbed to raise on any call, a sync that heals an empty stray directory still succeeds **[verified]**.
|
|
158
|
+
|
|
159
|
+
The stand-in proves `FileTree` never classifies by `lstat`. It does not prove a real filesystem behaves as the stand-in does, or that `stat` is right everywhere. It exercises the symptom, not the cause.
|
|
160
|
+
|
|
161
|
+
### Real trees
|
|
162
|
+
|
|
163
|
+
`spec/units/filesystem/file_tree_spec.rb` builds real trees in a temporary directory and covers:
|
|
164
|
+
- nested trees, dotfiles, and empty directories;
|
|
165
|
+
- names holding glob metacharacters;
|
|
166
|
+
- an unchanged file keeping its modification time, and a changed file replaced, even at the same size;
|
|
167
|
+
- extra destination files left alone;
|
|
168
|
+
- each kind of healing, the destination root, a dangling link, and a link to a directory included, and a heal another run made first;
|
|
169
|
+
- links: a linked root, a linked file, a linked directory, a loop, and a dangling link;
|
|
170
|
+
- retries on each transient error, on a destination that cannot yet be read, and on each kind of heal, retries spent, and errors that fail at once;
|
|
171
|
+
- entries `skip` names, directories that cannot be listed, entries that cannot be read, and real paths that cannot be resolved;
|
|
172
|
+
- `.sync_file`.
|
|
173
|
+
|
|
174
|
+
**The canary.** One example shows `Dir.glob( '**/*' )` skipping a linked directory's contents. It characterizes Ruby, not `FileTree`. If it ever fails, Ruby changed, and rule 2's reasoning deserves another look.
|
|
175
|
+
|
|
176
|
+
Examples needing links skip themselves where links cannot be made. The glob-name examples write only names the filesystem accepts.
|
|
177
|
+
|
|
178
|
+
Specs elsewhere cover the callers:
|
|
179
|
+
- `spec/units/filesystem/file_wrapper_spec.rb` covers retries through `#sync_tree`, `#replace_file`'s writable bits and linked destinations, and `#tree_files`.
|
|
180
|
+
- `spec/integration/vendor_copy_transient_io_retry_spec.rb` vendors real trees through `ConfiguratorSetup`: a stray directory at any file, a misreporting `lstat`, read-only files, and faults that clear, persist, or cannot clear.
|
|
181
|
+
- `spec/integration/actionator_file_operations_spec.rb` covers sorted status lines, hidden entries and the hidden directories never entered, glob-named directories, linked directories, and a misreporting `lstat`.
|
|
182
|
+
|
|
183
|
+
### Windows reparse points
|
|
184
|
+
|
|
185
|
+
`spec/units/filesystem/file_tree_windows_spec.rb` runs only on Windows.
|
|
186
|
+
- **Junction.** The first example makes a junction with `mklink /J` and copies through it. A canary line prints what `File.symlink?`, `lstat`, and `stat` report for the junction, without asserting any of them. On a UNC share, where Windows cannot make a junction, it skips without trying.
|
|
187
|
+
- **WOF canary.** The second compresses a file with `compact /exe:lzx`, confirms its reparse tag is `0x80000017`, and prints the same canary line for the file. It then asserts the copy is a regular file of full size.
|
|
188
|
+
- Each skips when its runner cannot make the reparse point. The skip names the command that failed, its exit status, and its output. A failed `compact` also reports the state of the WOF driver.
|
|
189
|
+
|
|
190
|
+
On hosted Windows runners:
|
|
191
|
+
- The junction example passes on the `C:` and `D:` drives and on a VHD-backed NTFS volume. Its canary reads `symlink?=true lstat.ftype=link stat.ftype=directory` on each **[verified]**.
|
|
192
|
+
- The WOF example cannot run on a hosted runner, though the WOF driver runs there **[verified]**. It skips on every volume, saying why:
|
|
193
|
+
|
|
194
|
+
| Volume | What happens |
|
|
195
|
+
| --- | --- |
|
|
196
|
+
| `D:`, and a VHD-backed NTFS volume | `compact /exe` fails with "The file system does not support compression" |
|
|
197
|
+
| `C:` | `compact /exe` succeeds, but the file carries no reparse point |
|
|
198
|
+
| The UNC share | `compact /exe` fails with "The request is not supported" |
|
|
199
|
+
|
|
200
|
+
- Whether Ruby misreports a WOF file is therefore unknown **[unverified]**. The example still runs, and its canary still prints, on a Windows machine that can compress a file this way.
|
|
201
|
+
|
|
202
|
+
### Other filesystems
|
|
203
|
+
|
|
204
|
+
Every `FileTree` spec carries `filesystem: true`, so `rspec spec/units/filesystem --tag filesystem` selects them. `CEEDLING_TEST_FS_ROOT` names a directory on another filesystem, and the specs build their trees there instead of the system temporary directory.
|
|
205
|
+
|
|
206
|
+
| Filesystem | What differs | Where it runs | Status |
|
|
207
|
+
| --- | --- | --- | --- |
|
|
208
|
+
| APFS | Case-insensitive by default | macOS test legs, and locally | **[verified]** |
|
|
209
|
+
| ext4 | The Linux baseline | Linux test legs, and locally in Docker | **[verified]** |
|
|
210
|
+
| tmpfs | Memory-backed | The Linux Ruby 3.3 leg, with `CEEDLING_TEST_FS_ROOT=/dev/shm` | **[verified]** in CI and locally in Docker |
|
|
211
|
+
| NTFS | The Windows baseline | Windows test legs | **[verified]** |
|
|
212
|
+
| UNC administrative share | Reaches the local drive through the SMB client | The Filesystem Substrates job | **[verified]**, the junction and WOF examples skipping |
|
|
213
|
+
| System drive `C:` | The Windows system volume | The Filesystem Substrates job | **[verified]**, the WOF example skipping |
|
|
214
|
+
| VHD-backed NTFS volume | A separate volume made with `diskpart` | The Filesystem Substrates job | **[verified]**, the WOF example skipping |
|
|
215
|
+
| Loopback FAT image | No links, restricted names, 2 s timestamps | The Filesystem Substrates job | **[verified]**, the link examples skipping |
|
|
216
|
+
|
|
217
|
+
A failing spec in the Filesystem Substrates job fails CI Gate. A step that cannot set up its filesystem warns, names why, and skips, since that says nothing about Ceedling. Each step runs whatever the others did. The UNC step first enables Windows' evaluation of links on a share that name places on a share.
|
|
218
|
+
|
|
219
|
+
### Proof the specs can fail
|
|
220
|
+
|
|
221
|
+
**Mutations.** `FileTree` was broken on purpose, one change at a time **[verified]**:
|
|
222
|
+
- `File.stat` changed to `File.lstat` in the walk made 7 examples fail;
|
|
223
|
+
- `Dir.children` changed to `Dir.glob` in the walk made 5 fail.
|
|
224
|
+
|
|
225
|
+
**Seed sweeps.** 100 random orderings of the spec file pass on APFS, ext4, and tmpfs **[verified]**.
|
|
226
|
+
|
|
227
|
+
### Coverage
|
|
228
|
+
|
|
229
|
+
| Scenario | Layer | Linux | macOS | Windows |
|
|
230
|
+
| --- | --- | --- | --- | --- |
|
|
231
|
+
| `lstat` misreports a file | Stand-in | yes | yes | yes |
|
|
232
|
+
| No walk or copy calls `File.lstat` | Stand-in guard | yes | yes | yes |
|
|
233
|
+
| Tree shapes, glob names, unchanged and changed files | Real trees | yes | yes | yes |
|
|
234
|
+
| Healing entries of the wrong type | Real trees | yes | yes | yes |
|
|
235
|
+
| Links | Real trees, skipping without links | yes | yes | where permitted |
|
|
236
|
+
| Junctions | Windows real trees, canary | n/a | n/a | yes, except on a UNC share, which cannot hold one |
|
|
237
|
+
| WOF reparse points | Windows real trees, canary | n/a | n/a | not on hosted runners, which cannot make a WOF file |
|
|
238
|
+
| Retries and failures | Injected faults | yes | yes | yes |
|
|
239
|
+
| Other filesystems | `CEEDLING_TEST_FS_ROOT` | tmpfs, FAT | none | UNC, `C:`, VHD |
|
|
240
|
+
|
|
241
|
+
### Not covered
|
|
242
|
+
|
|
243
|
+
- A real network share whose `lstat` answers change from call to call, and real Data Deduplication.
|
|
244
|
+
- Any non-link reparse point on a hosted runner. WOF compression, the one tried, cannot be applied there.
|
|
245
|
+
- Antivirus and cloud-sync software holding files open.
|
|
246
|
+
- Two Ceedling processes copying into one `build/vendor/` at once.
|
|
247
|
+
- Windows paths near 260 characters with the temporary name's suffix.
|
|
248
|
+
- `stat` misreporting a type. The design assumes it does not.
|
|
249
|
+
|
|
250
|
+
## Sources
|
|
251
|
+
|
|
252
|
+
- Microsoft, [Reparse Tags](https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4) (MS-FSCC 2.1.2.1), for tag values and name surrogates.
|
|
253
|
+
- Microsoft, [Data Deduplication overview](https://learn.microsoft.com/en-us/windows-server/storage/data-deduplication/overview), for how optimized files are stored.
|