@brainpilot/skills 0.0.6 → 0.0.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +2 -2
- package/skills/01_Meta-Skills/academic-research-hub/SKILL.md +108 -0
- package/skills/01_Meta-Skills/academic-research-hub/scripts/requirements.txt +17 -0
- package/skills/01_Meta-Skills/academic-research-hub/scripts/research.py +781 -0
- package/skills/01_Meta-Skills/beautiful-log/SKILL.md +64 -0
- package/skills/01_Meta-Skills/beautiful-log/scripts/beautiful_log.py +274 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/SKILL.md +130 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/assets/config.template.yaml +54 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/assets/top5_digest_template.md +5 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/build_top5_digest.py +300 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/common.py +137 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/merge_results.py +106 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/run_pipeline.py +177 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/search_arxiv.py +162 -0
- package/skills/01_Meta-Skills/ethoclaw-daily-paper/scripts/search_pubmed.py +202 -0
- package/skills/01_Meta-Skills/ethoclaw-normalize-tabular/SKILL.md +173 -0
- package/skills/01_Meta-Skills/ethoclaw-normalize-tabular/scripts/normalize_data.py +874 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/SKILL.md +134 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/references/confirmation-prompts.md +31 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/references/output-patterns.md +45 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/scripts/build_markdown_deliverables.py +41 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/scripts/build_research_log.py +84 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/scripts/build_summary_md.py +63 -0
- package/skills/01_Meta-Skills/ethoclaw-pdf-research/scripts/extract_pdf_bundle.py +140 -0
- package/skills/01_Meta-Skills/experiment-controller/SKILL.md +140 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/SKILL.md +366 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/entity_resolution.py +120 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/extraction_prompt_template.txt +19 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/graph_query.py +106 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/hypothesis_cli_reference.py +42 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/new_data_source_template.py +116 -0
- package/skills/01_Meta-Skills/knowledge-graph-builder/scripts/requirements.txt +15 -0
- package/skills/01_Meta-Skills/method-design/SKILL.md +61 -0
- package/skills/01_Meta-Skills/multi-search-engine/SKILL.md +119 -0
- package/skills/01_Meta-Skills/research-idea/SKILL.md +65 -0
- package/skills/05_EEG_ERP/eeg-skill/SKILL.md +197 -0
- package/skills/05_EEG_ERP/meg-skill/SKILL.md +188 -0
- package/skills/05_EEG_ERP/meg-skill/scripts/time_frequency.py +223 -0
- package/skills/05_EEG_ERP/mne-eeg-tool/SKILL.md +165 -0
- package/skills/05_EEG_ERP/mne-eeg-tool/scripts/eeg_pipeline_reference.py +231 -0
- package/skills/05_EEG_ERP/seed-iv-skill/SKILL.md +184 -0
- package/skills/05_EEG_ERP/seed-iv-skill/scripts/classify_seed_iv.py +154 -0
- package/skills/05_EEG_ERP/seed-iv-skill/scripts/extract_seed_iv_features.py +190 -0
- package/skills/05_EEG_ERP/seed-iv-skill/scripts/validate_seed_iv.py +102 -0
- package/skills/05_EEG_ERP/seed-vig-skill/SKILL.md +182 -0
- package/skills/05_EEG_ERP/seed-vig-skill/scripts/classify_seed_vig.py +165 -0
- package/skills/05_EEG_ERP/seed-vig-skill/scripts/extract_seed_vig_features.py +185 -0
- package/skills/05_EEG_ERP/seed-vig-skill/scripts/validate_seed_vig.py +88 -0
- package/skills/06_fMRI_Neuroimaging/abcd-skill/SKILL.md +308 -0
- package/skills/06_fMRI_Neuroimaging/abcd-skill/scripts/abcd_qc_summary.py +449 -0
- package/skills/06_fMRI_Neuroimaging/abcd-skill/scripts/extract_abcd_phenotype.py +292 -0
- package/skills/06_fMRI_Neuroimaging/abcd-skill/scripts/reorganize_abcd.py +387 -0
- package/skills/06_fMRI_Neuroimaging/abide-skill/SKILL.md +302 -0
- package/skills/06_fMRI_Neuroimaging/abide-skill/scripts/abide_qc_summary.py +317 -0
- package/skills/06_fMRI_Neuroimaging/abide-skill/scripts/extract_abide_phenotype.py +267 -0
- package/skills/06_fMRI_Neuroimaging/abide-skill/scripts/reorganize_abide.py +387 -0
- package/skills/06_fMRI_Neuroimaging/adhd200-skill/SKILL.md +244 -0
- package/skills/06_fMRI_Neuroimaging/adhd200-skill/scripts/adhd200_qc_summary.py +98 -0
- package/skills/06_fMRI_Neuroimaging/adhd200-skill/scripts/extract_adhd200_phenotype.py +134 -0
- package/skills/06_fMRI_Neuroimaging/adhd200-skill/scripts/reorganize_adhd200.py +206 -0
- package/skills/06_fMRI_Neuroimaging/adni-skill/SKILL.md +358 -0
- package/skills/06_fMRI_Neuroimaging/adni-skill/scripts/generate_adni_task_files.py +1305 -0
- package/skills/06_fMRI_Neuroimaging/adni-skill/scripts/generate_vqa_from_tasks.py +766 -0
- package/skills/06_fMRI_Neuroimaging/adni-skill/scripts/reorganize_adni.py +491 -0
- package/skills/06_fMRI_Neuroimaging/aibl-skill/SKILL.md +295 -0
- package/skills/06_fMRI_Neuroimaging/aibl-skill/scripts/aibl_qc_summary.py +260 -0
- package/skills/06_fMRI_Neuroimaging/aibl-skill/scripts/extract_aibl_phenotype.py +365 -0
- package/skills/06_fMRI_Neuroimaging/aibl-skill/scripts/reorganize_aibl.py +394 -0
- package/skills/06_fMRI_Neuroimaging/aomic-skill/SKILL.md +292 -0
- package/skills/06_fMRI_Neuroimaging/aomic-skill/scripts/aomic_qc_summary.py +258 -0
- package/skills/06_fMRI_Neuroimaging/aomic-skill/scripts/extract_aomic_phenotype.py +284 -0
- package/skills/06_fMRI_Neuroimaging/aomic-skill/scripts/reorganize_aomic.py +322 -0
- package/skills/06_fMRI_Neuroimaging/asl-skill/SKILL.md +168 -0
- package/skills/06_fMRI_Neuroimaging/asl-skill/scripts/compute_cbf.py +224 -0
- package/skills/06_fMRI_Neuroimaging/bids-organizer/SKILL.md +241 -0
- package/skills/06_fMRI_Neuroimaging/bold5000-skill/SKILL.md +186 -0
- package/skills/06_fMRI_Neuroimaging/bold5000-skill/scripts/bold5000_qc_summary.py +96 -0
- package/skills/06_fMRI_Neuroimaging/bold5000-skill/scripts/extract_bold5000_stimulus.py +125 -0
- package/skills/06_fMRI_Neuroimaging/bold5000-skill/scripts/reorganize_bold5000.py +102 -0
- package/skills/06_fMRI_Neuroimaging/camcan-skill/SKILL.md +213 -0
- package/skills/06_fMRI_Neuroimaging/camcan-skill/scripts/camcan_qc_summary.py +131 -0
- package/skills/06_fMRI_Neuroimaging/camcan-skill/scripts/extract_camcan_phenotype.py +145 -0
- package/skills/06_fMRI_Neuroimaging/camcan-skill/scripts/validate_camcan.py +141 -0
- package/skills/06_fMRI_Neuroimaging/cobre-skill/SKILL.md +201 -0
- package/skills/06_fMRI_Neuroimaging/cobre-skill/scripts/cobre_qc_summary.py +95 -0
- package/skills/06_fMRI_Neuroimaging/cobre-skill/scripts/extract_cobre_phenotype.py +104 -0
- package/skills/06_fMRI_Neuroimaging/cobre-skill/scripts/reorganize_cobre.py +140 -0
- package/skills/06_fMRI_Neuroimaging/conn-tool/SKILL.md +180 -0
- package/skills/06_fMRI_Neuroimaging/dcm2nii/SKILL.md +189 -0
- package/skills/06_fMRI_Neuroimaging/dmt-har-med-skill/SKILL.md +183 -0
- package/skills/06_fMRI_Neuroimaging/dmt-har-med-skill/scripts/dmt_har_med_qc_summary.py +96 -0
- package/skills/06_fMRI_Neuroimaging/dmt-har-med-skill/scripts/extract_dmt_har_med_phenotype.py +121 -0
- package/skills/06_fMRI_Neuroimaging/dmt-har-med-skill/scripts/reorganize_dmt_har_med.py +125 -0
- package/skills/06_fMRI_Neuroimaging/dwi-skill/SKILL.md +359 -0
- package/skills/06_fMRI_Neuroimaging/fmri-skill/SKILL.md +371 -0
- package/skills/06_fMRI_Neuroimaging/fmriprep-tool/SKILL.md +228 -0
- package/skills/06_fMRI_Neuroimaging/freesurfer-tool/SKILL.md +286 -0
- package/skills/06_fMRI_Neuroimaging/freesurfer-tool/scripts/freesurfer_processor.py +145 -0
- package/skills/06_fMRI_Neuroimaging/fsl-tool/SKILL.md +208 -0
- package/skills/06_fMRI_Neuroimaging/hbn-skill/SKILL.md +271 -0
- package/skills/06_fMRI_Neuroimaging/hbn-skill/scripts/extract_hbn_phenotype.py +107 -0
- package/skills/06_fMRI_Neuroimaging/hbn-skill/scripts/hbn_qc_summary.py +96 -0
- package/skills/06_fMRI_Neuroimaging/hbn-skill/scripts/reorganize_hbn.py +150 -0
- package/skills/06_fMRI_Neuroimaging/hcpa-skill/SKILL.md +210 -0
- package/skills/06_fMRI_Neuroimaging/hcpa-skill/scripts/extract_hcpa_phenotype.py +146 -0
- package/skills/06_fMRI_Neuroimaging/hcpa-skill/scripts/hcpa_qc_summary.py +120 -0
- package/skills/06_fMRI_Neuroimaging/hcpa-skill/scripts/reorganize_hcpa.py +155 -0
- package/skills/06_fMRI_Neuroimaging/hcpd-skill/SKILL.md +210 -0
- package/skills/06_fMRI_Neuroimaging/hcpd-skill/scripts/extract_hcpd_phenotype.py +148 -0
- package/skills/06_fMRI_Neuroimaging/hcpd-skill/scripts/hcpd_qc_summary.py +125 -0
- package/skills/06_fMRI_Neuroimaging/hcpd-skill/scripts/reorganize_hcpd.py +146 -0
- package/skills/06_fMRI_Neuroimaging/hcpep-skill/SKILL.md +215 -0
- package/skills/06_fMRI_Neuroimaging/hcpep-skill/scripts/extract_hcpep_phenotype.py +157 -0
- package/skills/06_fMRI_Neuroimaging/hcpep-skill/scripts/hcpep_qc_summary.py +143 -0
- package/skills/06_fMRI_Neuroimaging/hcpep-skill/scripts/reorganize_hcpep.py +146 -0
- package/skills/06_fMRI_Neuroimaging/hcppipeline-tool/SKILL.md +217 -0
- package/skills/06_fMRI_Neuroimaging/hcpya-skill/SKILL.md +214 -0
- package/skills/06_fMRI_Neuroimaging/hcpya-skill/scripts/extract_hcpya_phenotype.py +190 -0
- package/skills/06_fMRI_Neuroimaging/hcpya-skill/scripts/hcpya_qc_summary.py +152 -0
- package/skills/06_fMRI_Neuroimaging/hcpya-skill/scripts/reorganize_hcpya.py +203 -0
- package/skills/06_fMRI_Neuroimaging/ixi-skill/SKILL.md +198 -0
- package/skills/06_fMRI_Neuroimaging/ixi-skill/scripts/ixi_qc_summary.py +137 -0
- package/skills/06_fMRI_Neuroimaging/ixi-skill/scripts/reorganize_ixi.py +190 -0
- package/skills/06_fMRI_Neuroimaging/mnd-skill/SKILL.md +191 -0
- package/skills/06_fMRI_Neuroimaging/mnd-skill/scripts/extract_mnd_phenotype.py +143 -0
- package/skills/06_fMRI_Neuroimaging/mnd-skill/scripts/mnd_qc_summary.py +120 -0
- package/skills/06_fMRI_Neuroimaging/mnd-skill/scripts/validate_mnd.py +107 -0
- package/skills/06_fMRI_Neuroimaging/mschallenge-skill/SKILL.md +203 -0
- package/skills/06_fMRI_Neuroimaging/mschallenge-skill/scripts/analyze_lesions.py +119 -0
- package/skills/06_fMRI_Neuroimaging/mschallenge-skill/scripts/longitudinal_lesion.py +148 -0
- package/skills/06_fMRI_Neuroimaging/mschallenge-skill/scripts/mschallenge_qc_summary.py +132 -0
- package/skills/06_fMRI_Neuroimaging/mschallenge-skill/scripts/validate_mschallenge.py +116 -0
- package/skills/06_fMRI_Neuroimaging/nibabel-skill/SKILL.md +184 -0
- package/skills/06_fMRI_Neuroimaging/nibabel-skill/scripts/atlas_coordinate_reference.py +61 -0
- package/skills/06_fMRI_Neuroimaging/nibabel-skill/scripts/freesurfer_io_reference.py +34 -0
- package/skills/06_fMRI_Neuroimaging/nibabel-skill/scripts/nifti_inspection_reference.py +35 -0
- package/skills/06_fMRI_Neuroimaging/nifd-skill/SKILL.md +205 -0
- package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/extract_nifd_phenotype.py +132 -0
- package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/nifd_qc_summary.py +111 -0
- package/skills/06_fMRI_Neuroimaging/nifd-skill/scripts/validate_nifd.py +111 -0
- package/skills/06_fMRI_Neuroimaging/nii2dcm/SKILL.md +143 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/SKILL.md +266 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/connectome_reference.py +65 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/denoise_timeseries_reference.py +58 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/hierarchical_parcellation_reference.py +53 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/kmeans_parcellation_reference.py +53 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/preprocess_bold_reference.py +76 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/rest_dictlearning_reference.py +56 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/rest_ica_reference.py +59 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/second_level_glm_reference.py +58 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/spacenet_classifier_reference.py +59 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/svm_classifier_reference.py +60 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/task_glm_reference.py +63 -0
- package/skills/06_fMRI_Neuroimaging/nilearn-tool/scripts/zalff_summary_reference.py +109 -0
- package/skills/06_fMRI_Neuroimaging/nsd-skill/SKILL.md +210 -0
- package/skills/06_fMRI_Neuroimaging/nsd-skill/scripts/extract_nsd_stimulus.py +171 -0
- package/skills/06_fMRI_Neuroimaging/nsd-skill/scripts/nsd_qc_summary.py +142 -0
- package/skills/06_fMRI_Neuroimaging/nsd-skill/scripts/validate_nsd.py +142 -0
- package/skills/06_fMRI_Neuroimaging/oasis-skill/SKILL.md +205 -0
- package/skills/06_fMRI_Neuroimaging/oasis-skill/scripts/extract_oasis_phenotype.py +126 -0
- package/skills/06_fMRI_Neuroimaging/oasis-skill/scripts/oasis_qc_summary.py +115 -0
- package/skills/06_fMRI_Neuroimaging/oasis-skill/scripts/validate_oasis.py +119 -0
- package/skills/06_fMRI_Neuroimaging/pet-skill/SKILL.md +173 -0
- package/skills/06_fMRI_Neuroimaging/pet-skill/scripts/compute_suvr.py +202 -0
- package/skills/06_fMRI_Neuroimaging/pnc-skill/SKILL.md +206 -0
- package/skills/06_fMRI_Neuroimaging/pnc-skill/scripts/extract_pnc_phenotype.py +136 -0
- package/skills/06_fMRI_Neuroimaging/pnc-skill/scripts/pnc_qc_summary.py +116 -0
- package/skills/06_fMRI_Neuroimaging/pnc-skill/scripts/validate_pnc.py +120 -0
- package/skills/06_fMRI_Neuroimaging/ppmi-skill/SKILL.md +209 -0
- package/skills/06_fMRI_Neuroimaging/ppmi-skill/scripts/extract_ppmi_phenotype.py +138 -0
- package/skills/06_fMRI_Neuroimaging/ppmi-skill/scripts/ppmi_qc_summary.py +111 -0
- package/skills/06_fMRI_Neuroimaging/ppmi-skill/scripts/validate_ppmi.py +117 -0
- package/skills/06_fMRI_Neuroimaging/qsiprep-tool/SKILL.md +320 -0
- package/skills/06_fMRI_Neuroimaging/rest-mneta-mdd-skill/SKILL.md +215 -0
- package/skills/06_fMRI_Neuroimaging/rest-mneta-mdd-skill/scripts/extract_rest_mdd_phenotype.py +132 -0
- package/skills/06_fMRI_Neuroimaging/rest-mneta-mdd-skill/scripts/harmonize_sites.py +152 -0
- package/skills/06_fMRI_Neuroimaging/rest-mneta-mdd-skill/scripts/rest_mdd_qc_summary.py +124 -0
- package/skills/06_fMRI_Neuroimaging/rest-mneta-mdd-skill/scripts/validate_rest_mdd.py +103 -0
- package/skills/06_fMRI_Neuroimaging/smri-skill/SKILL.md +302 -0
- package/skills/06_fMRI_Neuroimaging/tcp-skill/SKILL.md +204 -0
- package/skills/06_fMRI_Neuroimaging/tcp-skill/scripts/extract_tcp_phenotype.py +139 -0
- package/skills/06_fMRI_Neuroimaging/tcp-skill/scripts/tcp_qc_summary.py +111 -0
- package/skills/06_fMRI_Neuroimaging/tcp-skill/scripts/validate_tcp.py +99 -0
- package/skills/06_fMRI_Neuroimaging/ucla-cnp-skill/SKILL.md +217 -0
- package/skills/06_fMRI_Neuroimaging/ucla-cnp-skill/scripts/extract_ucla_cnp_phenotype.py +145 -0
- package/skills/06_fMRI_Neuroimaging/ucla-cnp-skill/scripts/ucla_cnp_qc_summary.py +111 -0
- package/skills/06_fMRI_Neuroimaging/ucla-cnp-skill/scripts/validate_ucla_cnp.py +113 -0
- package/skills/06_fMRI_Neuroimaging/ukb-skill/SKILL.md +310 -0
- package/skills/06_fMRI_Neuroimaging/ukb-skill/scripts/build_ukb_survival.py +210 -0
- package/skills/06_fMRI_Neuroimaging/ukb-skill/scripts/extract_ukb_cases.py +308 -0
- package/skills/06_fMRI_Neuroimaging/ukb-skill/scripts/extract_ukb_phenotype.py +232 -0
- package/skills/06_fMRI_Neuroimaging/ukb-skill/scripts/ukb_qc_summary.py +158 -0
- package/skills/06_fMRI_Neuroimaging/wmh-segmentation/SKILL.md +133 -0
- package/skills/07_Computational_Modeling/detrending/SKILL.md +118 -0
- package/skills/07_Computational_Modeling/dictlearning/SKILL.md +122 -0
- package/skills/07_Computational_Modeling/filtering/SKILL.md +121 -0
- package/skills/07_Computational_Modeling/glm/SKILL.md +153 -0
- package/skills/07_Computational_Modeling/hierarchical/SKILL.md +121 -0
- package/skills/07_Computational_Modeling/ica/SKILL.md +122 -0
- package/skills/07_Computational_Modeling/kmeans/SKILL.md +119 -0
- package/skills/07_Computational_Modeling/run_models/SKILL.md +427 -0
- package/skills/07_Computational_Modeling/spacenet/SKILL.md +122 -0
- package/skills/07_Computational_Modeling/svm/SKILL.md +120 -0
- package/skills/08_Computational_Neuroscience/brain_gnn/SKILL.md +183 -0
- package/skills/08_Computational_Neuroscience/dipy-tool/SKILL.md +239 -0
- package/skills/08_Computational_Neuroscience/dipy-tool/scripts/dti_metrics_reference.py +70 -0
- package/skills/08_Computational_Neuroscience/dipy-tool/scripts/load_and_mask_reference.py +76 -0
- package/skills/08_Computational_Neuroscience/dipy-tool/scripts/roi_stats_reference.py +59 -0
- package/skills/08_Computational_Neuroscience/fm_app/SKILL.md +195 -0
- package/skills/08_Computational_Neuroscience/neurostorm/SKILL.md +151 -0
- package/skills/13_Visualization/brain-visualization/SKILL.md +191 -0
- package/skills/13_Visualization/brain-visualization/scripts/connectome_reference.py +108 -0
- package/skills/13_Visualization/brain-visualization/scripts/freesurfer_ply_reference.py +54 -0
- package/skills/13_Visualization/brain-visualization/scripts/zalff_summary_reference.py +116 -0
- package/skills/13_Visualization/ethoclaw-paper-figure-layout/SKILL.md +78 -0
- package/skills/13_Visualization/ethoclaw-paper-figure-layout/assets/naturecomm_figures.tex +74 -0
- package/skills/13_Visualization/ethoclaw-paper-figure-layout/scripts/layout_results_foldered.py +579 -0
- package/skills/14_Writing/overleaf-skill/SKILL.md +184 -0
- package/skills/14_Writing/overleaf-skill/scripts/install.sh +30 -0
- package/skills/14_Writing/paper-writing/SKILL.md +146 -0
- package/skills/14_Writing/paper-writing/scripts/data_statement_templates.py +164 -0
- package/skills/14_Writing/paper-writing/scripts/figure_templates.py +315 -0
- package/skills/14_Writing/paper-writing/scripts/nature_figure_style.py +214 -0
- package/skills/14_Writing/paper-writing/scripts/section_phrasebank.py +246 -0
- package/skills/16_Animal_Behavior/deeplabcut/SKILL.md +154 -0
- package/skills/16_Animal_Behavior/deeplabcut/references/3d-pose.md +89 -0
- package/skills/16_Animal_Behavior/deeplabcut/references/maDLC.md +123 -0
- package/skills/16_Animal_Behavior/deeplabcut/references/modelzoo.md +98 -0
- package/skills/16_Animal_Behavior/deeplabcut/references/standard-pipeline.md +165 -0
- package/skills/16_Animal_Behavior/deeplabcut/references/utilities.md +146 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/SKILL.md +274 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/report_template_en.html +112 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/report_template_en.md +21 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/cluster-section.md +5 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/heatmap-section.md +5 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/integrated-interpretation.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/overview.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/project-summary.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/radar-section.md +5 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/raw-trajectory.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/sample-check.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/single-subject-section.md +3 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/assets/section_templates/stats-section.md +5 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/epm.md +52 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/fst.md +37 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/nor.md +39 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/oft.md +43 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/tcst.md +45 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/experiment-types/tst.md +36 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/input-types.md +59 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/interpretation-guardrails.md +45 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/metadata-schema.md +57 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/report-sections.md +86 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/references/section-selection-rules.md +169 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/scripts/build_report_manifest.py +27 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/scripts/render_report.py +34 -0
- package/skills/16_Animal_Behavior/ethoclaw-analysis-report/scripts/report_utils.py +1121 -0
- package/skills/16_Animal_Behavior/ethoclaw-animal-grounding/SKILL.md +390 -0
- package/skills/16_Animal_Behavior/ethoclaw-animal-grounding/reference_code.py +98 -0
- package/skills/16_Animal_Behavior/ethoclaw-animal-pose-estimation/SKILL.md +336 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/README.md +21 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/SKILL.md +41 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/batch_kinematic_generator.py +663 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/config.json +19 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/generate_kinematic_parameter.py +401 -0
- package/skills/16_Animal_Behavior/ethoclaw-kinematic-parameter-generator/kinematic_generator.py +265 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-clustermap-generate/SKILL.md +72 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-clustermap-generate/references/config.example.toml +56 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-clustermap-generate/scripts/cluster_all_params.py +232 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-clustermap-generate/scripts/cluster_all_params_from_config.py +236 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-radar-generate/SKILL.md +68 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-radar-generate/references/notes.md +5 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-radar-generate/scripts/plot_h5_radar.py +513 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/SKILL.md +52 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/config.toml +81 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/references/stats-rule.md +18 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/scripts/h5_inspect.py +79 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/scripts/h5_violin_batch.py +624 -0
- package/skills/16_Animal_Behavior/ethoclaw-multiparameter-violin-stats-generate/scripts/h5_violin_stats.py +438 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/SKILL.md +280 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/core_scripts/heatmap_trajectory.py +790 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/core_scripts/heatmap_velocity.py +855 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/reference_data/reference_2d.csv +101 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/reference_data/reference_2d.h5 +0 -0
- package/skills/16_Animal_Behavior/ethoclaw-trajectory-velocity-heatmap-generate/reference_data/reference_data_readme.md +126 -0
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: ethoclaw-pdf-research
|
|
3
|
+
description: "Read local PDF files for analysis, summarization, and research-note generation. Use when the user uploads a PDF or provides a local PDF path and wants the model to inspect reports, papers, whitepapers, slides, or scanned documents by extracting text, rendering page images, and producing a structured summary or research log."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# EthoClaw PDF Research Reader
|
|
7
|
+
|
|
8
|
+
Use scripts first. Prefer a two-path workflow: extract text for speed, inspect rendered page images when formatting is important or the PDF is scan-heavy.
|
|
9
|
+
|
|
10
|
+
## Interaction rule
|
|
11
|
+
|
|
12
|
+
Do not parse a newly received PDF immediately by default.
|
|
13
|
+
|
|
14
|
+
When the user uploads a PDF without a clear output request, first confirm what they want:
|
|
15
|
+
|
|
16
|
+
- short summary
|
|
17
|
+
- detailed analysis
|
|
18
|
+
- research log returned directly in chat
|
|
19
|
+
- markdown file only if explicitly requested
|
|
20
|
+
- page/section/figure-focused reading
|
|
21
|
+
|
|
22
|
+
Only start parsing after the user confirms the desired output.
|
|
23
|
+
|
|
24
|
+
Exception: if the user already asked for a specific output together with the file or path, treat that as confirmation and proceed directly.
|
|
25
|
+
|
|
26
|
+
## Quick start
|
|
27
|
+
|
|
28
|
+
Prepare an analysis bundle:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
python3 scripts/extract_pdf_bundle.py /path/to/file.pdf \
|
|
32
|
+
--output-dir /tmp/pdf_bundle \
|
|
33
|
+
--text-last-page 0 \
|
|
34
|
+
--render-last-page 8
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
If the user explicitly wants markdown files, generate both deliverables:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
python3 scripts/build_markdown_deliverables.py \
|
|
41
|
+
/tmp/pdf_bundle/manifest.json \
|
|
42
|
+
--output-dir /tmp/pdf_bundle/md
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
If the user explicitly wants only a research-log file:
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
python3 scripts/build_research_log.py \
|
|
49
|
+
/tmp/pdf_bundle/manifest.json \
|
|
50
|
+
--output /tmp/pdf_bundle/research-log.md
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
## Workflow
|
|
54
|
+
|
|
55
|
+
1. Resolve the source PDF.
|
|
56
|
+
- If the user uploaded a file, use the local file path provided by the runtime.
|
|
57
|
+
- If the user gave a path, validate that it exists before proceeding.
|
|
58
|
+
2. Check whether the user already specified the output.
|
|
59
|
+
- If yes, proceed directly.
|
|
60
|
+
- If no, ask a short confirmation question before parsing.
|
|
61
|
+
- Read `references/confirmation-prompts.md` for the default phrasing.
|
|
62
|
+
3. After confirmation, run `scripts/extract_pdf_bundle.py`.
|
|
63
|
+
- This produces `manifest.json`, `document.txt`, and rendered PNG page previews.
|
|
64
|
+
4. Read `manifest.json` first.
|
|
65
|
+
- Check page count, extracted-text size, and rendered image paths.
|
|
66
|
+
5. Read `document.txt` for the main pass.
|
|
67
|
+
- Use this for normal text PDFs, reports, and most academic papers.
|
|
68
|
+
6. If the extracted text is sparse, garbled, or loses key layout, inspect the PNG pages directly.
|
|
69
|
+
- Read the first pages for title/abstract/executive summary.
|
|
70
|
+
- Read result-heavy or conclusion-heavy pages when needed.
|
|
71
|
+
7. Produce the requested output.
|
|
72
|
+
- Default: return the summary or research log directly in chat.
|
|
73
|
+
- For a plain summary: give topic, core argument, evidence, conclusions, caveats.
|
|
74
|
+
- For a research log: write the full research log directly in chat unless the user explicitly asked for a file.
|
|
75
|
+
- Only generate markdown files when the user explicitly asks for markdown, wants a reusable local artifact, or needs the result saved for later editing.
|
|
76
|
+
- If markdown files are requested, use `build_markdown_deliverables.py` or `build_research_log.py` as scaffolding helpers, then fill the sections with actual findings.
|
|
77
|
+
|
|
78
|
+
## Script behavior
|
|
79
|
+
|
|
80
|
+
### `scripts/extract_pdf_bundle.py`
|
|
81
|
+
|
|
82
|
+
- Read PDF metadata via `pdfinfo`
|
|
83
|
+
- Extract text with `pdftotext -layout`
|
|
84
|
+
- Render selected pages to PNG via `pdftoppm`
|
|
85
|
+
- Write a reusable bundle for later reading:
|
|
86
|
+
- `manifest.json`
|
|
87
|
+
- `document.txt`
|
|
88
|
+
- `images/page-XXXX.png`
|
|
89
|
+
|
|
90
|
+
Important flags:
|
|
91
|
+
|
|
92
|
+
- `--text-last-page 0`: read all pages as text
|
|
93
|
+
- `--render-last-page N`: render the first N pages for visual inspection
|
|
94
|
+
- `--dpi 144`: default PNG quality; raise it if formulas or small print are hard to read
|
|
95
|
+
- `--clean`: replace an existing output directory
|
|
96
|
+
|
|
97
|
+
### `scripts/build_research_log.py`
|
|
98
|
+
|
|
99
|
+
- Convert `manifest.json` into a markdown note scaffold
|
|
100
|
+
- Keep source paths and reading instructions in the log
|
|
101
|
+
- Use it when the user explicitly wants a research log, reading note, or reusable markdown artifact
|
|
102
|
+
- Do not stop at the empty scaffold; fill the sections with actual findings after reading the PDF bundle
|
|
103
|
+
|
|
104
|
+
### `scripts/build_summary_md.py`
|
|
105
|
+
|
|
106
|
+
- Convert `manifest.json` into a short markdown summary scaffold
|
|
107
|
+
- Use it when the user wants a concise deliverable or when you want a companion file next to the research log
|
|
108
|
+
|
|
109
|
+
### `scripts/build_markdown_deliverables.py`
|
|
110
|
+
|
|
111
|
+
- Create both `summary.md` and `research-log.md` in one step
|
|
112
|
+
- Use it as the default markdown-deliverable generator after the user confirms they want markdown output
|
|
113
|
+
|
|
114
|
+
## Heuristics
|
|
115
|
+
|
|
116
|
+
- Prefer text first for normal reports and papers.
|
|
117
|
+
- Prefer image inspection when the PDF is scanned, multi-column extraction is messy, or tables/figures matter more than raw prose.
|
|
118
|
+
- For very long PDFs, do not read every rendered page by default. Start with:
|
|
119
|
+
- title / abstract / executive-summary pages
|
|
120
|
+
- table-of-contents page if it helps navigation
|
|
121
|
+
- conclusion / discussion pages
|
|
122
|
+
- result figures or tables requested by the user
|
|
123
|
+
- If the user asks for deep extraction of a specific section, rerun the script with a tighter page range rather than rendering the whole file at high DPI.
|
|
124
|
+
|
|
125
|
+
## Notes
|
|
126
|
+
|
|
127
|
+
- The current version relies on local command-line tools already available on the host: `pdfinfo`, `pdftotext`, and `pdftoppm`.
|
|
128
|
+
- This skill does not perform OCR beyond what the local PDF/text tools can recover. For image-only scans, rely on rendered PNG pages and vision-capable analysis.
|
|
129
|
+
- Keep external writes minimal. Most tasks can be completed locally inside the workspace.
|
|
130
|
+
|
|
131
|
+
## References
|
|
132
|
+
|
|
133
|
+
- Read `references/confirmation-prompts.md` when a PDF arrives without a clear requested output.
|
|
134
|
+
- Read `references/output-patterns.md` for compact output shapes and suggested summary formats.
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Confirmation Prompts
|
|
2
|
+
|
|
3
|
+
When a PDF arrives, do not start parsing immediately unless the user explicitly asked for immediate analysis.
|
|
4
|
+
|
|
5
|
+
Use a short confirmation message like this:
|
|
6
|
+
|
|
7
|
+
## Default confirmation
|
|
8
|
+
|
|
9
|
+
I have received this PDF. How would you like me to handle it?
|
|
10
|
+
- Brief summary
|
|
11
|
+
- Detailed analysis
|
|
12
|
+
- Research log (direct reply to you)
|
|
13
|
+
- Focus on certain pages / sections / figures
|
|
14
|
+
- If you want, I can also organize it into a markdown file
|
|
15
|
+
|
|
16
|
+
I will start parsing after you confirm.
|
|
17
|
+
|
|
18
|
+
## If the file looks like a manual / SOP / guide
|
|
19
|
+
|
|
20
|
+
I have received this PDF, it looks like an operation manual/documentation. How would you like me to output it?
|
|
21
|
+
- Quick summary
|
|
22
|
+
- Organize by "usage-process-output-precautions"
|
|
23
|
+
- Direct reply as research log
|
|
24
|
+
- Organize as operation guide for newcomers
|
|
25
|
+
- If you want, I can also generate a markdown file
|
|
26
|
+
|
|
27
|
+
I will parse after you confirm.
|
|
28
|
+
|
|
29
|
+
## If the user already gave a goal
|
|
30
|
+
|
|
31
|
+
If the user says things like "help me summarize", "make a research log", "focus on methods section", that counts as confirmation. Start parsing directly and produce the requested output without asking again.
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Output Patterns
|
|
2
|
+
|
|
3
|
+
Use one of these compact shapes after reading a PDF bundle.
|
|
4
|
+
|
|
5
|
+
## 1. Short summary
|
|
6
|
+
|
|
7
|
+
Use when the user only wants the gist.
|
|
8
|
+
|
|
9
|
+
- Topic:
|
|
10
|
+
- Core claim:
|
|
11
|
+
- Main evidence:
|
|
12
|
+
- Key takeaway:
|
|
13
|
+
- Caveat:
|
|
14
|
+
|
|
15
|
+
## 2. Research log
|
|
16
|
+
|
|
17
|
+
Use when the user wants a reusable note.
|
|
18
|
+
|
|
19
|
+
- Basic metadata: title, source path, page count
|
|
20
|
+
- One-paragraph overview
|
|
21
|
+
- Key points
|
|
22
|
+
- Methods / data / evidence
|
|
23
|
+
- Results / claims
|
|
24
|
+
- Caveats
|
|
25
|
+
- Useful quotes or numbers
|
|
26
|
+
- Follow-up questions
|
|
27
|
+
|
|
28
|
+
## 3. Figure/table-oriented reading
|
|
29
|
+
|
|
30
|
+
Use when layout matters more than prose extraction.
|
|
31
|
+
|
|
32
|
+
- What the figure/table shows
|
|
33
|
+
- Why it matters
|
|
34
|
+
- Any exact numbers visible
|
|
35
|
+
- Any ambiguity caused by resolution or cropping
|
|
36
|
+
|
|
37
|
+
## 4. Comparison note
|
|
38
|
+
|
|
39
|
+
Use when multiple PDFs are involved.
|
|
40
|
+
|
|
41
|
+
- Shared topic
|
|
42
|
+
- Agreement points
|
|
43
|
+
- Disagreement points
|
|
44
|
+
- Evidence quality by document
|
|
45
|
+
- Final synthesis
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
import argparse
|
|
3
|
+
import subprocess
|
|
4
|
+
import sys
|
|
5
|
+
from pathlib import Path
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
SCRIPT_DIR = Path(__file__).resolve().parent
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
def run(cmd):
|
|
12
|
+
subprocess.run(cmd, check=True)
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
def main():
|
|
16
|
+
parser = argparse.ArgumentParser(description="Create markdown deliverable scaffolds from a PDF bundle manifest.")
|
|
17
|
+
parser.add_argument("manifest", help="Path to manifest.json produced by extract_pdf_bundle.py")
|
|
18
|
+
parser.add_argument("--output-dir", required=True, help="Directory for markdown deliverables")
|
|
19
|
+
parser.add_argument("--title", default="", help="Optional title override")
|
|
20
|
+
args = parser.parse_args()
|
|
21
|
+
|
|
22
|
+
manifest = Path(args.manifest).expanduser().resolve()
|
|
23
|
+
outdir = Path(args.output_dir).expanduser().resolve()
|
|
24
|
+
outdir.mkdir(parents=True, exist_ok=True)
|
|
25
|
+
|
|
26
|
+
summary_path = outdir / "summary.md"
|
|
27
|
+
research_log_path = outdir / "research-log.md"
|
|
28
|
+
|
|
29
|
+
common = [str(manifest)]
|
|
30
|
+
if args.title:
|
|
31
|
+
common += ["--title", args.title]
|
|
32
|
+
|
|
33
|
+
run([sys.executable, str(SCRIPT_DIR / "build_summary_md.py"), *common, "--output", str(summary_path)])
|
|
34
|
+
run([sys.executable, str(SCRIPT_DIR / "build_research_log.py"), *common, "--output", str(research_log_path)])
|
|
35
|
+
|
|
36
|
+
print(summary_path)
|
|
37
|
+
print(research_log_path)
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
if __name__ == "__main__":
|
|
41
|
+
main()
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
import argparse
|
|
3
|
+
import json
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
|
|
6
|
+
|
|
7
|
+
def main():
|
|
8
|
+
parser = argparse.ArgumentParser(description="Build a markdown research-log template from an extracted PDF bundle manifest.")
|
|
9
|
+
parser.add_argument("manifest", help="Path to manifest.json produced by extract_pdf_bundle.py")
|
|
10
|
+
parser.add_argument("--title", default="", help="Override document title")
|
|
11
|
+
parser.add_argument("--output", default="", help="Write markdown to this path; otherwise print to stdout")
|
|
12
|
+
args = parser.parse_args()
|
|
13
|
+
|
|
14
|
+
manifest_path = Path(args.manifest).expanduser().resolve()
|
|
15
|
+
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
|
16
|
+
pdfinfo = manifest.get("pdfinfo", {})
|
|
17
|
+
title = args.title or pdfinfo.get("Title") or Path(manifest["source_pdf"]).stem
|
|
18
|
+
|
|
19
|
+
image_files = manifest.get("images", {}).get("files", [])
|
|
20
|
+
image_list = "\n".join(f"- {p}" for p in image_files[:12]) or "- (none)"
|
|
21
|
+
|
|
22
|
+
md = f"""# Research Log: {title}
|
|
23
|
+
|
|
24
|
+
## Source
|
|
25
|
+
- PDF: {manifest['source_pdf']}
|
|
26
|
+
- Total pages: {manifest.get('page_count', 'unknown')}
|
|
27
|
+
- Extracted text: {manifest['text']['path']}
|
|
28
|
+
- Rendered preview pages: {manifest['images']['first_page']}-{manifest['images']['last_page']} @ {manifest['images']['dpi']} DPI
|
|
29
|
+
|
|
30
|
+
## Suggested reading plan for the model
|
|
31
|
+
1. Read `manifest.json` first for metadata.
|
|
32
|
+
2. Read `document.txt` for the raw extracted text.
|
|
33
|
+
3. If the text looks broken, inspect the rendered PNG pages directly.
|
|
34
|
+
4. Summarize the paper/report in the structure below.
|
|
35
|
+
|
|
36
|
+
## Rendered page files
|
|
37
|
+
{image_list}
|
|
38
|
+
|
|
39
|
+
## Summary
|
|
40
|
+
- One-paragraph overview:
|
|
41
|
+
- Document type / topic:
|
|
42
|
+
- Main question or purpose:
|
|
43
|
+
|
|
44
|
+
## Key points
|
|
45
|
+
-
|
|
46
|
+
-
|
|
47
|
+
-
|
|
48
|
+
|
|
49
|
+
## Methods / data / evidence
|
|
50
|
+
-
|
|
51
|
+
-
|
|
52
|
+
-
|
|
53
|
+
|
|
54
|
+
## Results / claims
|
|
55
|
+
-
|
|
56
|
+
-
|
|
57
|
+
-
|
|
58
|
+
|
|
59
|
+
## Caveats
|
|
60
|
+
-
|
|
61
|
+
-
|
|
62
|
+
-
|
|
63
|
+
|
|
64
|
+
## Useful quotes or numbers
|
|
65
|
+
-
|
|
66
|
+
-
|
|
67
|
+
-
|
|
68
|
+
|
|
69
|
+
## My follow-up notes
|
|
70
|
+
-
|
|
71
|
+
-
|
|
72
|
+
-
|
|
73
|
+
"""
|
|
74
|
+
|
|
75
|
+
if args.output:
|
|
76
|
+
out = Path(args.output).expanduser().resolve()
|
|
77
|
+
out.parent.mkdir(parents=True, exist_ok=True)
|
|
78
|
+
out.write_text(md, encoding="utf-8")
|
|
79
|
+
else:
|
|
80
|
+
print(md)
|
|
81
|
+
|
|
82
|
+
|
|
83
|
+
if __name__ == "__main__":
|
|
84
|
+
main()
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
import argparse
|
|
3
|
+
import json
|
|
4
|
+
from pathlib import Path
|
|
5
|
+
|
|
6
|
+
|
|
7
|
+
def main():
|
|
8
|
+
parser = argparse.ArgumentParser(description="Build a markdown summary template from an extracted PDF bundle manifest.")
|
|
9
|
+
parser.add_argument("manifest", help="Path to manifest.json produced by extract_pdf_bundle.py")
|
|
10
|
+
parser.add_argument("--title", default="", help="Override document title")
|
|
11
|
+
parser.add_argument("--output", default="", help="Write markdown to this path; otherwise print to stdout")
|
|
12
|
+
args = parser.parse_args()
|
|
13
|
+
|
|
14
|
+
manifest_path = Path(args.manifest).expanduser().resolve()
|
|
15
|
+
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
|
16
|
+
pdfinfo = manifest.get("pdfinfo", {})
|
|
17
|
+
title = args.title or pdfinfo.get("Title") or Path(manifest["source_pdf"]).stem
|
|
18
|
+
|
|
19
|
+
md = f"""# Summary: {title}
|
|
20
|
+
|
|
21
|
+
## Source
|
|
22
|
+
- PDF: {manifest['source_pdf']}
|
|
23
|
+
- Total pages: {manifest.get('page_count', 'unknown')}
|
|
24
|
+
- Extracted text: {manifest['text']['path']}
|
|
25
|
+
- Preview images: {manifest['images']['dir']}
|
|
26
|
+
|
|
27
|
+
## Executive summary
|
|
28
|
+
- Topic:
|
|
29
|
+
- Purpose:
|
|
30
|
+
- Core message:
|
|
31
|
+
|
|
32
|
+
## Key findings
|
|
33
|
+
-
|
|
34
|
+
-
|
|
35
|
+
-
|
|
36
|
+
|
|
37
|
+
## Evidence / basis
|
|
38
|
+
-
|
|
39
|
+
-
|
|
40
|
+
-
|
|
41
|
+
|
|
42
|
+
## Practical takeaway
|
|
43
|
+
-
|
|
44
|
+
-
|
|
45
|
+
|
|
46
|
+
## Caveats
|
|
47
|
+
-
|
|
48
|
+
-
|
|
49
|
+
|
|
50
|
+
## Suggested next step
|
|
51
|
+
-
|
|
52
|
+
"""
|
|
53
|
+
|
|
54
|
+
if args.output:
|
|
55
|
+
out = Path(args.output).expanduser().resolve()
|
|
56
|
+
out.parent.mkdir(parents=True, exist_ok=True)
|
|
57
|
+
out.write_text(md, encoding="utf-8")
|
|
58
|
+
else:
|
|
59
|
+
print(md)
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
if __name__ == "__main__":
|
|
63
|
+
main()
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
import argparse
|
|
3
|
+
import json
|
|
4
|
+
import os
|
|
5
|
+
import re
|
|
6
|
+
import shutil
|
|
7
|
+
import subprocess
|
|
8
|
+
import sys
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def run(cmd):
|
|
13
|
+
return subprocess.run(cmd, check=True, text=True, capture_output=True)
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+
def parse_pdfinfo(pdf_path: Path):
|
|
17
|
+
result = run(["pdfinfo", str(pdf_path)])
|
|
18
|
+
info = {}
|
|
19
|
+
for line in result.stdout.splitlines():
|
|
20
|
+
if ":" not in line:
|
|
21
|
+
continue
|
|
22
|
+
key, value = line.split(":", 1)
|
|
23
|
+
info[key.strip()] = value.strip()
|
|
24
|
+
pages = int(info.get("Pages", "0") or 0)
|
|
25
|
+
return {"raw": info, "pages": pages}
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def sanitize_text(text: str) -> str:
|
|
29
|
+
text = text.replace("\x0c", "\n")
|
|
30
|
+
text = text.replace("\r", "")
|
|
31
|
+
text = re.sub(r"\n{3,}", "\n\n", text)
|
|
32
|
+
return text.strip() + "\n"
|
|
33
|
+
|
|
34
|
+
|
|
35
|
+
def extract_text(pdf_path: Path, output_path: Path, first_page=None, last_page=None):
|
|
36
|
+
cmd = ["pdftotext", "-enc", "UTF-8", "-layout"]
|
|
37
|
+
if first_page is not None:
|
|
38
|
+
cmd += ["-f", str(first_page)]
|
|
39
|
+
if last_page is not None:
|
|
40
|
+
cmd += ["-l", str(last_page)]
|
|
41
|
+
cmd += [str(pdf_path), "-"]
|
|
42
|
+
result = run(cmd)
|
|
43
|
+
text = sanitize_text(result.stdout)
|
|
44
|
+
output_path.write_text(text, encoding="utf-8")
|
|
45
|
+
return text
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+
def render_pages(pdf_path: Path, output_dir: Path, start_page: int, end_page: int, dpi: int):
|
|
49
|
+
prefix = output_dir / "page"
|
|
50
|
+
cmd = [
|
|
51
|
+
"pdftoppm",
|
|
52
|
+
"-png",
|
|
53
|
+
"-r",
|
|
54
|
+
str(dpi),
|
|
55
|
+
"-f",
|
|
56
|
+
str(start_page),
|
|
57
|
+
"-l",
|
|
58
|
+
str(end_page),
|
|
59
|
+
str(pdf_path),
|
|
60
|
+
str(prefix),
|
|
61
|
+
]
|
|
62
|
+
run(cmd)
|
|
63
|
+
images = sorted(output_dir.glob("page-*.png"))
|
|
64
|
+
renamed = []
|
|
65
|
+
for image in images:
|
|
66
|
+
m = re.search(r"page-(\d+)\.png$", image.name)
|
|
67
|
+
if not m:
|
|
68
|
+
continue
|
|
69
|
+
page_num = int(m.group(1))
|
|
70
|
+
new_path = output_dir / f"page-{page_num:04d}.png"
|
|
71
|
+
image.rename(new_path)
|
|
72
|
+
renamed.append(new_path)
|
|
73
|
+
return renamed
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
def main():
|
|
77
|
+
parser = argparse.ArgumentParser(description="Extract text and rendered pages from a PDF into a reusable analysis bundle.")
|
|
78
|
+
parser.add_argument("pdf", help="Path to the input PDF")
|
|
79
|
+
parser.add_argument("--output-dir", required=True, help="Directory for extracted artifacts")
|
|
80
|
+
parser.add_argument("--text-first-page", type=int, default=1, help="First page for text extraction")
|
|
81
|
+
parser.add_argument("--text-last-page", type=int, default=0, help="Last page for text extraction; 0 means all pages")
|
|
82
|
+
parser.add_argument("--render-first-page", type=int, default=1, help="First page to render as images")
|
|
83
|
+
parser.add_argument("--render-last-page", type=int, default=8, help="Last page to render as images")
|
|
84
|
+
parser.add_argument("--dpi", type=int, default=144, help="PNG render DPI")
|
|
85
|
+
parser.add_argument("--clean", action="store_true", help="Delete output directory before writing")
|
|
86
|
+
args = parser.parse_args()
|
|
87
|
+
|
|
88
|
+
pdf_path = Path(args.pdf).expanduser().resolve()
|
|
89
|
+
output_dir = Path(args.output_dir).expanduser().resolve()
|
|
90
|
+
|
|
91
|
+
if not pdf_path.exists():
|
|
92
|
+
raise SystemExit(f"PDF not found: {pdf_path}")
|
|
93
|
+
|
|
94
|
+
if shutil.which("pdfinfo") is None or shutil.which("pdftotext") is None or shutil.which("pdftoppm") is None:
|
|
95
|
+
raise SystemExit("Missing required commands. Need pdfinfo, pdftotext, and pdftoppm in PATH.")
|
|
96
|
+
|
|
97
|
+
if args.clean and output_dir.exists():
|
|
98
|
+
shutil.rmtree(output_dir)
|
|
99
|
+
output_dir.mkdir(parents=True, exist_ok=True)
|
|
100
|
+
|
|
101
|
+
info = parse_pdfinfo(pdf_path)
|
|
102
|
+
total_pages = info["pages"]
|
|
103
|
+
text_last_page = total_pages if args.text_last_page in (0, None) else min(args.text_last_page, total_pages)
|
|
104
|
+
render_last_page = min(args.render_last_page, total_pages)
|
|
105
|
+
|
|
106
|
+
text_path = output_dir / "document.txt"
|
|
107
|
+
text = extract_text(pdf_path, text_path, args.text_first_page, text_last_page)
|
|
108
|
+
char_count = len(text.strip())
|
|
109
|
+
|
|
110
|
+
images_dir = output_dir / "images"
|
|
111
|
+
images_dir.mkdir(parents=True, exist_ok=True)
|
|
112
|
+
image_paths = render_pages(pdf_path, images_dir, args.render_first_page, render_last_page, args.dpi)
|
|
113
|
+
|
|
114
|
+
manifest = {
|
|
115
|
+
"source_pdf": str(pdf_path),
|
|
116
|
+
"output_dir": str(output_dir),
|
|
117
|
+
"page_count": total_pages,
|
|
118
|
+
"text": {
|
|
119
|
+
"path": str(text_path),
|
|
120
|
+
"first_page": args.text_first_page,
|
|
121
|
+
"last_page": text_last_page,
|
|
122
|
+
"char_count": char_count,
|
|
123
|
+
},
|
|
124
|
+
"images": {
|
|
125
|
+
"dir": str(images_dir),
|
|
126
|
+
"first_page": args.render_first_page,
|
|
127
|
+
"last_page": render_last_page,
|
|
128
|
+
"dpi": args.dpi,
|
|
129
|
+
"files": [str(p) for p in image_paths],
|
|
130
|
+
},
|
|
131
|
+
"pdfinfo": info["raw"],
|
|
132
|
+
}
|
|
133
|
+
manifest_path = output_dir / "manifest.json"
|
|
134
|
+
manifest_path.write_text(json.dumps(manifest, ensure_ascii=False, indent=2), encoding="utf-8")
|
|
135
|
+
|
|
136
|
+
print(json.dumps(manifest, ensure_ascii=False, indent=2))
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
if __name__ == "__main__":
|
|
140
|
+
main()
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: experiment-controller
|
|
3
|
+
description: "Use this skill whenever the user wants to execute experiments based on a finalized method and record results. Triggers include: 'run experiment', 'experiment controller', 'implement experiment', 'run model', 'execute training', 'experiment-controller', 'record results', 'ablation study', or any request to turn METHOD.md into concrete runs and output to EXPERIMENT.md. This skill is the **mandatory interface-layer experiment executor** in NeuroClaw: it searches literature/GitHub for matching experimental setups and codebases, proposes one scheme + repo after user discussion, uses git skills to download and setup, runs the experiment(s), and iteratively appends every result + observation to EXPERIMENT.md."
|
|
4
|
+
license: MIT License (NeuroClaw custom skill – freely modifiable within the project)
|
|
5
|
+
layer: interface
|
|
6
|
+
skill_type: workflow
|
|
7
|
+
dependencies:
|
|
8
|
+
- claw-shell
|
|
9
|
+
- dependency-planner
|
|
10
|
+
- git-essentials
|
|
11
|
+
---
|
|
12
|
+
# Experiment Controller
|
|
13
|
+
## Overview
|
|
14
|
+
This skill implements the **Literature/GitHub Search → Scheme Confirmation → Git Execution → Iterative Logging** process for the NeuroClaw experiment-controller phase.
|
|
15
|
+
|
|
16
|
+
It acts as the Experiment Manager within the multi-agent framework:
|
|
17
|
+
- Reads the latest **IDEA.md** and **METHOD.md** from the workspace.
|
|
18
|
+
- Searches recent literature (multi-search-engine, arxiv-search, pubmed-search) and GitHub for reproducible experimental setups and open-source repositories that match the proposed architecture.
|
|
19
|
+
- Summarizes candidate schemes (hyperparameters, datasets, baselines, training protocols) and proposes the most suitable GitHub repo.
|
|
20
|
+
- Iteratively discusses with the user to confirm the exact scheme/repo.
|
|
21
|
+
- After confirmation: uses git-essentials/git-workflows to clone, dependency-planner to install environment, claw-shell to run the experiment (training/inference/ablation).
|
|
22
|
+
- After every run (or ablation), automatically records: setup details, metrics, logs, observations, and any issues.
|
|
23
|
+
- Saves everything in **EXPERIMENT.md** (with dated sections for each run).
|
|
24
|
+
|
|
25
|
+
**Research use only** — the output is a complete, reproducible EXPERIMENT.md ready for paper-writing and future replication.
|
|
26
|
+
|
|
27
|
+
## Quick Reference (Experiment Flow)
|
|
28
|
+
| Step | Description | Output File |
|
|
29
|
+
|------|------------------------------------------|---------------------------|
|
|
30
|
+
| 1. Read & Parse | Load IDEA.md + METHOD.md | 01_idea_method_summary.md |
|
|
31
|
+
| 2. Literature & GitHub Search | Find matching setups & repos | 02_search_results.md |
|
|
32
|
+
| 3. Proposal | Recommend best scheme + repo | 03_proposal.md |
|
|
33
|
+
| 4. User Discussion | Confirm scheme/repo | 04_discussion.md |
|
|
34
|
+
| 5. Git Clone & Setup | Clone + install dependencies | 05_setup_log.md |
|
|
35
|
+
| 6. Run Experiment | Execute training/inference/ablation | 06_run_log_*.md (per run) |
|
|
36
|
+
| 7. Record Results | Append metrics + observations | EXPERIMENT.md |
|
|
37
|
+
|
|
38
|
+
## Harness Engineering Protocol (Mandatory for Reproducibility)
|
|
39
|
+
|
|
40
|
+
All experiments executed via experiment-controller **must** follow the **Task Decomposition → Agent Initialization → Execution → Verification** protocol to ensure self-validation, auditability, and resumability.
|
|
41
|
+
|
|
42
|
+
### Phase 1: Task Decomposition
|
|
43
|
+
- Parse experimental goal from METHOD.md into discrete tasks (data preprocessing, feature extraction, model training, inference, evaluation)
|
|
44
|
+
- Define success criteria for each task (BIDS format compliance, feature vector shape/range, metric thresholds)
|
|
45
|
+
- Identify cross-task dependencies and data flow
|
|
46
|
+
- Output: `experiment_task_manifest.json` with task graph
|
|
47
|
+
|
|
48
|
+
### Phase 2: Agent Initialization
|
|
49
|
+
- **Pre-flight checks**: verify all dependencies installed, data paths accessible, Docker/GPU resources available (if required)
|
|
50
|
+
- **Environment snapshot**: capture Python version, library versions, hardware specs, random seeds → `environment_manifest.json`
|
|
51
|
+
- **Checkpoint management**: determine checkpoint frequency, rollback strategy, memory constraints
|
|
52
|
+
- **Logging setup**: initialize structured logging with unique experiment session ID (timestamp + hash)
|
|
53
|
+
|
|
54
|
+
### Phase 3: Execution with Incremental Logging
|
|
55
|
+
- Execute each task with **real-time progress tracking**
|
|
56
|
+
- Save checkpoint after each completed task (enables resumption on failure)
|
|
57
|
+
- Log detailed metrics, intermediate outputs, and timing information
|
|
58
|
+
- Generate cryptographic hash (SHA256) for each output artifact
|
|
59
|
+
- Stream results to EXPERIMENT.md in real-time sections with timestamps
|
|
60
|
+
|
|
61
|
+
### Phase 4: Verification and Post-Execution Validation
|
|
62
|
+
- **Self-verification checks** (module-specific):
|
|
63
|
+
- Preprocessing: verify BIDS compliance, check for NaN/Inf values, validate normalization ranges
|
|
64
|
+
- Feature extraction: check output shape consistency, verify statistical properties (mean/std within expected bounds)
|
|
65
|
+
- Model training: validate loss curve smoothness, check for NaN gradients, verify train/val split integrity
|
|
66
|
+
- Inference: cross-check predictions for domain-specific constraints (probability bounds, anatomical plausibility)
|
|
67
|
+
- **Result integrity validation**:
|
|
68
|
+
- Recompute hash of all output files and compare with stored values
|
|
69
|
+
- Flag any mismatches as potential corruption/tampering
|
|
70
|
+
- **Generate final audit report**: `experiment_audit_report.md` with task execution times, success/failure status, verification results, and reproducibility metadata
|
|
71
|
+
|
|
72
|
+
### Self-Verification Implementation Details
|
|
73
|
+
|
|
74
|
+
Each skill integrated into experiment-controller must include **automatic validation steps**:
|
|
75
|
+
|
|
76
|
+
**Preprocessing validation**:
|
|
77
|
+
```
|
|
78
|
+
- BIDS compliance: confirm file naming, JSON sidecars, required fields
|
|
79
|
+
- Data integrity: NaN/Inf count, range of pixel values, histogram sanity check
|
|
80
|
+
- Statistical bounds: mean/std within neuroimaging norms (e.g., T1w intensity ~0-4000 HU)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
**Feature extraction validation**:
|
|
84
|
+
```
|
|
85
|
+
- Output shape check: row/column count match expected dataset size
|
|
86
|
+
- Distribution check: ensure features are not constant or degenerate
|
|
87
|
+
- Correlations: detect and warn on features with >0.95 mutual correlation
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
**Training validation**:
|
|
91
|
+
```
|
|
92
|
+
- Loss curve smoothness: flag sudden spikes or plateau too early
|
|
93
|
+
- Gradient health: ensure no NaN/Inf gradients during backprop
|
|
94
|
+
- Validation metric monotonicity (for early stopping): warn if validation improves inconsistently
|
|
95
|
+
- Train/val split: cross-verify split ratio and no subject leakage
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
**Inference validation**:
|
|
99
|
+
```
|
|
100
|
+
- Output shape consistency: predictions match expected target cardinality
|
|
101
|
+
- Domain constraints: probability predictions in [0,1], regression outputs within physiologically plausible ranges
|
|
102
|
+
- Batch effect check: compare results across batch sizes (should be near-identical with same seed)
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Installation
|
|
106
|
+
```bash
|
|
107
|
+
# Place files in: skills/experiment-controller/
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
## Important Notes & Limitations
|
|
111
|
+
- Always starts from latest IDEA.md + METHOD.md; stops and prompts if missing.
|
|
112
|
+
- Every step and every run is saved as numbered Markdown files for full transparency and resumption.
|
|
113
|
+
- Git clone uses git-essentials/git-workflows (never manual commands outside skills).
|
|
114
|
+
- Dependencies are handled exclusively by dependency-planner.
|
|
115
|
+
- Multiple runs (e.g., ablations) are supported; results are appended with timestamps.
|
|
116
|
+
- Final output always saved/updated as `EXPERIMENT.md` in workspace root.
|
|
117
|
+
- Logs include: command executed, hyperparameters, metrics (accuracy, loss, Dice, etc.), runtime, observations, and any errors.
|
|
118
|
+
|
|
119
|
+
## When to Call This Skill
|
|
120
|
+
- Immediately after method-design completes METHOD.md
|
|
121
|
+
- When the user wants to run, reproduce, or compare experiments
|
|
122
|
+
- Before paper-writing (to populate quantitative results)
|
|
123
|
+
|
|
124
|
+
## Complementary / Related Skills
|
|
125
|
+
- `research-idea` → provides IDEA.md
|
|
126
|
+
- `method-design` → provides METHOD.md (input)
|
|
127
|
+
- `multi-search-engine` → literature & GitHub search
|
|
128
|
+
- `git-essentials` / `git-workflows` → clone and manage repositories
|
|
129
|
+
- `dependency-planner` → environment & dependency installation
|
|
130
|
+
- `claw-shell` → internal execution of training scripts
|
|
131
|
+
- `paper-writing` → consumes EXPERIMENT.md for results/tables
|
|
132
|
+
|
|
133
|
+
## Reference
|
|
134
|
+
NeuroClaw architecture (section 1.7 experiment-controller skill).
|
|
135
|
+
Flow: literature/GitHub search → user-confirmed scheme → git clone → dependency setup → iterative execution → EXPERIMENT.md logging.
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
Created At: 2026-03-24 00:00 HKT
|
|
139
|
+
Last Updated At: 2026-04-05 02:01 HKT
|
|
140
|
+
Author: chengwang96
|