phyddle 0.0.7__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (405) hide show
  1. phyddle-0.0.7/.github/workflows/documentation.yml +24 -0
  2. phyddle-0.0.7/.github/workflows/phyddle-tests.yml +41 -0
  3. phyddle-0.0.7/.gitignore +57 -0
  4. phyddle-0.0.7/LICENSE +21 -0
  5. phyddle-0.0.7/PKG-INFO +132 -0
  6. phyddle-0.0.7/README.md +110 -0
  7. phyddle-0.0.7/TODO.md +108 -0
  8. phyddle-0.0.7/_build/.buildinfo +4 -0
  9. phyddle-0.0.7/_build/.doctrees/about.doctree +0 -0
  10. phyddle-0.0.7/_build/.doctrees/configuration.doctree +0 -0
  11. phyddle-0.0.7/_build/.doctrees/environment.pickle +0 -0
  12. phyddle-0.0.7/_build/.doctrees/formats.doctree +0 -0
  13. phyddle-0.0.7/_build/.doctrees/glossary.doctree +0 -0
  14. phyddle-0.0.7/_build/.doctrees/index.doctree +0 -0
  15. phyddle-0.0.7/_build/.doctrees/installation.doctree +0 -0
  16. phyddle-0.0.7/_build/.doctrees/pipeline.doctree +0 -0
  17. phyddle-0.0.7/_build/.doctrees/quick_start.doctree +0 -0
  18. phyddle-0.0.7/_build/.doctrees/tricks.doctree +0 -0
  19. phyddle-0.0.7/_build/.doctrees/updates.doctree +0 -0
  20. phyddle-0.0.7/_build/.doctrees/workspace.doctree +0 -0
  21. phyddle-0.0.7/_build/_images/phyddle_pipeline.png +0 -0
  22. phyddle-0.0.7/_build/_sources/about.rst.txt +33 -0
  23. phyddle-0.0.7/_build/_sources/configuration.rst.txt +306 -0
  24. phyddle-0.0.7/_build/_sources/formats.rst.txt +247 -0
  25. phyddle-0.0.7/_build/_sources/glossary.rst.txt +16 -0
  26. phyddle-0.0.7/_build/_sources/index.rst.txt +40 -0
  27. phyddle-0.0.7/_build/_sources/installation.rst.txt +168 -0
  28. phyddle-0.0.7/_build/_sources/pipeline.rst.txt +372 -0
  29. phyddle-0.0.7/_build/_sources/quick_start.rst.txt +49 -0
  30. phyddle-0.0.7/_build/_sources/tricks.rst.txt +38 -0
  31. phyddle-0.0.7/_build/_sources/updates.rst.txt +91 -0
  32. phyddle-0.0.7/_build/_sources/workspace.rst.txt +205 -0
  33. phyddle-0.0.7/_build/_static/_sphinx_javascript_frameworks_compat.js +123 -0
  34. phyddle-0.0.7/_build/_static/basic.css +903 -0
  35. phyddle-0.0.7/_build/_static/css/badge_only.css +1 -0
  36. phyddle-0.0.7/_build/_static/css/custom.css +46 -0
  37. phyddle-0.0.7/_build/_static/css/fonts/Roboto-Slab-Bold.woff +0 -0
  38. phyddle-0.0.7/_build/_static/css/fonts/Roboto-Slab-Bold.woff2 +0 -0
  39. phyddle-0.0.7/_build/_static/css/fonts/Roboto-Slab-Regular.woff +0 -0
  40. phyddle-0.0.7/_build/_static/css/fonts/Roboto-Slab-Regular.woff2 +0 -0
  41. phyddle-0.0.7/_build/_static/css/fonts/fontawesome-webfont.eot +0 -0
  42. phyddle-0.0.7/_build/_static/css/fonts/fontawesome-webfont.svg +2671 -0
  43. phyddle-0.0.7/_build/_static/css/fonts/fontawesome-webfont.ttf +0 -0
  44. phyddle-0.0.7/_build/_static/css/fonts/fontawesome-webfont.woff +0 -0
  45. phyddle-0.0.7/_build/_static/css/fonts/fontawesome-webfont.woff2 +0 -0
  46. phyddle-0.0.7/_build/_static/css/fonts/lato-bold-italic.woff +0 -0
  47. phyddle-0.0.7/_build/_static/css/fonts/lato-bold-italic.woff2 +0 -0
  48. phyddle-0.0.7/_build/_static/css/fonts/lato-bold.woff +0 -0
  49. phyddle-0.0.7/_build/_static/css/fonts/lato-bold.woff2 +0 -0
  50. phyddle-0.0.7/_build/_static/css/fonts/lato-normal-italic.woff +0 -0
  51. phyddle-0.0.7/_build/_static/css/fonts/lato-normal-italic.woff2 +0 -0
  52. phyddle-0.0.7/_build/_static/css/fonts/lato-normal.woff +0 -0
  53. phyddle-0.0.7/_build/_static/css/fonts/lato-normal.woff2 +0 -0
  54. phyddle-0.0.7/_build/_static/css/theme.css +4 -0
  55. phyddle-0.0.7/_build/_static/doctools.js +156 -0
  56. phyddle-0.0.7/_build/_static/documentation_options.js +14 -0
  57. phyddle-0.0.7/_build/_static/file.png +0 -0
  58. phyddle-0.0.7/_build/_static/jquery.js +2 -0
  59. phyddle-0.0.7/_build/_static/js/badge_only.js +1 -0
  60. phyddle-0.0.7/_build/_static/js/html5shiv-printshiv.min.js +4 -0
  61. phyddle-0.0.7/_build/_static/js/html5shiv.min.js +4 -0
  62. phyddle-0.0.7/_build/_static/js/theme.js +1 -0
  63. phyddle-0.0.7/_build/_static/language_data.js +199 -0
  64. phyddle-0.0.7/_build/_static/minus.png +0 -0
  65. phyddle-0.0.7/_build/_static/phyddle_logo.png +0 -0
  66. phyddle-0.0.7/_build/_static/plus.png +0 -0
  67. phyddle-0.0.7/_build/_static/pygments.css +74 -0
  68. phyddle-0.0.7/_build/_static/searchtools.js +566 -0
  69. phyddle-0.0.7/_build/_static/sphinx_highlight.js +144 -0
  70. phyddle-0.0.7/_build/about.html +140 -0
  71. phyddle-0.0.7/_build/configuration.html +633 -0
  72. phyddle-0.0.7/_build/formats.html +322 -0
  73. phyddle-0.0.7/_build/genindex.html +119 -0
  74. phyddle-0.0.7/_build/glossary.html +242 -0
  75. phyddle-0.0.7/_build/index.html +141 -0
  76. phyddle-0.0.7/_build/installation.html +258 -0
  77. phyddle-0.0.7/_build/objects.inv +0 -0
  78. phyddle-0.0.7/_build/pipeline.html +434 -0
  79. phyddle-0.0.7/_build/quick_start.html +156 -0
  80. phyddle-0.0.7/_build/search.html +134 -0
  81. phyddle-0.0.7/_build/searchindex.js +1 -0
  82. phyddle-0.0.7/_build/tricks.html +146 -0
  83. phyddle-0.0.7/_build/updates.html +204 -0
  84. phyddle-0.0.7/_build/workspace.html +293 -0
  85. phyddle-0.0.7/build.sh +14 -0
  86. phyddle-0.0.7/docs/Makefile +20 -0
  87. phyddle-0.0.7/docs/README.md +10 -0
  88. phyddle-0.0.7/docs/make.bat +35 -0
  89. phyddle-0.0.7/docs/source/_static/css/custom.css +46 -0
  90. phyddle-0.0.7/docs/source/about.rst +33 -0
  91. phyddle-0.0.7/docs/source/conf.py +66 -0
  92. phyddle-0.0.7/docs/source/configuration.rst +306 -0
  93. phyddle-0.0.7/docs/source/formats.rst +247 -0
  94. phyddle-0.0.7/docs/source/glossary.rst +16 -0
  95. phyddle-0.0.7/docs/source/index.rst +40 -0
  96. phyddle-0.0.7/docs/source/installation.rst +168 -0
  97. phyddle-0.0.7/docs/source/phyddle_logo.png +0 -0
  98. phyddle-0.0.7/docs/source/phyddle_pipeline.png +0 -0
  99. phyddle-0.0.7/docs/source/pipeline.rst +372 -0
  100. phyddle-0.0.7/docs/source/quick_start.rst +49 -0
  101. phyddle-0.0.7/docs/source/tables/glossary.csv +36 -0
  102. phyddle-0.0.7/docs/source/tables/glossary_raw.csv +33 -0
  103. phyddle-0.0.7/docs/source/tables/make_settings_phyddle.sh +8 -0
  104. phyddle-0.0.7/docs/source/tables/settings_phyddle.csv +46 -0
  105. phyddle-0.0.7/docs/source/tricks.rst +38 -0
  106. phyddle-0.0.7/docs/source/updates.rst +91 -0
  107. phyddle-0.0.7/docs/source/workspace.rst +205 -0
  108. phyddle-0.0.7/git_info.txt +1 -0
  109. phyddle-0.0.7/pyproject.toml +41 -0
  110. phyddle-0.0.7/scripts/__config_default.py +91 -0
  111. phyddle-0.0.7/scripts/config.py +88 -0
  112. phyddle-0.0.7/scripts/config_MASTER.py +88 -0
  113. phyddle-0.0.7/scripts/config_R.py +89 -0
  114. phyddle-0.0.7/scripts/config_Rev.py +93 -0
  115. phyddle-0.0.7/scripts/config_SIRM.py +92 -0
  116. phyddle-0.0.7/scripts/nb_phyddle.ipynb +449 -0
  117. phyddle-0.0.7/scripts/run_phyddle.py +55 -0
  118. phyddle-0.0.7/scripts/sim/MASTER/master_util.py +459 -0
  119. phyddle-0.0.7/scripts/sim/MASTER/models/GeosseModel.py +412 -0
  120. phyddle-0.0.7/scripts/sim/MASTER/models/SirmModel.py +238 -0
  121. phyddle-0.0.7/scripts/sim/MASTER/models/TODO.md +19 -0
  122. phyddle-0.0.7/scripts/sim/MASTER/models/__init__.py +10 -0
  123. phyddle-0.0.7/scripts/sim/MASTER/models/classe_model.py +333 -0
  124. phyddle-0.0.7/scripts/sim/MASTER/models/cont_trait_model.py +221 -0
  125. phyddle-0.0.7/scripts/sim/MASTER/models/model.py +288 -0
  126. phyddle-0.0.7/scripts/sim/MASTER/sim_one.py +94 -0
  127. phyddle-0.0.7/scripts/sim/MASTER/sim_one_SIR.py +94 -0
  128. phyddle-0.0.7/scripts/sim/MASTER/sim_one_thin.py +94 -0
  129. phyddle-0.0.7/scripts/sim/MASTER/~/projects/phyddle/workspace/simulate/MASTER_example/sim.0.xml +200 -0
  130. phyddle-0.0.7/scripts/sim/MASTER/~/projects/phyddle/workspace/simulate/MASTER_example/sim.1.xml +200 -0
  131. phyddle-0.0.7/scripts/sim/R/sim_cont_one.R +98 -0
  132. phyddle-0.0.7/scripts/sim/R/sim_one.R +86 -0
  133. phyddle-0.0.7/scripts/sim/Rev/sim_one.Rev +78 -0
  134. phyddle-0.0.7/scripts/util/clean_project.sh +11 -0
  135. phyddle-0.0.7/scripts/util/download_all.sh +23 -0
  136. phyddle-0.0.7/scripts/util/download_networks.sh +10 -0
  137. phyddle-0.0.7/scripts/util/download_plots.sh +10 -0
  138. phyddle-0.0.7/scripts/util/download_tensors.sh +12 -0
  139. phyddle-0.0.7/scripts/util/make_fake_est_dataset.sh +19 -0
  140. phyddle-0.0.7/setup.py +26 -0
  141. phyddle-0.0.7/src/phyddle/__init__.py +33 -0
  142. phyddle-0.0.7/src/phyddle/__main__.py +8 -0
  143. phyddle-0.0.7/src/phyddle/command_line.py +53 -0
  144. phyddle-0.0.7/src/phyddle/estimate.py +409 -0
  145. phyddle-0.0.7/src/phyddle/format.py +1052 -0
  146. phyddle-0.0.7/src/phyddle/pipeline_step.py +52 -0
  147. phyddle-0.0.7/src/phyddle/plot.py +1021 -0
  148. phyddle-0.0.7/src/phyddle/simulate.py +242 -0
  149. phyddle-0.0.7/src/phyddle/train.py +777 -0
  150. phyddle-0.0.7/src/phyddle/utilities.py +2011 -0
  151. phyddle-0.0.7/test.sh +3 -0
  152. phyddle-0.0.7/tests/__config_default.py +92 -0
  153. phyddle-0.0.7/tests/config.py +90 -0
  154. phyddle-0.0.7/tests/data/sim.1.dat.nex +15 -0
  155. phyddle-0.0.7/tests/sim_one.R +55 -0
  156. phyddle-0.0.7/tests/test_estimate.py +26 -0
  157. phyddle-0.0.7/tests/test_format.py +26 -0
  158. phyddle-0.0.7/tests/test_pipeline_MASTER_simple.py +95 -0
  159. phyddle-0.0.7/tests/test_pipeline_R_simple.py +95 -0
  160. phyddle-0.0.7/tests/test_pipeline_Rev_simple.py +95 -0
  161. phyddle-0.0.7/tests/test_plot.py +26 -0
  162. phyddle-0.0.7/tests/test_simulate.py +26 -0
  163. phyddle-0.0.7/tests/test_train.py +26 -0
  164. phyddle-0.0.7/tests/test_utilities.py +26 -0
  165. phyddle-0.0.7/workspace/estimate/README.md +35 -0
  166. phyddle-0.0.7/workspace/estimate/example/new.1.cdvs.csv +1 -0
  167. phyddle-0.0.7/workspace/estimate/example/new.1.dat.nex +408 -0
  168. phyddle-0.0.7/workspace/estimate/example/new.1.extant.tre +1 -0
  169. phyddle-0.0.7/workspace/estimate/example/new.1.info.csv +6 -0
  170. phyddle-0.0.7/workspace/estimate/example/new.1.sim_batchsize128_numepoch20_nt500.pred_labels.csv +2 -0
  171. phyddle-0.0.7/workspace/estimate/example/new.1.summ_stat.csv +2 -0
  172. phyddle-0.0.7/workspace/estimate/example/new.1.tre +1 -0
  173. phyddle-0.0.7/workspace/format/README.md +24 -0
  174. phyddle-0.0.7/workspace/format/example/sim.nt200.hdf5 +0 -0
  175. phyddle-0.0.7/workspace/format/example/sim.nt500.hdf5 +0 -0
  176. phyddle-0.0.7/workspace/log/README.md +3 -0
  177. phyddle-0.0.7/workspace/plot/README.md +49 -0
  178. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.histogram_aux.pdf +0 -0
  179. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.histogram_label.pdf +0 -0
  180. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.network_architecture.pdf +0 -0
  181. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.pca_aux.pdf +0 -0
  182. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.pred_est_CI.pdf +0 -0
  183. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500.summary.pdf +0 -0
  184. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_history_.pdf +0 -0
  185. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_history_param_lower.pdf +0 -0
  186. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_history_param_upper.pdf +0 -0
  187. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_history_param_value.pdf +0 -0
  188. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_test_b_0_1.pdf +0 -0
  189. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_test_d_0_1.pdf +0 -0
  190. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_test_e_0.pdf +0 -0
  191. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_test_w_0.pdf +0 -0
  192. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_train_b_0_1.pdf +0 -0
  193. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_train_d_0_1.pdf +0 -0
  194. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_train_e_0.pdf +0 -0
  195. phyddle-0.0.7/workspace/plot/example/sim_batchsize128_numepoch20_nt500_train_w_0.pdf +0 -0
  196. phyddle-0.0.7/workspace/simulate/README.md +35 -0
  197. phyddle-0.0.7/workspace/simulate/example/README.md +1 -0
  198. phyddle-0.0.7/workspace/simulate/example/sim.0.beast.log +49 -0
  199. phyddle-0.0.7/workspace/simulate/example/sim.0.dat.nex +9 -0
  200. phyddle-0.0.7/workspace/simulate/example/sim.0.json +1 -0
  201. phyddle-0.0.7/workspace/simulate/example/sim.0.param_col.csv +28 -0
  202. phyddle-0.0.7/workspace/simulate/example/sim.0.param_row.csv +2 -0
  203. phyddle-0.0.7/workspace/simulate/example/sim.0.phy.nex +5 -0
  204. phyddle-0.0.7/workspace/simulate/example/sim.0.tre +1 -0
  205. phyddle-0.0.7/workspace/simulate/example/sim.0.xml +194 -0
  206. phyddle-0.0.7/workspace/simulate/example/sim.1.beast.log +1211 -0
  207. phyddle-0.0.7/workspace/simulate/example/sim.1.dat.nex +508 -0
  208. phyddle-0.0.7/workspace/simulate/example/sim.1.extant.tre +1 -0
  209. phyddle-0.0.7/workspace/simulate/example/sim.1.info.csv +6 -0
  210. phyddle-0.0.7/workspace/simulate/example/sim.1.json +1 -0
  211. phyddle-0.0.7/workspace/simulate/example/sim.1.param_col.csv +28 -0
  212. phyddle-0.0.7/workspace/simulate/example/sim.1.param_row.csv +2 -0
  213. phyddle-0.0.7/workspace/simulate/example/sim.1.phy.nex +5 -0
  214. phyddle-0.0.7/workspace/simulate/example/sim.1.summ_stat.csv +2 -0
  215. phyddle-0.0.7/workspace/simulate/example/sim.1.tre +1 -0
  216. phyddle-0.0.7/workspace/simulate/example/sim.1.xml +200 -0
  217. phyddle-0.0.7/workspace/simulate/example/sim.10.beast.log +86 -0
  218. phyddle-0.0.7/workspace/simulate/example/sim.10.dat.nex +23 -0
  219. phyddle-0.0.7/workspace/simulate/example/sim.10.extant.tre +1 -0
  220. phyddle-0.0.7/workspace/simulate/example/sim.10.info.csv +6 -0
  221. phyddle-0.0.7/workspace/simulate/example/sim.10.json +1 -0
  222. phyddle-0.0.7/workspace/simulate/example/sim.10.param_col.csv +28 -0
  223. phyddle-0.0.7/workspace/simulate/example/sim.10.param_row.csv +2 -0
  224. phyddle-0.0.7/workspace/simulate/example/sim.10.phy.nex +5 -0
  225. phyddle-0.0.7/workspace/simulate/example/sim.10.summ_stat.csv +2 -0
  226. phyddle-0.0.7/workspace/simulate/example/sim.10.tre +1 -0
  227. phyddle-0.0.7/workspace/simulate/example/sim.10.xml +197 -0
  228. phyddle-0.0.7/workspace/simulate/example/sim.11.beast.log +61 -0
  229. phyddle-0.0.7/workspace/simulate/example/sim.11.dat.nex +11 -0
  230. phyddle-0.0.7/workspace/simulate/example/sim.11.extant.tre +1 -0
  231. phyddle-0.0.7/workspace/simulate/example/sim.11.json +1 -0
  232. phyddle-0.0.7/workspace/simulate/example/sim.11.param_col.csv +28 -0
  233. phyddle-0.0.7/workspace/simulate/example/sim.11.param_row.csv +2 -0
  234. phyddle-0.0.7/workspace/simulate/example/sim.11.phy.nex +5 -0
  235. phyddle-0.0.7/workspace/simulate/example/sim.11.tre +1 -0
  236. phyddle-0.0.7/workspace/simulate/example/sim.11.xml +197 -0
  237. phyddle-0.0.7/workspace/simulate/example/sim.12.beast.log +76 -0
  238. phyddle-0.0.7/workspace/simulate/example/sim.12.dat.nex +25 -0
  239. phyddle-0.0.7/workspace/simulate/example/sim.12.extant.tre +1 -0
  240. phyddle-0.0.7/workspace/simulate/example/sim.12.json +1 -0
  241. phyddle-0.0.7/workspace/simulate/example/sim.12.param_col.csv +28 -0
  242. phyddle-0.0.7/workspace/simulate/example/sim.12.param_row.csv +2 -0
  243. phyddle-0.0.7/workspace/simulate/example/sim.12.phy.nex +5 -0
  244. phyddle-0.0.7/workspace/simulate/example/sim.12.tre +1 -0
  245. phyddle-0.0.7/workspace/simulate/example/sim.12.xml +197 -0
  246. phyddle-0.0.7/workspace/simulate/example/sim.13.beast.log +50 -0
  247. phyddle-0.0.7/workspace/simulate/example/sim.13.dat.nex +9 -0
  248. phyddle-0.0.7/workspace/simulate/example/sim.13.json +1 -0
  249. phyddle-0.0.7/workspace/simulate/example/sim.13.param_col.csv +28 -0
  250. phyddle-0.0.7/workspace/simulate/example/sim.13.param_row.csv +2 -0
  251. phyddle-0.0.7/workspace/simulate/example/sim.13.phy.nex +5 -0
  252. phyddle-0.0.7/workspace/simulate/example/sim.13.tre +1 -0
  253. phyddle-0.0.7/workspace/simulate/example/sim.13.xml +194 -0
  254. phyddle-0.0.7/workspace/simulate/example/sim.14.beast.log +687 -0
  255. phyddle-0.0.7/workspace/simulate/example/sim.14.dat.nex +508 -0
  256. phyddle-0.0.7/workspace/simulate/example/sim.14.extant.tre +1 -0
  257. phyddle-0.0.7/workspace/simulate/example/sim.14.info.csv +6 -0
  258. phyddle-0.0.7/workspace/simulate/example/sim.14.json +1 -0
  259. phyddle-0.0.7/workspace/simulate/example/sim.14.param_col.csv +28 -0
  260. phyddle-0.0.7/workspace/simulate/example/sim.14.param_row.csv +2 -0
  261. phyddle-0.0.7/workspace/simulate/example/sim.14.phy.nex +5 -0
  262. phyddle-0.0.7/workspace/simulate/example/sim.14.summ_stat.csv +2 -0
  263. phyddle-0.0.7/workspace/simulate/example/sim.14.tre +1 -0
  264. phyddle-0.0.7/workspace/simulate/example/sim.14.xml +194 -0
  265. phyddle-0.0.7/workspace/simulate/example/sim.15.beast.log +153 -0
  266. phyddle-0.0.7/workspace/simulate/example/sim.15.dat.nex +82 -0
  267. phyddle-0.0.7/workspace/simulate/example/sim.15.extant.tre +1 -0
  268. phyddle-0.0.7/workspace/simulate/example/sim.15.info.csv +6 -0
  269. phyddle-0.0.7/workspace/simulate/example/sim.15.json +1 -0
  270. phyddle-0.0.7/workspace/simulate/example/sim.15.param_col.csv +28 -0
  271. phyddle-0.0.7/workspace/simulate/example/sim.15.param_row.csv +2 -0
  272. phyddle-0.0.7/workspace/simulate/example/sim.15.phy.nex +5 -0
  273. phyddle-0.0.7/workspace/simulate/example/sim.15.summ_stat.csv +2 -0
  274. phyddle-0.0.7/workspace/simulate/example/sim.15.tre +1 -0
  275. phyddle-0.0.7/workspace/simulate/example/sim.15.xml +197 -0
  276. phyddle-0.0.7/workspace/simulate/example/sim.16.beast.log +49 -0
  277. phyddle-0.0.7/workspace/simulate/example/sim.16.dat.nex +9 -0
  278. phyddle-0.0.7/workspace/simulate/example/sim.16.json +1 -0
  279. phyddle-0.0.7/workspace/simulate/example/sim.16.param_col.csv +28 -0
  280. phyddle-0.0.7/workspace/simulate/example/sim.16.param_row.csv +2 -0
  281. phyddle-0.0.7/workspace/simulate/example/sim.16.phy.nex +5 -0
  282. phyddle-0.0.7/workspace/simulate/example/sim.16.tre +1 -0
  283. phyddle-0.0.7/workspace/simulate/example/sim.16.xml +194 -0
  284. phyddle-0.0.7/workspace/simulate/example/sim.17.beast.log +121 -0
  285. phyddle-0.0.7/workspace/simulate/example/sim.17.dat.nex +73 -0
  286. phyddle-0.0.7/workspace/simulate/example/sim.17.extant.tre +1 -0
  287. phyddle-0.0.7/workspace/simulate/example/sim.17.info.csv +6 -0
  288. phyddle-0.0.7/workspace/simulate/example/sim.17.json +1 -0
  289. phyddle-0.0.7/workspace/simulate/example/sim.17.param_col.csv +28 -0
  290. phyddle-0.0.7/workspace/simulate/example/sim.17.param_row.csv +2 -0
  291. phyddle-0.0.7/workspace/simulate/example/sim.17.phy.nex +5 -0
  292. phyddle-0.0.7/workspace/simulate/example/sim.17.summ_stat.csv +2 -0
  293. phyddle-0.0.7/workspace/simulate/example/sim.17.tre +1 -0
  294. phyddle-0.0.7/workspace/simulate/example/sim.17.xml +194 -0
  295. phyddle-0.0.7/workspace/simulate/example/sim.18.beast.log +97 -0
  296. phyddle-0.0.7/workspace/simulate/example/sim.18.dat.nex +36 -0
  297. phyddle-0.0.7/workspace/simulate/example/sim.18.extant.tre +1 -0
  298. phyddle-0.0.7/workspace/simulate/example/sim.18.info.csv +6 -0
  299. phyddle-0.0.7/workspace/simulate/example/sim.18.json +1 -0
  300. phyddle-0.0.7/workspace/simulate/example/sim.18.param_col.csv +28 -0
  301. phyddle-0.0.7/workspace/simulate/example/sim.18.param_row.csv +2 -0
  302. phyddle-0.0.7/workspace/simulate/example/sim.18.phy.nex +5 -0
  303. phyddle-0.0.7/workspace/simulate/example/sim.18.summ_stat.csv +2 -0
  304. phyddle-0.0.7/workspace/simulate/example/sim.18.tre +1 -0
  305. phyddle-0.0.7/workspace/simulate/example/sim.18.xml +200 -0
  306. phyddle-0.0.7/workspace/simulate/example/sim.19.beast.log +60 -0
  307. phyddle-0.0.7/workspace/simulate/example/sim.19.dat.nex +15 -0
  308. phyddle-0.0.7/workspace/simulate/example/sim.19.extant.tre +1 -0
  309. phyddle-0.0.7/workspace/simulate/example/sim.19.json +1 -0
  310. phyddle-0.0.7/workspace/simulate/example/sim.19.param_col.csv +28 -0
  311. phyddle-0.0.7/workspace/simulate/example/sim.19.param_row.csv +2 -0
  312. phyddle-0.0.7/workspace/simulate/example/sim.19.phy.nex +5 -0
  313. phyddle-0.0.7/workspace/simulate/example/sim.19.tre +1 -0
  314. phyddle-0.0.7/workspace/simulate/example/sim.19.xml +197 -0
  315. phyddle-0.0.7/workspace/simulate/example/sim.2.beast.log +51 -0
  316. phyddle-0.0.7/workspace/simulate/example/sim.2.dat.nex +10 -0
  317. phyddle-0.0.7/workspace/simulate/example/sim.2.extant.tre +1 -0
  318. phyddle-0.0.7/workspace/simulate/example/sim.2.json +1 -0
  319. phyddle-0.0.7/workspace/simulate/example/sim.2.param_col.csv +28 -0
  320. phyddle-0.0.7/workspace/simulate/example/sim.2.param_row.csv +2 -0
  321. phyddle-0.0.7/workspace/simulate/example/sim.2.phy.nex +5 -0
  322. phyddle-0.0.7/workspace/simulate/example/sim.2.tre +1 -0
  323. phyddle-0.0.7/workspace/simulate/example/sim.2.xml +194 -0
  324. phyddle-0.0.7/workspace/simulate/example/sim.3.beast.log +65 -0
  325. phyddle-0.0.7/workspace/simulate/example/sim.3.dat.nex +16 -0
  326. phyddle-0.0.7/workspace/simulate/example/sim.3.extant.tre +1 -0
  327. phyddle-0.0.7/workspace/simulate/example/sim.3.json +1 -0
  328. phyddle-0.0.7/workspace/simulate/example/sim.3.param_col.csv +28 -0
  329. phyddle-0.0.7/workspace/simulate/example/sim.3.param_row.csv +2 -0
  330. phyddle-0.0.7/workspace/simulate/example/sim.3.phy.nex +5 -0
  331. phyddle-0.0.7/workspace/simulate/example/sim.3.tre +1 -0
  332. phyddle-0.0.7/workspace/simulate/example/sim.3.xml +194 -0
  333. phyddle-0.0.7/workspace/simulate/example/sim.4.beast.log +820 -0
  334. phyddle-0.0.7/workspace/simulate/example/sim.4.dat.nex +508 -0
  335. phyddle-0.0.7/workspace/simulate/example/sim.4.extant.tre +1 -0
  336. phyddle-0.0.7/workspace/simulate/example/sim.4.info.csv +6 -0
  337. phyddle-0.0.7/workspace/simulate/example/sim.4.json +1 -0
  338. phyddle-0.0.7/workspace/simulate/example/sim.4.param_col.csv +28 -0
  339. phyddle-0.0.7/workspace/simulate/example/sim.4.param_row.csv +2 -0
  340. phyddle-0.0.7/workspace/simulate/example/sim.4.phy.nex +5 -0
  341. phyddle-0.0.7/workspace/simulate/example/sim.4.summ_stat.csv +2 -0
  342. phyddle-0.0.7/workspace/simulate/example/sim.4.tre +1 -0
  343. phyddle-0.0.7/workspace/simulate/example/sim.4.xml +197 -0
  344. phyddle-0.0.7/workspace/simulate/example/sim.5.beast.log +805 -0
  345. phyddle-0.0.7/workspace/simulate/example/sim.5.dat.nex +508 -0
  346. phyddle-0.0.7/workspace/simulate/example/sim.5.extant.tre +1 -0
  347. phyddle-0.0.7/workspace/simulate/example/sim.5.info.csv +6 -0
  348. phyddle-0.0.7/workspace/simulate/example/sim.5.json +1 -0
  349. phyddle-0.0.7/workspace/simulate/example/sim.5.param_col.csv +28 -0
  350. phyddle-0.0.7/workspace/simulate/example/sim.5.param_row.csv +2 -0
  351. phyddle-0.0.7/workspace/simulate/example/sim.5.phy.nex +5 -0
  352. phyddle-0.0.7/workspace/simulate/example/sim.5.summ_stat.csv +2 -0
  353. phyddle-0.0.7/workspace/simulate/example/sim.5.tre +1 -0
  354. phyddle-0.0.7/workspace/simulate/example/sim.5.xml +200 -0
  355. phyddle-0.0.7/workspace/simulate/example/sim.6.beast.log +67 -0
  356. phyddle-0.0.7/workspace/simulate/example/sim.6.dat.nex +20 -0
  357. phyddle-0.0.7/workspace/simulate/example/sim.6.extant.tre +1 -0
  358. phyddle-0.0.7/workspace/simulate/example/sim.6.info.csv +6 -0
  359. phyddle-0.0.7/workspace/simulate/example/sim.6.json +1 -0
  360. phyddle-0.0.7/workspace/simulate/example/sim.6.param_col.csv +28 -0
  361. phyddle-0.0.7/workspace/simulate/example/sim.6.param_row.csv +2 -0
  362. phyddle-0.0.7/workspace/simulate/example/sim.6.phy.nex +5 -0
  363. phyddle-0.0.7/workspace/simulate/example/sim.6.summ_stat.csv +2 -0
  364. phyddle-0.0.7/workspace/simulate/example/sim.6.tre +1 -0
  365. phyddle-0.0.7/workspace/simulate/example/sim.6.xml +197 -0
  366. phyddle-0.0.7/workspace/simulate/example/sim.7.beast.log +57 -0
  367. phyddle-0.0.7/workspace/simulate/example/sim.7.dat.nex +14 -0
  368. phyddle-0.0.7/workspace/simulate/example/sim.7.extant.tre +1 -0
  369. phyddle-0.0.7/workspace/simulate/example/sim.7.json +1 -0
  370. phyddle-0.0.7/workspace/simulate/example/sim.7.param_col.csv +28 -0
  371. phyddle-0.0.7/workspace/simulate/example/sim.7.param_row.csv +2 -0
  372. phyddle-0.0.7/workspace/simulate/example/sim.7.phy.nex +5 -0
  373. phyddle-0.0.7/workspace/simulate/example/sim.7.tre +1 -0
  374. phyddle-0.0.7/workspace/simulate/example/sim.7.xml +194 -0
  375. phyddle-0.0.7/workspace/simulate/example/sim.8.beast.log +52 -0
  376. phyddle-0.0.7/workspace/simulate/example/sim.8.dat.nex +10 -0
  377. phyddle-0.0.7/workspace/simulate/example/sim.8.json +1 -0
  378. phyddle-0.0.7/workspace/simulate/example/sim.8.param_col.csv +28 -0
  379. phyddle-0.0.7/workspace/simulate/example/sim.8.param_row.csv +2 -0
  380. phyddle-0.0.7/workspace/simulate/example/sim.8.phy.nex +5 -0
  381. phyddle-0.0.7/workspace/simulate/example/sim.8.tre +1 -0
  382. phyddle-0.0.7/workspace/simulate/example/sim.8.xml +197 -0
  383. phyddle-0.0.7/workspace/simulate/example/sim.9.beast.log +692 -0
  384. phyddle-0.0.7/workspace/simulate/example/sim.9.dat.nex +508 -0
  385. phyddle-0.0.7/workspace/simulate/example/sim.9.extant.tre +1 -0
  386. phyddle-0.0.7/workspace/simulate/example/sim.9.info.csv +6 -0
  387. phyddle-0.0.7/workspace/simulate/example/sim.9.json +1 -0
  388. phyddle-0.0.7/workspace/simulate/example/sim.9.param_col.csv +28 -0
  389. phyddle-0.0.7/workspace/simulate/example/sim.9.param_row.csv +2 -0
  390. phyddle-0.0.7/workspace/simulate/example/sim.9.phy.nex +5 -0
  391. phyddle-0.0.7/workspace/simulate/example/sim.9.summ_stat.csv +2 -0
  392. phyddle-0.0.7/workspace/simulate/example/sim.9.tre +1 -0
  393. phyddle-0.0.7/workspace/simulate/example/sim.9.xml +197 -0
  394. phyddle-0.0.7/workspace/train/README.md +31 -0
  395. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.cqr_interval_adjustments.csv +3 -0
  396. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.hdf5 +0 -0
  397. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.test_labels.csv +157 -0
  398. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.test_pred.csv +157 -0
  399. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.test_pred_nocalib.csv +157 -0
  400. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_history.json +1 -0
  401. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_label_norm.csv +5 -0
  402. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_labels.csv +1001 -0
  403. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_pred.csv +2197 -0
  404. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_pred_nocalib.csv +2197 -0
  405. phyddle-0.0.7/workspace/train/example/sim_batchsize128_numepoch20_nt500.train_summ_stat_norm.csv +22 -0
@@ -0,0 +1,24 @@
1
+ name: Docs
2
+ on: [push, pull_request, workflow_dispatch]
3
+ permissions:
4
+ contents: write
5
+ jobs:
6
+ docs:
7
+ runs-on: ubuntu-latest
8
+ steps:
9
+ - uses: actions/checkout@v3
10
+ - uses: actions/setup-python@v3
11
+ - name: Install dependencies
12
+ run: |
13
+ pip install sphinx sphinx_rtd_theme
14
+ - name: Sphinx build
15
+ run: |
16
+ sphinx-build docs/source _build
17
+ - name: Deploy
18
+ uses: peaceiris/actions-gh-pages@v3
19
+ if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
20
+ with:
21
+ publish_branch: gh-pages
22
+ github_token: ${{ secrets.GITHUB_TOKEN }}
23
+ publish_dir: _build/
24
+ force_orphan: true
@@ -0,0 +1,41 @@
1
+ # https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
2
+
3
+ name: Python package
4
+
5
+ on: [push]
6
+
7
+ jobs:
8
+ build:
9
+
10
+ runs-on: ubuntu-latest
11
+ strategy:
12
+ matrix:
13
+ python-version: ['3.11']
14
+ R: [ '4.2.2' ]
15
+
16
+ steps:
17
+ - uses: actions/checkout@v3
18
+ - name: Setup R ${{ matrix.R }}
19
+ uses: r-lib/actions/setup-r@v2
20
+ with:
21
+ r-version: ${{ matrix.R }}
22
+ - name: Set up Python ${{ matrix.python-version }}
23
+ uses: actions/setup-python@v4
24
+ with:
25
+ python-version: ${{ matrix.python-version }}
26
+ - name: Install Python dependencies
27
+ run: |
28
+ Rscript -e "install.packages('ape', repos='http://cran.us.r-project.org')"
29
+ python -m pip install --upgrade pip
30
+ pip install .
31
+ pip install ruff pytest scipy numpy matplotlib tensorflow keras scikit-learn dendropy pandas pypdf joblib tqdm
32
+ if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
33
+ # - name: Lint with ruff
34
+ # run: |
35
+ # # stop the build if there are Python syntax errors or undefined names
36
+ # ruff --format=github --select=E9,F63,F7,F82 --target-version=py37 .
37
+ # # default set of ruff rules with GitHub Annotations
38
+ # ruff --format=github --target-version=py37 .
39
+ - name: Test with pytest
40
+ run: |
41
+ pytest tests/
@@ -0,0 +1,57 @@
1
+ # Ignore phyddle workspace dirs
2
+ workspace/simulate/*/*
3
+ workspace/format/*/*
4
+ workspace/train/*/*
5
+ workspace/estimate/*/*
6
+ workspace/plot/*/*
7
+ workspace/log/*/*
8
+
9
+ # ... except workspace `example` project
10
+ !workspace/simulate/example/*
11
+ !workspace/format/example/*
12
+ !workspace/train/example/*
13
+ !workspace/estimate/example/*
14
+ !workspace/plot/example/*
15
+ !workspace/log/example/*
16
+
17
+ # Ignore extras in tests
18
+ tests/workspace/*
19
+
20
+ # Ignore phyddle accessory dirs
21
+ sandbox/
22
+ paper/
23
+ dist/
24
+ code/
25
+ _old/
26
+ scripts/config
27
+
28
+ # Ignore phyddle accessory files
29
+ notes.txt
30
+ pkg_notes.txt
31
+
32
+ # Ignore everywhere
33
+ #.Rhistory
34
+ #.swp
35
+ #.vscode/
36
+ #.ipynb_checkpoints/
37
+ #__pycache__
38
+ #.DS_Store
39
+ #.DS_Store?
40
+ **/.Rhistory
41
+ **/*.swp
42
+ **/.vscode/
43
+ **/.ipynb_checkpoints/
44
+ **/__pycache__
45
+ **/.DS_Store
46
+ **/.DS_Store?
47
+
48
+ # Ignore Sphinx extras
49
+ #docs/build/doctrees
50
+ #docs/build/doctrees/*
51
+ #docs/build/html
52
+ docs/build/*
53
+
54
+ # Ignore OS files
55
+
56
+ # Ignore programming files
57
+
phyddle-0.0.7/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ # Released under MIT License
2
+
3
+ Copyright (c) 2023 Michael Landis
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy of
6
+ this software and associated documentation files (phyddle; the “Software”), to deal in
7
+ the Software without restriction, including without limitation the rights to
8
+ use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
9
+ of the Software, and to permit persons to whom the Software is furnished to do
10
+ so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
17
+ FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
18
+ COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
19
+ IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
20
+ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
21
+
phyddle-0.0.7/PKG-INFO ADDED
@@ -0,0 +1,132 @@
1
+ Metadata-Version: 2.1
2
+ Name: phyddle
3
+ Version: 0.0.7
4
+ Summary: A package to fiddle around with phylogenetic models and deep learning
5
+ Project-URL: Homepage, https://github.com/mlandis/phyddle
6
+ Project-URL: Bug Reports, https://github.com/mlandis/phyddle/issues
7
+ Project-URL: Documentation, https://mlandis.github.io/phyddle
8
+ Author-email: Michael Landis <michael.landis@wustl.edu>, Ammon Thompson <ammonthompson@gmail.com>
9
+ License-File: LICENSE
10
+ Classifier: Development Status :: 3 - Alpha
11
+ Classifier: Environment :: Console
12
+ Classifier: Intended Audience :: Science/Research
13
+ Classifier: License :: OSI Approved :: MIT License
14
+ Classifier: Natural Language :: English
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
18
+ Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
19
+ Classifier: Topic :: Scientific/Engineering :: Mathematics
20
+ Requires-Python: >=3.7
21
+ Description-Content-Type: text/markdown
22
+
23
+ # phyddle
24
+
25
+ A pipeline-based toolkit for fiddling around with phylogenetic models and deep learning
26
+
27
+ > <b>NOTE: *This private beta version of phyddle is still under development.</b> Code on the main brain is tested and stable with respect to the standard use cases. Code on the development branch contains new features, but is not as rigorously tested. Most phyddle development occurs on a 16-core Intel Macbook Pro laptop and a 64-core Intel Ubuntu server, so there are also unknown portability/scalability issues to correct. Any feedback is appreciated! michael.landis@wustl.edu*
28
+
29
+
30
+ ## Overview
31
+
32
+ <img align="right" src="https://github.com/landislab/landislab.github.io/blob/5bb4685a12ebf4c99dd773de6d87b44cc3c47090/assets/research/img/phyddle_pipeline.png?raw=true" width="35%">
33
+
34
+ A standard phyddle analysis performs the following tasks for you:
35
+
36
+ - **Pipeline configuration** applies analysis settings provided through a config file and/or command line arguments.
37
+ - **Model configuration** constructs a base simulating model to be *Simulated* (states, events, rates).
38
+ - **Simulating** simulates a large training dataset under the model to be *Formatted* (parallelized, partly compressed). Saves output in the [`raw_data`](workspace/raw_data) directory.
39
+ - **Formatting** encodes the raw simulated data into tensor format for *Learning* (parallelized, compressed). Saves output in the [`tensor_data`](workspace/tensor_data) directory.
40
+ - **Learning** shuffles and splits training data, builds a network, then trains and saves the network with the data for *Prediction*. Saves output in the [`network`](workspace/network) directory.
41
+ - **Predicting** estimates model parameters for a new dataset with the trained network. Saves output in the [`predict`](workspace/predict) directory.
42
+ - **Plotting** generates figures that summarize the training data (*Formatting*), the network and its training (*Learning*), and any new predictions (*Predicting*). Saves output in the [`plot`](workspace/plot) directory.
43
+
44
+
45
+ ## Quick installation
46
+
47
+ > Note: phyddle is currently on a private repository. Installing phyddle and viewing documentation will be easier once the repository is made public.
48
+
49
+ Currently, phyddle must be manually installed on the local filesystem to be used.
50
+ To install it, clone the repository, enter the new phyddle directory, then run the `build.sh` script.
51
+
52
+ ```shell
53
+ git clone git@github.com:mlandis/phyddle.git
54
+ cd phyddle
55
+ git checkout development
56
+ ./build.sh
57
+ ```
58
+
59
+ To get the newest features added to phyddle, you will want to pull changes into the development branch, then rebuild the package
60
+ ```shell
61
+ git pull
62
+ ./build.sh
63
+ ```
64
+
65
+ Project [documentation](docs/build/html/index.html) can be viewed from your local web browser, but will be hosted online once the repo is made public. The `build.sh` script will build a local copy of the documentation on your filesystem. The terminal command `open docs/build/html/index.html` will open the documentation on Mac OS X in your web browser.
66
+
67
+ The full documentation explains that running phyddle requires a recent version of Python (3.10+) and recent versions of several Python packages. The packages can be installed using pip with this command.
68
+
69
+ ```shell
70
+ python3 -m ensurepip --upgrade
71
+ python3 -m pip install --upgrade pip
72
+ python3 -m pip install argparse h5py joblib keras matplotlib numpy pandas Pillow pydot_ng pypdf scipy scikit-learn tensorflow tqdm
73
+ ```
74
+
75
+ phyddle uses third-party simulators to generate training datasets. The standard workflow assumes that BEAST v2.7.3 with MASTER v7.0.0 (plugin) is installed on your machine and be executed from terminal with the command `beast`. The documentation explains how to configure BEAST and MASTER for use with phyddle.
76
+
77
+ ## Quick start
78
+
79
+ To run a phyddle analysis enter the `scripts` directory:
80
+ ```shell
81
+ cd ~/projects/phyddle/scripts
82
+ ```
83
+
84
+ Then create and run a pipeline under the settings you've specified in `my_config.py`:
85
+ ```shell
86
+ ./run_phyddle.py --cfg my_config
87
+ ```
88
+
89
+ This will run a phyddle analysis for a simple 3-region GeoSSE model with just 500 training examples. In practice, you'll want to generate a larger training dataset with anywhere from 10k to 1M examples, depending on the model.
90
+
91
+ To add new examples to your training set
92
+ ```shell
93
+ # simulate new training examples and store in raw_data/my_project
94
+ ./run_phyddle.py -s sim -c my_config --start_idx 500 --end_idx 15000
95
+
96
+ # encode all raw_data examples as tensors in tensor_data/my_project
97
+ ./run_phyddle.py -s fmt -c my_config --start_idx 0 --end_idx 15000
98
+
99
+ # train network with tensor_data, but override batch size, then store in network/my_project
100
+ ./run_phyddle.py -s lrn -c my_config --batch_size 256
101
+
102
+ # make prediction with example dataset
103
+ ./run_phyddle.py -s prd -c my_config
104
+
105
+ # generate figures and store in plot
106
+ ./run_phyddle.spy -s plt -c my_config
107
+ ```
108
+
109
+ Pipeline options are applied to all pipeline stages. See the full list of currently supported options with
110
+ ```shell
111
+ ./run_phyddle.sh --help
112
+ ```
113
+
114
+ ## Features
115
+
116
+ Current features:
117
+ - trained network generates parameter estimates and coverage-calibrated prediction intervals (CPIs) for input datasets
118
+ - provides several state-dependent birth-death model types and variants (more to come)
119
+ - parallelized simulating, formatting, and learning
120
+ - encoding of phylogenetic-state tensor from serial and extant-only input with multiple states (CBLV+S and CDV+S extensions)
121
+ - encoding of auxiliary data tensor from automatically computed summary statistics and "known" parameter (e.g. sampling rate)
122
+ - HDF5 with gzip compression for tensor data
123
+ - shuffles and splits input tensors into training, test, validation, and calibration datasets for supervised learning
124
+ - builds network with convolution, pooling, and dense layers that match input tensors
125
+ - trains network and saves history
126
+ - automatic figure generation with Matplotlib
127
+
128
+ ## About
129
+ Thanks for your interest in phyddle. The phyddle project emerged from a phylogenetic deep learning study led by Ammon Thompson ([paper](https://www.biorxiv.org/content/10.1101/2023.02.08.527714v2)). The goal of phyddle is to provide its users with a generalizable pipeline workflow for phylogenetic modeling and deep learning. This hopefully will make it easier for phylogenetic model enthusiasts and developers to explore and apply models that do not have tractable likelihood functions. It's also intended for use by methods developers who want to characterize how deep learning methods perform under different conditions for standard phylogenetic estimation tasks.
130
+
131
+ The phyddle project is developed by [Michael Landis](https://landislab.org) and [Ammon Thompson](https://scholar.google.com/citations?user=_EpmmTwAAAAJ&hl=en&oi=ao).
132
+
@@ -0,0 +1,110 @@
1
+ # phyddle
2
+
3
+ A pipeline-based toolkit for fiddling around with phylogenetic models and deep learning
4
+
5
+ > <b>NOTE: *This private beta version of phyddle is still under development.</b> Code on the main brain is tested and stable with respect to the standard use cases. Code on the development branch contains new features, but is not as rigorously tested. Most phyddle development occurs on a 16-core Intel Macbook Pro laptop and a 64-core Intel Ubuntu server, so there are also unknown portability/scalability issues to correct. Any feedback is appreciated! michael.landis@wustl.edu*
6
+
7
+
8
+ ## Overview
9
+
10
+ <img align="right" src="https://github.com/landislab/landislab.github.io/blob/5bb4685a12ebf4c99dd773de6d87b44cc3c47090/assets/research/img/phyddle_pipeline.png?raw=true" width="35%">
11
+
12
+ A standard phyddle analysis performs the following tasks for you:
13
+
14
+ - **Pipeline configuration** applies analysis settings provided through a config file and/or command line arguments.
15
+ - **Model configuration** constructs a base simulating model to be *Simulated* (states, events, rates).
16
+ - **Simulating** simulates a large training dataset under the model to be *Formatted* (parallelized, partly compressed). Saves output in the [`raw_data`](workspace/raw_data) directory.
17
+ - **Formatting** encodes the raw simulated data into tensor format for *Learning* (parallelized, compressed). Saves output in the [`tensor_data`](workspace/tensor_data) directory.
18
+ - **Learning** shuffles and splits training data, builds a network, then trains and saves the network with the data for *Prediction*. Saves output in the [`network`](workspace/network) directory.
19
+ - **Predicting** estimates model parameters for a new dataset with the trained network. Saves output in the [`predict`](workspace/predict) directory.
20
+ - **Plotting** generates figures that summarize the training data (*Formatting*), the network and its training (*Learning*), and any new predictions (*Predicting*). Saves output in the [`plot`](workspace/plot) directory.
21
+
22
+
23
+ ## Quick installation
24
+
25
+ > Note: phyddle is currently on a private repository. Installing phyddle and viewing documentation will be easier once the repository is made public.
26
+
27
+ Currently, phyddle must be manually installed on the local filesystem to be used.
28
+ To install it, clone the repository, enter the new phyddle directory, then run the `build.sh` script.
29
+
30
+ ```shell
31
+ git clone git@github.com:mlandis/phyddle.git
32
+ cd phyddle
33
+ git checkout development
34
+ ./build.sh
35
+ ```
36
+
37
+ To get the newest features added to phyddle, you will want to pull changes into the development branch, then rebuild the package
38
+ ```shell
39
+ git pull
40
+ ./build.sh
41
+ ```
42
+
43
+ Project [documentation](docs/build/html/index.html) can be viewed from your local web browser, but will be hosted online once the repo is made public. The `build.sh` script will build a local copy of the documentation on your filesystem. The terminal command `open docs/build/html/index.html` will open the documentation on Mac OS X in your web browser.
44
+
45
+ The full documentation explains that running phyddle requires a recent version of Python (3.10+) and recent versions of several Python packages. The packages can be installed using pip with this command.
46
+
47
+ ```shell
48
+ python3 -m ensurepip --upgrade
49
+ python3 -m pip install --upgrade pip
50
+ python3 -m pip install argparse h5py joblib keras matplotlib numpy pandas Pillow pydot_ng pypdf scipy scikit-learn tensorflow tqdm
51
+ ```
52
+
53
+ phyddle uses third-party simulators to generate training datasets. The standard workflow assumes that BEAST v2.7.3 with MASTER v7.0.0 (plugin) is installed on your machine and be executed from terminal with the command `beast`. The documentation explains how to configure BEAST and MASTER for use with phyddle.
54
+
55
+ ## Quick start
56
+
57
+ To run a phyddle analysis enter the `scripts` directory:
58
+ ```shell
59
+ cd ~/projects/phyddle/scripts
60
+ ```
61
+
62
+ Then create and run a pipeline under the settings you've specified in `my_config.py`:
63
+ ```shell
64
+ ./run_phyddle.py --cfg my_config
65
+ ```
66
+
67
+ This will run a phyddle analysis for a simple 3-region GeoSSE model with just 500 training examples. In practice, you'll want to generate a larger training dataset with anywhere from 10k to 1M examples, depending on the model.
68
+
69
+ To add new examples to your training set
70
+ ```shell
71
+ # simulate new training examples and store in raw_data/my_project
72
+ ./run_phyddle.py -s sim -c my_config --start_idx 500 --end_idx 15000
73
+
74
+ # encode all raw_data examples as tensors in tensor_data/my_project
75
+ ./run_phyddle.py -s fmt -c my_config --start_idx 0 --end_idx 15000
76
+
77
+ # train network with tensor_data, but override batch size, then store in network/my_project
78
+ ./run_phyddle.py -s lrn -c my_config --batch_size 256
79
+
80
+ # make prediction with example dataset
81
+ ./run_phyddle.py -s prd -c my_config
82
+
83
+ # generate figures and store in plot
84
+ ./run_phyddle.spy -s plt -c my_config
85
+ ```
86
+
87
+ Pipeline options are applied to all pipeline stages. See the full list of currently supported options with
88
+ ```shell
89
+ ./run_phyddle.sh --help
90
+ ```
91
+
92
+ ## Features
93
+
94
+ Current features:
95
+ - trained network generates parameter estimates and coverage-calibrated prediction intervals (CPIs) for input datasets
96
+ - provides several state-dependent birth-death model types and variants (more to come)
97
+ - parallelized simulating, formatting, and learning
98
+ - encoding of phylogenetic-state tensor from serial and extant-only input with multiple states (CBLV+S and CDV+S extensions)
99
+ - encoding of auxiliary data tensor from automatically computed summary statistics and "known" parameter (e.g. sampling rate)
100
+ - HDF5 with gzip compression for tensor data
101
+ - shuffles and splits input tensors into training, test, validation, and calibration datasets for supervised learning
102
+ - builds network with convolution, pooling, and dense layers that match input tensors
103
+ - trains network and saves history
104
+ - automatic figure generation with Matplotlib
105
+
106
+ ## About
107
+ Thanks for your interest in phyddle. The phyddle project emerged from a phylogenetic deep learning study led by Ammon Thompson ([paper](https://www.biorxiv.org/content/10.1101/2023.02.08.527714v2)). The goal of phyddle is to provide its users with a generalizable pipeline workflow for phylogenetic modeling and deep learning. This hopefully will make it easier for phylogenetic model enthusiasts and developers to explore and apply models that do not have tractable likelihood functions. It's also intended for use by methods developers who want to characterize how deep learning methods perform under different conditions for standard phylogenetic estimation tasks.
108
+
109
+ The phyddle project is developed by [Michael Landis](https://landislab.org) and [Ammon Thompson](https://scholar.google.com/citations?user=_EpmmTwAAAAJ&hl=en&oi=ao).
110
+
phyddle-0.0.7/TODO.md ADDED
@@ -0,0 +1,108 @@
1
+ # TODO
2
+
3
+ - More unit tests
4
+
5
+ - Improve simulator logging functionality
6
+ - simulator subprocess should collect useful stdout and stderr to debug issues
7
+ - sim_logging should tidy up logs
8
+
9
+ # Probably good, but not essential
10
+
11
+ - Better phyddle banner graphics, info, etc.
12
+
13
+ - Print all non-default file/command settings in header
14
+
15
+ - Better file format and file location checking and errors
16
+ - check all files and dirs exist before run
17
+ - check all csv/nexus/tree formats after reading
18
+
19
+ - Have phyddle report error when cmd_sim fails
20
+ - e.g. if script is set up wrong, how to report?
21
+
22
+ - Base class for pipeline step?
23
+ - virtual functions for set_args, load_input, run, make_results, save_results
24
+
25
+ - Utilities to combine hdf5 and csv tensors
26
+ - e.g. if we have hdf5 from multiple servers, how to combine them
27
+ - also, tools to convert hdf5 -> csv and csv -> hdf5
28
+
29
+ - Nickname for fileset within project, -n
30
+ - '?' to generate a random name
31
+ - adjective + species
32
+ - also, autogen project name
33
+ - random naem for new project?
34
+
35
+ - Summary text file of all results in one easily greppable file to be processed
36
+
37
+ - Plot network performance
38
+ - weight/bias
39
+ - activation?
40
+ - trained network or across epochs?
41
+ - Plot for node loadings.
42
+
43
+ - Ensure Format can handle multiple characters and states
44
+ - main issue is e.g. num char = 3
45
+ - num states = [3, 5, 2]
46
+
47
+ # Longterm
48
+
49
+ - Handling multiple trees/matrices per example dataset
50
+
51
+ - Model comparison
52
+
53
+ - Ancestral states
54
+
55
+
56
+ # Open questions
57
+
58
+ - What example models/platforms for simulation scripts.
59
+ - Languages: Python, R, RevBayes, MASTER, Julia, shell
60
+ - Models: CTMC, birth-death, SSE, SIR
61
+ - Programs: gen3sis, MESS, SLiM, FAVITES, FossilSim
62
+
63
+ - How to distribution MASTER Python wrappers
64
+ - package with phyddle in subdir
65
+ - separate repository, full package, etc.
66
+
67
+
68
+ # DONE
69
+
70
+ - Add default config to allow for three layers of assignment
71
+ - default -> my_config -> CLI
72
+ - perhaps distribute a default config that people can tweak
73
+
74
+ - Automatically index replicates for sim/fmt
75
+ - Current: --start_idx 100 --end_idx 1000
76
+ - Alternative: --add_sim_rep 10000, then figure out based on os.path.files()
77
+ - Then Format can be done on first 90%, leave out last 10%, then can work that into training/test split
78
+
79
+ - Downsampling of large trees.
80
+ - If tree is larger than max tree width category size, downsample to match max size.
81
+ - User sets downsampling strategy, e.g. uniform, diversified, etc.
82
+ - Downsampling enters downsampling_proportion into aux_data
83
+
84
+ - Change how test dataset is generated and used
85
+ - Currently, test dataset is sampled during Train
86
+ - Instead, have test data split out during Format and saved to file
87
+ - Then have test data fed through Estiamte to produce results
88
+ - This allows us to compare model performance against different batches of test data
89
+
90
+ - Common functions for normalize/denormalize/exp/log/etc.
91
+
92
+ - strip out tree_width_cats
93
+
94
+ - General problem handling bool argparse args
95
+ - https://stackoverflow.com/questions/15008758/parsing-boolean-values-with-argparse
96
+
97
+ - PCA
98
+ - loadings
99
+ - PCA for state tensor too?
100
+ - https://statisticsglobe.com/visualization-pca-python
101
+ - https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.contour.html
102
+
103
+ - Runtime information
104
+ - start/end time for each step
105
+
106
+ - Add support for phyddle binary using __main__()
107
+ - basics working
108
+ - replace import with exec
@@ -0,0 +1,4 @@
1
+ # Sphinx build info version 1
2
+ # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
3
+ config: f43ea59069e53b7dbd496ad07dee7da1
4
+ tags: 645f666f9bcd5a90fca523b33c5a78b7
@@ -0,0 +1,33 @@
1
+ .. _about:
2
+
3
+ About
4
+ =====
5
+
6
+ Thanks for your interest in phyddle. The phyddle project emerged from a phylogenetic deep learning study led by Ammon Thompson (`paper <https://www.biorxiv.org/content/10.1101/2023.02.08.527714v2>`_). The goal of phyddle is to provide its users with a generalizable pipeline workflow for phylogenetic modeling and deep learning. This hopefully will make it easier for phylogenetic model enthusiasts and developers to explore and apply models that do not have tractable likelihood functions. It's also intended for use by methods developers who want to characterize how deep learning methods perform under different conditions for standard phylogenetic estimation tasks.
7
+
8
+ The phyddle project is developed by `Michael Landis <https://landislab.org>`__ and `Ammon Thompson <https://scholar.google.com/citations?user=_EpmmTwAAAAJ&hl=en&oi=ao>`__.
9
+
10
+ .. _issues_feedback:
11
+
12
+ Issues & Feedback
13
+ -----------------
14
+
15
+ Please use `Issues <https://github.com/mlandis/phyddle/issues>`__ to report bugs or request features that require modifying phyddle source code. Please contact `Michael Landis <mailto:michael.landis@wustl.edu>`__ to request troubleshooting support using phyddle.
16
+
17
+
18
+ .. _references:
19
+
20
+ References
21
+ ----------
22
+
23
+ EE Goldberg, LT Lancaster, RH Ree. 2011. Phylogenetic inference of reciprocal effects between geographic range evolution and diversification. Systematic Biology 60:451-465. doi: https://doi.org/10.1093/sysbio/syr046
24
+
25
+ S Lambert, J Voznica, H Morlon. 2022. Deep learning from phylogenies for diversification analyses.
26
+ bioRxiv. 2022.09.27.509667. doi: https://doi.org/10.1101/2022.09.27.509667
27
+
28
+ A Thompson, B Liebeskind, EJ Scully, MJ Landis. 2023. Deep learning approaches to viral phylogeography are fast and as robust as likelihood methods to model misspecification. bioRxiv 2023.02.08.527714; doi: https://doi.org/10.1101/2023.02.08.527714
29
+
30
+ TG Vaughan, AJ Drummond. 2013. A stochastic simulator of birth–death master equations with application to phylodynamics. Mol Biol Evol 30:1480–1493. doi: https://doi.org/10.1093/molbev/mst057
31
+
32
+ J Voznica, A Zhukova, V Boskova, E Saulnier, F Lemoine, M Moslonka-Lefebvre, O Gascuel. 2022. Deep learning from phylogenies to uncover the epidemiological dynamics of outbreaks. Nat Commun 13:3896. doi: https://doi.org/10.1038/s41467-022-31511-0
33
+