thursday-agent 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.next/BUILD_ID +1 -1
- package/.next/build-manifest.json +3 -3
- package/.next/prerender-manifest.json +3 -3
- package/.next/server/app/_global-error/page.js.nft.json +1 -1
- package/.next/server/app/_global-error.html +1 -1
- package/.next/server/app/_global-error.rsc +1 -1
- package/.next/server/app/_global-error.segments/__PAGE__.segment.rsc +1 -1
- package/.next/server/app/_global-error.segments/_full.segment.rsc +1 -1
- package/.next/server/app/_global-error.segments/_tree.segment.rsc +1 -1
- package/.next/server/app/_not-found/page.js.nft.json +1 -1
- package/.next/server/app/_not-found/page_client-reference-manifest.js +1 -1
- package/.next/server/app/_not-found.html +1 -1
- package/.next/server/app/_not-found.rsc +2 -2
- package/.next/server/app/_not-found.segments/_full.segment.rsc +2 -2
- package/.next/server/app/_not-found.segments/_not-found/__PAGE__.segment.rsc +2 -2
- package/.next/server/app/_not-found.segments/_tree.segment.rsc +2 -2
- package/.next/server/app/api/artifact/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/keep-working/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/memory/files/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/memory/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/thread/[id]/route.js.nft.json +1 -1
- package/.next/server/app/api/bot/thread/route.js.nft.json +1 -1
- package/.next/server/app/api/config/route.js.nft.json +1 -1
- package/.next/server/app/api/events/route.js.nft.json +1 -1
- package/.next/server/app/api/favicon/[host]/route.js.nft.json +1 -1
- package/.next/server/app/api/file/[...path]/route.js.nft.json +1 -1
- package/.next/server/app/api/llm-model/catalog/route.js.nft.json +1 -1
- package/.next/server/app/api/llm-model/credits/route.js.nft.json +1 -1
- package/.next/server/app/api/llm-model/route.js.nft.json +1 -1
- package/.next/server/app/api/llm-model/subscription/route.js.nft.json +1 -1
- package/.next/server/app/api/mcp/[name]/route.js.nft.json +1 -1
- package/.next/server/app/api/mcp/oauth/callback/route.js.nft.json +1 -1
- package/.next/server/app/api/mcp/route.js.nft.json +1 -1
- package/.next/server/app/api/mcp/tools/route.js.nft.json +1 -1
- package/.next/server/app/api/memory/edit/route.js +1 -1
- package/.next/server/app/api/memory/edit/route.js.nft.json +1 -1
- package/.next/server/app/api/memory/route.js.nft.json +1 -1
- package/.next/server/app/api/probe/route.js.nft.json +1 -1
- package/.next/server/app/api/reach/route.js +11 -11
- package/.next/server/app/api/reach/route.js.nft.json +1 -1
- package/.next/server/app/api/routine/route.js +6 -6
- package/.next/server/app/api/routine/route.js.nft.json +1 -1
- package/.next/server/app/api/signins/route.js.nft.json +1 -1
- package/.next/server/app/api/skills/[source]/[name]/route.js.nft.json +1 -1
- package/.next/server/app/api/skills/route.js.nft.json +1 -1
- package/.next/server/app/api/thursday/call/route.js.nft.json +1 -1
- package/.next/server/app/api/thursday/skills/route.js.nft.json +1 -1
- package/.next/server/app/api/thursday/text/route.js +11 -11
- package/.next/server/app/api/thursday/text/route.js.nft.json +1 -1
- package/.next/server/app/api/thursday/tool-call/route.js +6 -6
- package/.next/server/app/api/thursday/tool-call/route.js.nft.json +1 -1
- package/.next/server/app/api/workspace/files/route.js.nft.json +1 -1
- package/.next/server/app/api/workspace/route.js.nft.json +1 -1
- package/.next/server/app/artifact/[...path]/page/server-reference-manifest.json +1 -1
- package/.next/server/app/artifact/[...path]/page.js.nft.json +1 -1
- package/.next/server/app/artifact/[...path]/page_client-reference-manifest.js +1 -1
- package/.next/server/app/favicon.ico/route.js.nft.json +1 -1
- package/.next/server/app/manifest.webmanifest/route.js.nft.json +1 -1
- package/.next/server/app/page/server-reference-manifest.json +56 -56
- package/.next/server/app/page.js +3 -3
- package/.next/server/app/page.js.nft.json +1 -1
- package/.next/server/app/page_client-reference-manifest.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__012s2o8._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__04r4h-2._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__0mrk2it._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__0nmhgeo._.js +3 -0
- package/.next/server/chunks/[root-of-the-server]__1fy2g82._.js +3 -0
- package/.next/server/chunks/[root-of-the-server]__1ndi14y._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__1nmflqk._.js +3 -0
- package/.next/server/chunks/[root-of-the-server]__1v7-zke._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__1wnzmx9._.js +3 -0
- package/.next/server/chunks/[root-of-the-server]__1xv-5cq._.js +1 -1
- package/.next/server/chunks/[root-of-the-server]__1ygplx1._.js +1 -1
- package/.next/server/chunks/_05v8k21._.js +1 -1
- package/.next/server/chunks/_0awtznl._.js +4 -4
- package/.next/server/chunks/_0k6qjcf._.js +1 -1
- package/.next/server/chunks/_0na2fj1._.js +5 -5
- package/.next/server/chunks/_0x-ej3f._.js +21 -21
- package/.next/server/chunks/_1dc_d1q._.js +22 -20
- package/.next/server/chunks/_1jc5g4p._.js +2 -2
- package/.next/server/chunks/_1jn-aep._.js +50 -48
- package/.next/server/chunks/{_1u8ygbz._.js → _1m88lkh._.js} +7 -7
- package/.next/server/chunks/_1uuucy-._.js +4 -4
- package/.next/server/chunks/_1v9r2ke._.js +1 -1
- package/.next/server/chunks/ssr/{[root-of-the-server]__1h4bnrf._.js → [root-of-the-server]__0cxqzfc._.js} +2 -2
- package/.next/server/chunks/ssr/{[root-of-the-server]__1kyi568._.js → [root-of-the-server]__0tvavy9._.js} +1 -1
- package/.next/server/chunks/ssr/[root-of-the-server]__0tzekbl._.js +1 -1
- package/.next/server/chunks/ssr/[root-of-the-server]__1-d2tgh._.js +3 -0
- package/.next/server/chunks/ssr/[root-of-the-server]__1se25x0._.js +1 -1
- package/.next/server/chunks/ssr/{_1i3ddn7._.js → _05pl_u8._.js} +1 -1
- package/.next/server/chunks/ssr/_05rxyf9._.js +3 -0
- package/.next/server/chunks/ssr/_070nv1_._.js +3 -0
- package/.next/server/chunks/ssr/_097-ljd._.js +1 -1
- package/.next/server/chunks/ssr/{_10y6r1n._.js → _0bp-_fi._.js} +3 -3
- package/.next/server/chunks/ssr/_0fqhhwi._.js +1 -1
- package/.next/server/chunks/ssr/_0hlxij-._.js +3 -0
- package/.next/server/chunks/ssr/_0ld9_qq._.js +1 -1
- package/.next/server/chunks/ssr/{_0z8jpch._.js → _0p9pmxe._.js} +2 -2
- package/.next/server/chunks/ssr/{_1y-ea28._.js → _0s41irp._.js} +2 -2
- package/.next/server/chunks/ssr/_0tg0vf8._.js +164 -0
- package/.next/server/chunks/ssr/_0u5xv6g._.js +3 -0
- package/.next/server/chunks/ssr/{_1_rqu8x._.js → _12mijol._.js} +7 -7
- package/.next/server/chunks/ssr/_15zqt-t._.js +1 -1
- package/.next/server/chunks/ssr/{_1oijqjw._.js → _173siic._.js} +2 -2
- package/.next/server/chunks/ssr/_17v-33v._.js +20 -0
- package/.next/server/chunks/ssr/_184327j._.js +1 -1
- package/.next/server/chunks/ssr/{_1nod5hh._.js → _19n08g5._.js} +2 -2
- package/.next/server/chunks/ssr/_1alxym6._.js +3 -0
- package/.next/server/chunks/ssr/{_0hzz82u._.js → _1d423aj._.js} +25 -23
- package/.next/server/chunks/ssr/{_0ewory0._.js → _1doidrw._.js} +2 -2
- package/.next/server/chunks/ssr/{_0pm-2p-._.js → _1flsjeo._.js} +2 -2
- package/.next/server/chunks/ssr/_1mqn9mc._.js +1 -1
- package/.next/server/chunks/ssr/_1neoj22._.js +1 -1
- package/.next/server/chunks/ssr/_1ng831c._.js +3 -0
- package/.next/server/chunks/ssr/{_19g570d._.js → _1u2jw0i._.js} +3 -3
- package/.next/server/chunks/ssr/_1xoch6f._.js +1 -1
- package/.next/server/chunks/ssr/{_1u1mxu9._.js → _1yk6yak._.js} +2 -2
- package/.next/server/chunks/ssr/features_bot_1s_-2vj._.js +1 -1
- package/.next/server/chunks/ssr/features_bot_1yr-_pm._.js +3 -0
- package/.next/server/chunks/ssr/features_connectors_components_mcp-setting_tsx_0jmbgvp._.js +1 -1
- package/.next/server/chunks/ssr/features_memory_components_memory-setting_tsx_0oa43sw._.js +1 -1
- package/.next/server/chunks/ssr/features_routine_components_routine-setting_tsx_055fknz._.js +1 -1
- package/.next/server/chunks/ssr/{node_modules_1_8mv4v._.js → node_modules_048wmmj._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1p9rdsy._.js → node_modules_04t200t._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_03ptrl9._.js → node_modules_0511ha-._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_07ih6wb._.js → node_modules_08g12f-._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_01f8_se._.js → node_modules_0cdu6jq._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_111sh9o._.js → node_modules_0eb4rx7._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_10gbm2y._.js → node_modules_0f2k5l0._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1ix-g49._.js → node_modules_0piws2a._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0t-44lj._.js → node_modules_0pk6i-e._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0lg75qx._.js → node_modules_0s1lwc9._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1v9tc1c._.js → node_modules_0wq4-ts._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1m3gb16._.js → node_modules_0zjqwz7._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0b8df3s._.js → node_modules_11hbkuj._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_00jv9_3._.js → node_modules_18woe20._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1x825hz._.js → node_modules_1_afpz7._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0woscox._.js → node_modules_1_ban4u._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0gt-uw9._.js → node_modules_1cxc8kn._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_059vtf2._.js → node_modules_1dcbvfw._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1lsyy7k._.js → node_modules_1expacw._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1-7qawg._.js → node_modules_1ksh9gy._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_06w645b._.js → node_modules_1m2ho33._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0sz5yg1._.js → node_modules_1n0h9sf._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_1teyzfa._.js → node_modules_1qehmwm._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0yv-re6._.js → node_modules_1xzo2wn._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_20xmwtu._.js → node_modules_1zf46lp._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_0ii1l89._.js → node_modules_1zkpek5._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_@base-ui_react_0jne69c._.js → node_modules_@base-ui_react_0buz1-j._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_@mermaid-js_parser_dist_chunks_mermaid-parser_core_00q1oa3._.js → node_modules_@mermaid-js_parser_dist_chunks_mermaid-parser_core_0avl8-1._.js} +1 -1
- package/.next/server/chunks/ssr/{node_modules_@mermaid-js_parser_dist_chunks_mermaid-parser_core_1-2_5z7._.js → node_modules_@mermaid-js_parser_dist_chunks_mermaid-parser_core_14oslro._.js} +1 -1
- package/.next/server/middleware-build-manifest.js +3 -3
- package/.next/server/pages/404.html +1 -1
- package/.next/server/pages/500.html +1 -1
- package/.next/server/server-reference-manifest.js +1 -1
- package/.next/server/server-reference-manifest.json +57 -57
- package/.next/static/chunks/03kgfyrli_0gz.js +1 -0
- package/.next/static/chunks/{1e4yrzs5evq_r.js → 09120tms63y21.js} +1 -1
- package/.next/static/chunks/{2zhfdbq50ee5i.js → 0o980477kfqez.js} +1 -1
- package/.next/static/chunks/{1ocy4w0v20a3v.js → 0oz3rs47yy3p8.js} +2 -2
- package/.next/static/chunks/{2be08eqxsfc1y.js → 0w20wnnfrscq7.js} +1 -1
- package/.next/static/chunks/{09c13vnqy7al3.js → 19ojn4lomsk0r.js} +2 -2
- package/.next/static/chunks/{2om4lkg86feb4.js → 1mwee7xvitlp9.js} +1 -1
- package/.next/static/chunks/1p764d75_0lhv.js +18 -0
- package/.next/static/chunks/{0oz7a68qanm99.js → 1pjygp7hqib6j.js} +1 -1
- package/.next/static/chunks/22shnca_6zfzs.js +1 -0
- package/.next/static/chunks/{34s52v8f841zo.js → 24dst4mgurbsh.js} +1 -1
- package/.next/static/chunks/25zq8gzhbdsr7.js +1 -0
- package/.next/static/chunks/2d4z3er53ng8r.js +1 -0
- package/.next/static/chunks/{2d3yhjtafgmtw.js → 2ky81z-jgvasc.js} +1 -1
- package/.next/static/chunks/2m6jd3586ijb-.js +1 -0
- package/.next/static/chunks/{0k5tkt_qi52kb.js → 2ndw4w4u2x51r.js} +1 -1
- package/.next/static/chunks/2qod5tqlpz8b7.js +17 -0
- package/.next/static/chunks/{21gf2t_whhurc.js → 30skcjexa57zn.js} +2 -2
- package/.next/static/chunks/{2upc9mkcc3nlq.js → 35kwn237o4wdn.js} +1 -1
- package/.next/static/chunks/{2u9yjkpkoij75.js → 3exjsi8pfdvzj.js} +1 -1
- package/.next/static/chunks/3obf9n3cxj99k.js +1 -0
- package/.next/static/chunks/3qq4o8w2rv_e9.css +1 -0
- package/.next/static/chunks/{0g1p98s2pfx2w.js → 3vynnr71ghvom.js} +2 -2
- package/.next/static/chunks/{44kjmi5h_p8px.js → 42890jqk049q1.js} +1 -1
- package/README.md +35 -14
- package/bin/port.mjs +11 -7
- package/bin/thursday.mjs +1 -1
- package/components/ui/radio-group.tsx +1 -1
- package/components/ui/shiny-text.tsx +3 -22
- package/docs/how-it-works.md +11 -3
- package/docs/images/first-run.png +0 -0
- package/docs/images/keeps-going.png +0 -0
- package/docs/images/two-minds.png +0 -0
- package/features/ai/components/model-browser.tsx +1 -1
- package/features/ai/components/model-picker.tsx +1 -1
- package/features/ai/guide.ts +9 -4
- package/features/ai/prompts/bot.prompt.ts +4 -1
- package/features/ai/tools/routine.tool.ts +28 -16
- package/features/bot/bot.runner.ts +3 -1
- package/features/bot/components/bot-mark.tsx +6 -1
- package/features/bot/components/bot-setting.tsx +4 -6
- package/features/bot/components/bot-tool.tsx +2 -0
- package/features/bot/components/room-list.tsx +12 -3
- package/features/bot/components/room-pill.tsx +26 -19
- package/features/config/components/config-setting.tsx +17 -5
- package/features/config/components/voice-key.tsx +38 -18
- package/features/connectors/mcp.const.ts +90 -59
- package/features/intro/components/intro.tsx +50 -27
- package/features/routine/components/routine-mark.tsx +4 -6
- package/features/routine/components/routine-setting.tsx +332 -143
- package/features/routine/routine.query.ts +24 -3
- package/features/routine/routine.schema.ts +55 -4
- package/features/settings/components/install-app.tsx +1 -4
- package/features/settings/components/setting-ui.tsx +1 -1
- package/features/skills/skills.discover.ts +2 -1
- package/features/skills/skills.query.ts +5 -1
- package/features/skills/skills.schema.ts +2 -0
- package/features/thursday/components/ascii-orb.tsx +39 -10
- package/features/thursday/components/connect-wave.tsx +284 -0
- package/features/thursday/components/side-captions.tsx +29 -17
- package/features/thursday/components/thursday-setting.tsx +3 -3
- package/features/thursday/components/thursday.tsx +130 -58
- package/features/thursday/components/voice-picker.tsx +1 -1
- package/features/thursday/components/write-line.tsx +2 -2
- package/features/thursday/open-work.ts +11 -3
- package/features/thursday/tool-line.ts +94 -12
- package/features/thursday/use-text-call.ts +3 -2
- package/features/thursday/use-thursday.ts +5 -2
- package/features/workspace/components/artifact-view.tsx +129 -7
- package/guide/bots.md +17 -5
- package/guide/calls.md +15 -7
- package/guide/setup.md +10 -2
- package/knip.json +1 -0
- package/lib/desktop-notify.ts +3 -2
- package/package.json +1 -1
- package/scripts/bot-context.test.mts +1 -1
- package/seed-skills/analyst/README.md +19 -0
- package/seed-skills/analyst/data-report/SKILL.md +59 -0
- package/seed-skills/analyst/data-report/references/brief.md +31 -0
- package/seed-skills/analyst/data-report/references/company.md +41 -0
- package/seed-skills/analyst/data-report/references/comparison.md +42 -0
- package/seed-skills/analyst/data-report/references/decision.md +28 -0
- package/seed-skills/analyst/data-report/references/market-size.md +49 -0
- package/seed-skills/analyst/data-report/references/price-check.md +33 -0
- package/seed-skills/analyst/data-report/references/sources.md +51 -0
- package/seed-skills/analyst/data-report/references/trend.md +39 -0
- package/seed-skills/analyst/data-report/scripts/chart.mjs +679 -0
- package/seed-skills/analyst/data-report/scripts/fetch.mjs +521 -0
- package/seed-skills/insta/instagram/SKILL.md +33 -0
- package/seed-skills/insta/instagram/references/dm.md +49 -0
- package/seed-skills/insta/instagram/references/post.md +74 -0
- package/seed-skills/insta/instagram/scripts/compose.mjs +329 -0
- package/seed-skills/insta/instagram/scripts/image-size.mjs +30 -0
- package/seed-skills/insta/instagram/scripts/inbox.mjs +159 -0
- package/seed-skills/insta/instagram/scripts/lib.mjs +91 -0
- package/seed-skills/insta/instagram/scripts/media.mjs +104 -0
- package/seed-skills/insta/instagram/scripts/profile.mjs +139 -0
- package/seed-skills/insta/instagram/scripts/read.mjs +240 -0
- package/seed-skills/insta/instagram/scripts/render.mjs +126 -0
- package/seed-skills/insta/instagram/scripts/search.mjs +133 -0
- package/seed-skills/insta/instagram/scripts/selectors.mjs +56 -0
- package/seed-skills/insta/instagram/scripts/sheet.mjs +100 -0
- package/seed-skills/insta/instagram/scripts/webimage.mjs +105 -0
- package/seed-skills/insta/instagram/templates/carousel.html +62 -0
- package/seed-skills/mail/mail/SKILL.md +37 -0
- package/seed-skills/mail/mail/recipes/gmail.mjs +448 -0
- package/seed-skills/mail/mail/references/recipes.md +55 -0
- package/seed-skills/mail/mail/scripts/mail.mjs +202 -0
- package/seed-skills/marketer/README.md +6 -4
- package/seed-skills/marketer/copywriting/SKILL.md +1 -1
- package/seed-skills/marketer/emails/SKILL.md +1 -1
- package/seed-skills/marketer/launch/SKILL.md +1 -1
- package/seed-skills/marketer/product-marketing/SKILL.md +5 -5
- package/seed-skills/marketer/seo-audit/SKILL.md +1 -1
- package/seed-skills/marketer/social/SKILL.md +1 -1
- package/skills/computer/SKILL.md +1 -0
- package/skills/interactive-page/SKILL.md +4 -0
- package/.next/server/chunks/[root-of-the-server]__0n-l_n_._.js +0 -3
- package/.next/server/chunks/[root-of-the-server]__0wvo2of._.js +0 -3
- package/.next/server/chunks/[root-of-the-server]__1e6aut3._.js +0 -3
- package/.next/server/chunks/[root-of-the-server]__1ziojdq._.js +0 -3
- package/.next/server/chunks/ssr/[root-of-the-server]__1srp8l5._.js +0 -3
- package/.next/server/chunks/ssr/_0mes4s0._.js +0 -3
- package/.next/server/chunks/ssr/_0ya0cb0._.js +0 -3
- package/.next/server/chunks/ssr/_0yto5s7._.js +0 -20
- package/.next/server/chunks/ssr/_10dw2nv._.js +0 -3
- package/.next/server/chunks/ssr/_1p0gxeo._.js +0 -164
- package/.next/server/chunks/ssr/_1r41yvh._.js +0 -3
- package/.next/server/chunks/ssr/_1r79ksv._.js +0 -3
- package/.next/server/chunks/ssr/features_bot_1ogjt52._.js +0 -3
- package/.next/server/chunks/ssr/node_modules_@base-ui_react_0b3d4m9._.js +0 -3
- package/.next/static/chunks/03omwz_vi2ndg.js +0 -1
- package/.next/static/chunks/0_1bqgd6wt_og.js +0 -1
- package/.next/static/chunks/0abgkzikaznxc.js +0 -1
- package/.next/static/chunks/0bddkpash59-o.js +0 -17
- package/.next/static/chunks/0c2-bzhy3a0ib.js +0 -1
- package/.next/static/chunks/0iilfb1ocevnx.js +0 -1
- package/.next/static/chunks/0imbhx-j830a_.css +0 -1
- package/.next/static/chunks/24h1y9dhy3m_2.js +0 -1
- package/.next/static/chunks/2f_5saocsko_b.js +0 -1
- package/.next/static/chunks/2lwbr5vx0ixaj.js +0 -18
- /package/.next/static/{UiOMPYpQ4fqGyu5PzS-7j → Ar5V4kMdL4CE1trx8anrH}/_buildManifest.js +0 -0
- /package/.next/static/{UiOMPYpQ4fqGyu5PzS-7j → Ar5V4kMdL4CE1trx8anrH}/_clientMiddlewareManifest.js +0 -0
- /package/.next/static/{UiOMPYpQ4fqGyu5PzS-7j → Ar5V4kMdL4CE1trx8anrH}/_ssgManifest.js +0 -0
|
@@ -0,0 +1,521 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// Fetches a published series into one CSV: a `date` column, one column per series, and
|
|
3
|
+
// `#` lines above the header naming where it came from, so the page drawn from it can
|
|
4
|
+
// cite the source (chart.mjs reads them). Node only, no keys.
|
|
5
|
+
//
|
|
6
|
+
// node fetch.mjs fred <ID[,ID…]> FRED series (US and much of the OECD)
|
|
7
|
+
// node fetch.mjs worldbank <ISO2[,ISO2…]> <INDICATOR> World Bank, one column per country, yearly
|
|
8
|
+
// node fetch.mjs yahoo <TICKER[,TICKER…]> daily/weekly/monthly close (--interval 1d|1wk|1mo)
|
|
9
|
+
// node fetch.mjs fx <BASE> <QUOTE[,QUOTE…]> ECB reference rates, business days
|
|
10
|
+
// node fetch.mjs sec <TICKER> <CONCEPT[,CONCEPT…]> US filers' reported figures (--quarterly)
|
|
11
|
+
// node fetch.mjs sec <TICKER> --find <word> which concepts a company reports
|
|
12
|
+
// node fetch.mjs pageviews <ARTICLE[,ARTICLE…]> Wikipedia monthly views (--wiki en|ko|…)
|
|
13
|
+
//
|
|
14
|
+
// Every command takes --out <file.csv> (required, except --find), --from and --to
|
|
15
|
+
// (YYYY, YYYY-MM or YYYY-MM-DD) and --label <name[,name…]> to rename the columns.
|
|
16
|
+
import { mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
17
|
+
import { dirname } from "node:path";
|
|
18
|
+
|
|
19
|
+
class Stop extends Error {}
|
|
20
|
+
|
|
21
|
+
/** The comment at the top of this file, which is its manual. */
|
|
22
|
+
const usage = () => {
|
|
23
|
+
const lines = readFileSync(new URL(import.meta.url), "utf8")
|
|
24
|
+
.split("\n")
|
|
25
|
+
.slice(1);
|
|
26
|
+
const end = lines.findIndex((line) => !line.startsWith("//"));
|
|
27
|
+
return lines
|
|
28
|
+
.slice(0, end)
|
|
29
|
+
.map((line) => line.replace(/^\/\/ ?/, ""))
|
|
30
|
+
.join("\n");
|
|
31
|
+
};
|
|
32
|
+
|
|
33
|
+
// The SEC turns away a user agent carrying a URL; Wikimedia asks for one that names the tool
|
|
34
|
+
const AGENT = "thursday-agent data-report";
|
|
35
|
+
const today = new Date().toISOString().slice(0, 10);
|
|
36
|
+
|
|
37
|
+
function parseArgs(argv) {
|
|
38
|
+
const positional = [];
|
|
39
|
+
const flags = {};
|
|
40
|
+
for (let i = 0; i < argv.length; i++) {
|
|
41
|
+
const arg = argv[i];
|
|
42
|
+
if (arg.startsWith("--")) {
|
|
43
|
+
const key = arg.slice(2);
|
|
44
|
+
const next = argv[i + 1];
|
|
45
|
+
if (next === undefined || next.startsWith("--")) flags[key] = true;
|
|
46
|
+
else {
|
|
47
|
+
flags[key] = next;
|
|
48
|
+
i++;
|
|
49
|
+
}
|
|
50
|
+
} else positional.push(arg);
|
|
51
|
+
}
|
|
52
|
+
return { positional, flags };
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
const list = (value) =>
|
|
56
|
+
String(value ?? "")
|
|
57
|
+
.split(",")
|
|
58
|
+
.map((one) => one.trim())
|
|
59
|
+
.filter(Boolean);
|
|
60
|
+
|
|
61
|
+
/** YYYY, YYYY-MM or YYYY-MM-DD, padded to a full day at the start or end of the period. */
|
|
62
|
+
function day(value, end = false) {
|
|
63
|
+
if (!value) return null;
|
|
64
|
+
const m = String(value).match(/^(\d{4})(?:-(\d{2}))?(?:-(\d{2}))?$/);
|
|
65
|
+
if (!m)
|
|
66
|
+
throw new Stop(
|
|
67
|
+
`"${value}" is not a date: use YYYY, YYYY-MM or YYYY-MM-DD.`,
|
|
68
|
+
);
|
|
69
|
+
const [, y, mo, d] = m;
|
|
70
|
+
if (d) return `${y}-${mo}-${d}`;
|
|
71
|
+
if (mo) {
|
|
72
|
+
if (!end) return `${y}-${mo}-01`;
|
|
73
|
+
const last = new Date(Date.UTC(+y, +mo, 0)).getUTCDate();
|
|
74
|
+
return `${y}-${mo}-${String(last).padStart(2, "0")}`;
|
|
75
|
+
}
|
|
76
|
+
return end ? `${y}-12-31` : `${y}-01-01`;
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
async function get(url, as = "json", headers = {}) {
|
|
80
|
+
let last;
|
|
81
|
+
for (let attempt = 0; attempt < 2; attempt++) {
|
|
82
|
+
try {
|
|
83
|
+
const res = await fetch(url, {
|
|
84
|
+
headers,
|
|
85
|
+
signal: AbortSignal.timeout(45_000),
|
|
86
|
+
});
|
|
87
|
+
if (res.status === 404) return null;
|
|
88
|
+
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
|
89
|
+
return as === "json" ? await res.json() : await res.text();
|
|
90
|
+
} catch (error) {
|
|
91
|
+
last = error;
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
throw new Stop(
|
|
95
|
+
`${url} did not answer: ${last?.cause?.code ?? last?.message}`,
|
|
96
|
+
);
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
/** Joins series on their date: { name: Map(date → value) } → sorted rows. */
|
|
100
|
+
function join(series) {
|
|
101
|
+
const names = Object.keys(series);
|
|
102
|
+
const dates = new Set();
|
|
103
|
+
for (const values of Object.values(series))
|
|
104
|
+
for (const d of values.keys()) dates.add(d);
|
|
105
|
+
const rows = [...dates]
|
|
106
|
+
.sort()
|
|
107
|
+
.map((date) => [
|
|
108
|
+
date,
|
|
109
|
+
...names.map((name) => series[name].get(date) ?? ""),
|
|
110
|
+
]);
|
|
111
|
+
return { names, rows };
|
|
112
|
+
}
|
|
113
|
+
|
|
114
|
+
function clip(rows, from, to) {
|
|
115
|
+
return rows.filter(([date]) => {
|
|
116
|
+
const full = day(date.length === 4 ? date : date.slice(0, 10));
|
|
117
|
+
return (!from || full >= from) && (!to || full <= to);
|
|
118
|
+
});
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
const cell = (value) =>
|
|
122
|
+
/[",\n]/.test(String(value))
|
|
123
|
+
? `"${String(value).replaceAll('"', '""')}"`
|
|
124
|
+
: String(value);
|
|
125
|
+
|
|
126
|
+
// ── Sources ────────────────────────────────────────────────────────────────
|
|
127
|
+
|
|
128
|
+
async function fred([ids], { from, to }) {
|
|
129
|
+
if (!ids)
|
|
130
|
+
throw new Stop(
|
|
131
|
+
"Name the FRED series: fetch.mjs fred CPIAUCSL --out cpi.csv",
|
|
132
|
+
);
|
|
133
|
+
const series = {};
|
|
134
|
+
const apis = [];
|
|
135
|
+
for (const id of list(ids)) {
|
|
136
|
+
const url = `https://fred.stlouisfed.org/graph/fredgraph.csv?id=${encodeURIComponent(id)}${from ? `&cosd=${from}` : ""}${to ? `&coed=${to}` : ""}`;
|
|
137
|
+
apis.push(url);
|
|
138
|
+
// FRED refuses a browser's user agent on this endpoint; Node's own passes
|
|
139
|
+
const text = await get(url, "text");
|
|
140
|
+
if (!text || !text.startsWith("observation_date"))
|
|
141
|
+
throw new Stop(
|
|
142
|
+
`FRED has no series "${id}". Find the id on https://fred.stlouisfed.org (search, then the id after /series/).`,
|
|
143
|
+
);
|
|
144
|
+
const values = new Map();
|
|
145
|
+
for (const line of text.trim().split("\n").slice(1)) {
|
|
146
|
+
const [date, value] = line.split(",");
|
|
147
|
+
values.set(date, value === "." ? "" : (value ?? "").trim());
|
|
148
|
+
}
|
|
149
|
+
series[id] = values;
|
|
150
|
+
}
|
|
151
|
+
const ids_ = list(ids);
|
|
152
|
+
return {
|
|
153
|
+
series,
|
|
154
|
+
title: ids_.join(", "),
|
|
155
|
+
source: ids_
|
|
156
|
+
.map((id) => `https://fred.stlouisfed.org/series/${id}`)
|
|
157
|
+
.join(" "),
|
|
158
|
+
api: apis.join(" "),
|
|
159
|
+
};
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
async function worldbank([countries, indicator], { from, to }) {
|
|
163
|
+
if (!countries || !indicator)
|
|
164
|
+
throw new Stop(
|
|
165
|
+
"Name countries and an indicator: fetch.mjs worldbank KR,JP NY.GDP.MKTP.CD --out gdp.csv",
|
|
166
|
+
);
|
|
167
|
+
const codes = list(countries).join(";");
|
|
168
|
+
const years =
|
|
169
|
+
from || to
|
|
170
|
+
? `&date=${(from ?? "1960").slice(0, 4)}:${(to ?? today).slice(0, 4)}`
|
|
171
|
+
: "";
|
|
172
|
+
const api = `https://api.worldbank.org/v2/country/${codes}/indicator/${indicator}?format=json&per_page=20000${years}`;
|
|
173
|
+
const data = await get(api);
|
|
174
|
+
if (!Array.isArray(data) || !Array.isArray(data[1]))
|
|
175
|
+
throw new Stop(
|
|
176
|
+
`The World Bank answered no rows for ${codes} / ${indicator}: ${JSON.stringify(data?.[0]?.message ?? data).slice(0, 200)}`,
|
|
177
|
+
);
|
|
178
|
+
const series = {};
|
|
179
|
+
let title = indicator;
|
|
180
|
+
for (const row of data[1]) {
|
|
181
|
+
title = row.indicator?.value ?? title;
|
|
182
|
+
const name = row.country?.value ?? row.countryiso3code;
|
|
183
|
+
series[name] ??= new Map();
|
|
184
|
+
if (row.value !== null) series[name].set(row.date, String(row.value));
|
|
185
|
+
}
|
|
186
|
+
return {
|
|
187
|
+
series,
|
|
188
|
+
title,
|
|
189
|
+
source: `https://data.worldbank.org/indicator/${indicator}?locations=${list(countries).join("-")}`,
|
|
190
|
+
api,
|
|
191
|
+
};
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
async function yahoo([tickers], { from, to, flags }) {
|
|
195
|
+
if (!tickers)
|
|
196
|
+
throw new Stop(
|
|
197
|
+
"Name the ticker: fetch.mjs yahoo AAPL,005930.KS --out prices.csv",
|
|
198
|
+
);
|
|
199
|
+
const interval = flags.interval ?? "1d";
|
|
200
|
+
const start = Math.floor(
|
|
201
|
+
Date.parse(from ?? day(String(new Date().getUTCFullYear() - 1))) / 1000,
|
|
202
|
+
);
|
|
203
|
+
const end = Math.floor(
|
|
204
|
+
(to ? Date.parse(to) + 86_400_000 : Date.now()) / 1000,
|
|
205
|
+
);
|
|
206
|
+
const series = {};
|
|
207
|
+
const names = [];
|
|
208
|
+
const apis = [];
|
|
209
|
+
const units = new Set();
|
|
210
|
+
for (const ticker of list(tickers)) {
|
|
211
|
+
const api = `https://query1.finance.yahoo.com/v8/finance/chart/${encodeURIComponent(ticker)}?period1=${start}&period2=${end}&interval=${interval}&events=split,div`;
|
|
212
|
+
apis.push(api);
|
|
213
|
+
const data = await get(api);
|
|
214
|
+
const result = data?.chart?.result?.[0];
|
|
215
|
+
if (!result?.timestamp)
|
|
216
|
+
throw new Stop(
|
|
217
|
+
`Yahoo has no prices for "${ticker}" (${data?.chart?.error?.description ?? "no rows"}). A Korean listing ends in .KS (KOSPI) or .KQ (KOSDAQ); an index starts with ^.`,
|
|
218
|
+
);
|
|
219
|
+
const meta = result.meta;
|
|
220
|
+
const close =
|
|
221
|
+
result.indicators.adjclose?.[0]?.adjclose ??
|
|
222
|
+
result.indicators.quote[0].close;
|
|
223
|
+
const values = new Map();
|
|
224
|
+
result.timestamp.forEach((t, i) => {
|
|
225
|
+
if (close[i] === null || close[i] === undefined) return;
|
|
226
|
+
const date = new Date((t + (meta.gmtoffset ?? 0)) * 1000)
|
|
227
|
+
.toISOString()
|
|
228
|
+
.slice(0, 10);
|
|
229
|
+
values.set(date, String(+close[i].toFixed(4)));
|
|
230
|
+
});
|
|
231
|
+
series[ticker] = values;
|
|
232
|
+
names.push(`${ticker} (${meta.longName ?? meta.shortName ?? ticker})`);
|
|
233
|
+
if (meta.currency) units.add(meta.currency);
|
|
234
|
+
}
|
|
235
|
+
return {
|
|
236
|
+
series,
|
|
237
|
+
title: `${names.join(", ")}, ${interval === "1d" ? "daily" : interval === "1wk" ? "weekly" : "monthly"} close adjusted for splits and dividends`,
|
|
238
|
+
unit: [...units].join(", "),
|
|
239
|
+
source: list(tickers)
|
|
240
|
+
.map(
|
|
241
|
+
(t) =>
|
|
242
|
+
`https://finance.yahoo.com/quote/${encodeURIComponent(t)}/history`,
|
|
243
|
+
)
|
|
244
|
+
.join(" "),
|
|
245
|
+
api: apis.join(" "),
|
|
246
|
+
};
|
|
247
|
+
}
|
|
248
|
+
|
|
249
|
+
async function fx([base, quotes], { from, to }) {
|
|
250
|
+
if (!base || !quotes)
|
|
251
|
+
throw new Stop(
|
|
252
|
+
"Name the currencies: fetch.mjs fx USD KRW,JPY --out fx.csv",
|
|
253
|
+
);
|
|
254
|
+
const start = from ?? day(String(new Date().getUTCFullYear() - 1));
|
|
255
|
+
const api = `https://api.frankfurter.dev/v1/${start}..${to ?? ""}?base=${base.toUpperCase()}&symbols=${list(quotes).join(",").toUpperCase()}`;
|
|
256
|
+
const data = await get(api);
|
|
257
|
+
if (!data?.rates)
|
|
258
|
+
throw new Stop(
|
|
259
|
+
`No rates for ${base} → ${quotes}. Only the ECB's ~30 currencies are here: https://api.frankfurter.dev/v1/currencies`,
|
|
260
|
+
);
|
|
261
|
+
const series = {};
|
|
262
|
+
for (const [date, rates] of Object.entries(data.rates))
|
|
263
|
+
for (const [quote, value] of Object.entries(rates)) {
|
|
264
|
+
series[`${base.toUpperCase()}/${quote}`] ??= new Map();
|
|
265
|
+
series[`${base.toUpperCase()}/${quote}`].set(date, String(value));
|
|
266
|
+
}
|
|
267
|
+
return {
|
|
268
|
+
series,
|
|
269
|
+
title: `${list(quotes)
|
|
270
|
+
.map((q) => `${base.toUpperCase()}/${q.toUpperCase()}`)
|
|
271
|
+
.join(
|
|
272
|
+
", ",
|
|
273
|
+
)}, ECB reference rate (units of the quote per one ${base.toUpperCase()})`,
|
|
274
|
+
source:
|
|
275
|
+
"https://www.ecb.europa.eu/stats/policy_and_exchange_rates/euro_reference_exchange_rates/html/index.en.html",
|
|
276
|
+
api,
|
|
277
|
+
};
|
|
278
|
+
}
|
|
279
|
+
|
|
280
|
+
async function secCompany(ticker) {
|
|
281
|
+
const headers = { "User-Agent": AGENT };
|
|
282
|
+
if (/^\d+$/.test(ticker))
|
|
283
|
+
return { cik: ticker.padStart(10, "0"), name: ticker, headers };
|
|
284
|
+
const map = await get(
|
|
285
|
+
"https://www.sec.gov/files/company_tickers.json",
|
|
286
|
+
"json",
|
|
287
|
+
headers,
|
|
288
|
+
);
|
|
289
|
+
const hit = Object.values(map ?? {}).find(
|
|
290
|
+
(c) => c.ticker.toUpperCase() === ticker.toUpperCase(),
|
|
291
|
+
);
|
|
292
|
+
if (!hit)
|
|
293
|
+
throw new Stop(
|
|
294
|
+
`"${ticker}" is not a US filer's ticker on the SEC's list. Pass its CIK instead, or take the figures from its own filings.`,
|
|
295
|
+
);
|
|
296
|
+
return {
|
|
297
|
+
cik: String(hit.cik_str).padStart(10, "0"),
|
|
298
|
+
name: hit.title,
|
|
299
|
+
headers,
|
|
300
|
+
};
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
async function sec([ticker, concepts], { from, to, flags }) {
|
|
304
|
+
if (!ticker)
|
|
305
|
+
throw new Stop(
|
|
306
|
+
"Name the company: fetch.mjs sec AAPL Revenues --out revenue.csv",
|
|
307
|
+
);
|
|
308
|
+
const { cik, name, headers } = await secCompany(ticker);
|
|
309
|
+
if (flags.find) {
|
|
310
|
+
const facts = await get(
|
|
311
|
+
`https://data.sec.gov/api/xbrl/companyfacts/CIK${cik}.json`,
|
|
312
|
+
"json",
|
|
313
|
+
headers,
|
|
314
|
+
);
|
|
315
|
+
const word = String(flags.find).toLowerCase();
|
|
316
|
+
const hits = [];
|
|
317
|
+
for (const [taxonomy, group] of Object.entries(facts?.facts ?? {}))
|
|
318
|
+
for (const [concept, fact] of Object.entries(group)) {
|
|
319
|
+
if (!`${concept} ${fact.label}`.toLowerCase().includes(word)) continue;
|
|
320
|
+
const rows = Object.values(fact.units)
|
|
321
|
+
.flat()
|
|
322
|
+
.filter((r) => r.fp === "FY" && /^CY\d{4}$/.test(r.frame ?? ""));
|
|
323
|
+
const lastRow = rows.at(-1);
|
|
324
|
+
if (lastRow)
|
|
325
|
+
hits.push(
|
|
326
|
+
`${taxonomy === "us-gaap" ? "" : `${taxonomy}:`}${concept} — ${fact.label}; latest ${lastRow.frame} ${lastRow.val}`,
|
|
327
|
+
);
|
|
328
|
+
}
|
|
329
|
+
hits.sort((a, b) => b.localeCompare(a));
|
|
330
|
+
console.log(
|
|
331
|
+
hits.length
|
|
332
|
+
? hits.join("\n")
|
|
333
|
+
: `No yearly concept of ${name} mentions "${flags.find}".`,
|
|
334
|
+
);
|
|
335
|
+
return null;
|
|
336
|
+
}
|
|
337
|
+
if (!concepts)
|
|
338
|
+
throw new Stop(
|
|
339
|
+
`Name the concept, or look for it: fetch.mjs sec ${ticker} --find revenue`,
|
|
340
|
+
);
|
|
341
|
+
const quarterly = Boolean(flags.quarterly);
|
|
342
|
+
const derived = new Set();
|
|
343
|
+
const series = {};
|
|
344
|
+
const apis = [];
|
|
345
|
+
const units = new Set();
|
|
346
|
+
for (const full of list(concepts)) {
|
|
347
|
+
const [taxonomy, concept] = full.includes(":")
|
|
348
|
+
? full.split(":")
|
|
349
|
+
: ["us-gaap", full];
|
|
350
|
+
const api = `https://data.sec.gov/api/xbrl/companyconcept/CIK${cik}/${taxonomy}/${concept}.json`;
|
|
351
|
+
apis.push(api);
|
|
352
|
+
const data = await get(api, "json", headers);
|
|
353
|
+
if (!data)
|
|
354
|
+
throw new Stop(
|
|
355
|
+
`${name} reports no "${full}". Look for the one it uses: fetch.mjs sec ${ticker} --find <word>`,
|
|
356
|
+
);
|
|
357
|
+
const [unit, rows] = Object.entries(data.units)[0];
|
|
358
|
+
units.add(unit);
|
|
359
|
+
// A period is a year or a quarter by its length, not by the filing's own labels; a later
|
|
360
|
+
// filing's figure for the same period wins, since it carries any restatement
|
|
361
|
+
const values = new Map();
|
|
362
|
+
const filed = new Map();
|
|
363
|
+
for (const row of rows) {
|
|
364
|
+
if (!/^(10-K|10-Q|20-F|40-F)/.test(row.form ?? "")) continue;
|
|
365
|
+
if (row.start) {
|
|
366
|
+
const days = (Date.parse(row.end) - Date.parse(row.start)) / 86_400_000;
|
|
367
|
+
if (quarterly ? days < 80 || days > 100 : days < 350 || days > 380)
|
|
368
|
+
continue;
|
|
369
|
+
} else if (!quarterly && row.fp !== "FY") continue;
|
|
370
|
+
if ((filed.get(row.end) ?? "") > row.filed) continue;
|
|
371
|
+
filed.set(row.end, row.filed);
|
|
372
|
+
values.set(row.end, String(row.val));
|
|
373
|
+
}
|
|
374
|
+
// An annual report states the year, not its last quarter: that quarter is the year
|
|
375
|
+
// less the three before it
|
|
376
|
+
if (quarterly)
|
|
377
|
+
for (const year of rows) {
|
|
378
|
+
if (!year.start || !/^(10-K|20-F|40-F)/.test(year.form ?? "")) continue;
|
|
379
|
+
const days =
|
|
380
|
+
(Date.parse(year.end) - Date.parse(year.start)) / 86_400_000;
|
|
381
|
+
if (days < 350 || days > 380 || values.has(year.end)) continue;
|
|
382
|
+
const inside = [...values.entries()].filter(
|
|
383
|
+
([end]) => end > year.start && end < year.end,
|
|
384
|
+
);
|
|
385
|
+
if (inside.length !== 3) continue;
|
|
386
|
+
values.set(
|
|
387
|
+
year.end,
|
|
388
|
+
String(year.val - inside.reduce((sum, [, v]) => sum + Number(v), 0)),
|
|
389
|
+
);
|
|
390
|
+
if ((!from || year.end >= from) && (!to || year.end <= to))
|
|
391
|
+
derived.add(year.end);
|
|
392
|
+
}
|
|
393
|
+
series[concept] = values;
|
|
394
|
+
}
|
|
395
|
+
return {
|
|
396
|
+
series,
|
|
397
|
+
title: `${name}, ${list(concepts).join(", ")} as reported to the SEC, ${quarterly ? "quarterly" : "fiscal years"} (dated by period end)`,
|
|
398
|
+
unit: [...units].join(", "),
|
|
399
|
+
note: derived.size
|
|
400
|
+
? `The quarters ending ${[...derived].sort().join(", ")} are the year less its first three quarters`
|
|
401
|
+
: null,
|
|
402
|
+
source: `https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=${cik}&type=10-K`,
|
|
403
|
+
api: apis.join(" "),
|
|
404
|
+
};
|
|
405
|
+
}
|
|
406
|
+
|
|
407
|
+
async function pageviews([articles], { from, to, flags }) {
|
|
408
|
+
if (!articles)
|
|
409
|
+
throw new Stop(
|
|
410
|
+
"Name the article: fetch.mjs pageviews Bitcoin,Ethereum --out interest.csv",
|
|
411
|
+
);
|
|
412
|
+
const wiki = flags.wiki ?? "en";
|
|
413
|
+
const start = (
|
|
414
|
+
from ?? day(String(new Date().getUTCFullYear() - 2))
|
|
415
|
+
).replaceAll("-", "");
|
|
416
|
+
// A month still under way reads as a fall; it ends at the last whole month
|
|
417
|
+
const lastWhole = new Date(
|
|
418
|
+
Date.UTC(new Date().getUTCFullYear(), new Date().getUTCMonth(), 0),
|
|
419
|
+
)
|
|
420
|
+
.toISOString()
|
|
421
|
+
.slice(0, 10);
|
|
422
|
+
const end = (to && to < lastWhole ? to : lastWhole).replaceAll("-", "");
|
|
423
|
+
const series = {};
|
|
424
|
+
const apis = [];
|
|
425
|
+
for (const article of list(articles)) {
|
|
426
|
+
const title = article.replaceAll(" ", "_");
|
|
427
|
+
const api = `https://wikimedia.org/api/rest_v1/metrics/pageviews/per-article/${wiki}.wikipedia/all-access/user/${encodeURIComponent(title)}/monthly/${start}/${end}`;
|
|
428
|
+
apis.push(api);
|
|
429
|
+
const data = await get(api, "json", { "User-Agent": AGENT });
|
|
430
|
+
if (!data?.items)
|
|
431
|
+
throw new Stop(
|
|
432
|
+
`${wiki}.wikipedia has no article "${article}" (the exact page title, as in its URL).`,
|
|
433
|
+
);
|
|
434
|
+
series[article] = new Map(
|
|
435
|
+
data.items.map((item) => [
|
|
436
|
+
`${item.timestamp.slice(0, 4)}-${item.timestamp.slice(4, 6)}`,
|
|
437
|
+
String(item.views),
|
|
438
|
+
]),
|
|
439
|
+
);
|
|
440
|
+
}
|
|
441
|
+
return {
|
|
442
|
+
series,
|
|
443
|
+
title: `Monthly views of ${list(articles).join(", ")} on ${wiki}.wikipedia, people only — a measure of interest`,
|
|
444
|
+
source: `https://pageviews.wmcloud.org/?project=${wiki}.wikipedia.org&pages=${list(
|
|
445
|
+
articles,
|
|
446
|
+
)
|
|
447
|
+
.map((a) => encodeURIComponent(a.replaceAll(" ", "_")))
|
|
448
|
+
.join("|")}`,
|
|
449
|
+
api: apis.join(" "),
|
|
450
|
+
};
|
|
451
|
+
}
|
|
452
|
+
|
|
453
|
+
const SOURCES = { fred, worldbank, yahoo, fx, sec, pageviews };
|
|
454
|
+
|
|
455
|
+
// ── Main ───────────────────────────────────────────────────────────────────
|
|
456
|
+
|
|
457
|
+
async function main() {
|
|
458
|
+
const [command, ...rest] = process.argv.slice(2);
|
|
459
|
+
const { positional, flags } = parseArgs(rest);
|
|
460
|
+
const source = SOURCES[command];
|
|
461
|
+
if (!source) throw new Stop(usage());
|
|
462
|
+
const from = day(flags.from);
|
|
463
|
+
const to = day(flags.to, true);
|
|
464
|
+
const got = await source(positional, { from, to, flags });
|
|
465
|
+
if (!got) return;
|
|
466
|
+
if (typeof flags.out !== "string")
|
|
467
|
+
throw new Stop("Say where the CSV goes: --out <file.csv>");
|
|
468
|
+
|
|
469
|
+
const { names, rows: all } = join(got.series);
|
|
470
|
+
const rows = clip(all, from, to);
|
|
471
|
+
if (!rows.length)
|
|
472
|
+
throw new Stop(
|
|
473
|
+
`No rows between ${from ?? "the start"} and ${to ?? "today"}; the source has ${all.length ? `${all[0][0]} → ${all.at(-1)[0]}` : "none"}. A series that stops is discontinued: look for its successor.`,
|
|
474
|
+
);
|
|
475
|
+
const labels = list(flags.label);
|
|
476
|
+
const header = ["date", ...names.map((name, i) => labels[i] ?? name)];
|
|
477
|
+
|
|
478
|
+
const lines = [
|
|
479
|
+
`# title: ${got.title}`,
|
|
480
|
+
...(got.unit ? [`# unit: ${got.unit}`] : []),
|
|
481
|
+
...(got.note ? [`# note: ${got.note}`] : []),
|
|
482
|
+
`# source: ${got.source}`,
|
|
483
|
+
`# api: ${got.api}`,
|
|
484
|
+
`# fetched: ${today}`,
|
|
485
|
+
header.map(cell).join(","),
|
|
486
|
+
...rows.map((row) => row.map(cell).join(",")),
|
|
487
|
+
];
|
|
488
|
+
mkdirSync(dirname(flags.out), { recursive: true });
|
|
489
|
+
writeFileSync(flags.out, `${lines.join("\n")}\n`);
|
|
490
|
+
|
|
491
|
+
// What the model needs to go on without opening the file: its shape and both ends
|
|
492
|
+
const missing = rows.reduce(
|
|
493
|
+
(n, row) => n + row.slice(1).filter((v) => v === "").length,
|
|
494
|
+
0,
|
|
495
|
+
);
|
|
496
|
+
const summary = [
|
|
497
|
+
`Wrote ${flags.out}: ${rows.length} rows × ${names.length} series, ${rows[0][0]} → ${rows.at(-1)[0]}${missing ? `, ${missing} empty cells (no value published)` : ""}.`,
|
|
498
|
+
`${got.title}${got.unit ? ` [${got.unit}]` : ""}`,
|
|
499
|
+
...(got.note ? [`Note: ${got.note}`] : []),
|
|
500
|
+
`first: ${header
|
|
501
|
+
.slice(1)
|
|
502
|
+
.map(
|
|
503
|
+
(h, i) => `${h}=${rows.find((r) => r[i + 1] !== "")?.[i + 1] ?? "—"}`,
|
|
504
|
+
)
|
|
505
|
+
.join(", ")}`,
|
|
506
|
+
`last: ${header
|
|
507
|
+
.slice(1)
|
|
508
|
+
.map(
|
|
509
|
+
(h, i) =>
|
|
510
|
+
`${h}=${rows.findLast((r) => r[i + 1] !== "")?.[i + 1] ?? "—"} (${rows.findLast((r) => r[i + 1] !== "")?.[0] ?? "—"})`,
|
|
511
|
+
)
|
|
512
|
+
.join(", ")}`,
|
|
513
|
+
];
|
|
514
|
+
console.log(summary.join("\n"));
|
|
515
|
+
}
|
|
516
|
+
|
|
517
|
+
main().catch((error) => {
|
|
518
|
+
if (!(error instanceof Stop)) throw error;
|
|
519
|
+
console.error(error.message);
|
|
520
|
+
process.exitCode = 1;
|
|
521
|
+
});
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: instagram
|
|
3
|
+
description: "Instagram by script instead of snapshots: read DMs, study reference accounts and search results, render slides at exact size, and fill the post composer up to Share."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Instagram
|
|
7
|
+
|
|
8
|
+
A snapshot of an Instagram page runs 10-25k characters and a DM thread takes one per
|
|
9
|
+
screen. These scripts read the same pages through your own browser session and print a
|
|
10
|
+
few lines, so run them first and snapshot only when one tells you to. Each is
|
|
11
|
+
`node <this skill's dir>/scripts/<name>.mjs`; chain them with `&&` in one bash call.
|
|
12
|
+
|
|
13
|
+
Sign in before any of them: `sign_in_use` with `instagram.com`, then `playwright-cli goto
|
|
14
|
+
https://www.instagram.com/`. A script that says "Not signed in" means that step is missing.
|
|
15
|
+
|
|
16
|
+
**Opening a DM thread marks it seen.** `inbox` never opens one; `read` does.
|
|
17
|
+
|
|
18
|
+
| Script | Does |
|
|
19
|
+
|---|---|
|
|
20
|
+
| `inbox.mjs [--folder primary\|general\|requests] [--unread]` | Threads without opening them: unread mark, name, last line, time, id |
|
|
21
|
+
| `read.mjs <thread id> --out <file.jsonl> [--since YYYY-MM-DD] [--max 200]` | One thread to a file, oldest first; prints a summary and the last 10 |
|
|
22
|
+
| `search.mjs "<topic>" [--type reel\|carousel\|image] [--sheet covers.png]` | What Instagram's search shows, with likes, and the accounts behind it |
|
|
23
|
+
| `profile.mjs <handle> [--posts 12] [--out posts.json] [--sheet covers.png]` | How an account posts: type, slides, ratio, caption and tags per post, and the pattern |
|
|
24
|
+
| `sheet.mjs --out sheet.png <posts.json> [--post N]` | Covers, or one post's slides, as one image to `look_at` once |
|
|
25
|
+
| `webimage.mjs <page url> --out <dir> [--all]` | A news page's own picture, saved, with the credit line |
|
|
26
|
+
| `render.mjs <slides.html> --size 1080x1350 --out <dir>` | Each `[data-slide]` to a PNG of exactly that size |
|
|
27
|
+
| `compose.mjs --ratio 4:5 --caption <file> [--ai-label] [--shot last.png] <png>...` | The composer up to its last screen, crop set; never presses Share. `--discard` throws a draft away |
|
|
28
|
+
|
|
29
|
+
A script that stops with "Selectors changed" means Instagram moved something: snapshot that
|
|
30
|
+
page once, fix the line in `scripts/selectors.mjs`, and run it again.
|
|
31
|
+
|
|
32
|
+
- DMs, reading a thread cheaply, and pictures in messages: `references/dm.md`
|
|
33
|
+
- Studying reference accounts and building a post the way they do: `references/post.md`
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# DMs
|
|
2
|
+
|
|
3
|
+
## Look before you open
|
|
4
|
+
|
|
5
|
+
`inbox.mjs` reads the thread list from the inbox page: a `*` marks an unread thread, and
|
|
6
|
+
the last field is the id `read.mjs` takes. General and Requests are `--folder general` and
|
|
7
|
+
`--folder requests`. Nothing is opened, so nothing is marked seen.
|
|
8
|
+
|
|
9
|
+
Open a thread only when the job is to read it. An unread thread the user has not asked
|
|
10
|
+
about stays unread.
|
|
11
|
+
|
|
12
|
+
## Reading a thread
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
node <skill>/scripts/read.mjs 1234567890 --out <scratch>/dm-1234567890.jsonl --since 2026-09-01
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
`<scratch>` is this job's scratch folder.
|
|
19
|
+
|
|
20
|
+
It walks the message pane up from the newest message a screen at a time — the pane drops
|
|
21
|
+
rows that leave the screen, so it is never scrolled with the mouse wheel or jumped to the
|
|
22
|
+
top — and stops at `--max`, at `--since`, at the top of the thread, or after two rounds
|
|
23
|
+
with nothing new. It prints one summary line, then the last 10 messages (`--tail 0` for
|
|
24
|
+
none). A thread of 50 messages takes about 20 seconds.
|
|
25
|
+
|
|
26
|
+
Each line of the file is one message:
|
|
27
|
+
`{ id, ts, at, from: "me" | "them", sender, text, images: [url], video, links: [url] }`.
|
|
28
|
+
Read it with `grep`, `tail`, `sed -n`, or `node -e`, not all at once. `sender` is the
|
|
29
|
+
sender's account number; in a group, the same number is the same person.
|
|
30
|
+
|
|
31
|
+
A shared post or reel is a message whose `links` holds its url and whose `text` holds its
|
|
32
|
+
caption as shown.
|
|
33
|
+
|
|
34
|
+
## Pictures in messages
|
|
35
|
+
|
|
36
|
+
`images` are the picture urls; they expire after a while, so use them in the same job.
|
|
37
|
+
When what a picture shows matters, put them on one sheet and look once:
|
|
38
|
+
|
|
39
|
+
```bash
|
|
40
|
+
node <skill>/scripts/sheet.mjs --out <scratch>/dm-pictures.png <url> <url> ...
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
A screenshot of the thread is the fallback when a message holds something that is neither
|
|
44
|
+
text nor a picture url.
|
|
45
|
+
|
|
46
|
+
## When something covers the page
|
|
47
|
+
|
|
48
|
+
A dialog (notifications, a daily time-limit screen) can cover the inbox; `inbox` then says
|
|
49
|
+
so. Snapshot once, close it with its own close button, and run the script again.
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# A post, made the way the accounts that do well make it
|
|
2
|
+
|
|
3
|
+
## 1. Find the reference accounts
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
node <skill>/scripts/search.mjs "<niche word>" --out <scratch>/search.json
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
One word or a `#tag` finds more than a phrase. The last line lists the accounts behind the
|
|
10
|
+
results by the likes they drew. For the three or four that fit the niche:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
node <skill>/scripts/profile.mjs <handle> --out <scratch>/<handle>.json --sheet <scratch>/<handle>.png
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Posts marked `pinned` are old ones kept on top; judge the pattern by the rest.
|
|
17
|
+
|
|
18
|
+
## 2. Study how they post
|
|
19
|
+
|
|
20
|
+
`profile` prints the numbers: types, ratio, slides per carousel, caption length, hashtag
|
|
21
|
+
count, cadence, and which posts did best. What the numbers cannot tell is how the slides
|
|
22
|
+
look — `look_at` the covers sheet once, then the best post's slides:
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
node <skill>/scripts/sheet.mjs --out <scratch>/best.png <scratch>/<handle>.json --post 3
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Write the pattern down before building, one line each: ratio and slide count; the cover
|
|
29
|
+
(hook length, type size and weight, where the text sits, the photo or not); the inner
|
|
30
|
+
slides (text alone, photo with a line, photo over its own blur); the background treatment
|
|
31
|
+
(darkened, blurred, a gradient at the bottom, a flat colour); the source credit; the
|
|
32
|
+
caption (first line, body, call to action, hashtags). Keep it in your memory with the
|
|
33
|
+
accounts and the date, and read it next time instead of studying again.
|
|
34
|
+
|
|
35
|
+
## 3. Pictures
|
|
36
|
+
|
|
37
|
+
- **Something that happened**: the real picture. `webimage.mjs <article url> --out <post dir>`
|
|
38
|
+
saves the page's own picture and prints the credit line; `--all` adds the large pictures
|
|
39
|
+
in the article. A person in the story: their photo from the article or an official page.
|
|
40
|
+
Put the credit on the slide.
|
|
41
|
+
- **An idea or a mood**: `generate_image` from the studio. It draws 1:1, 3:4, 4:3, 9:16 or
|
|
42
|
+
16:9 and saves to your artifacts; take the closest ratio (3:4 for a 4:5 slide), copy the
|
|
43
|
+
file into the post folder, and let the slide crop it (`object-fit: cover`). A drawn
|
|
44
|
+
picture is never passed off as the event. Posting one that looks real takes `--ai-label`.
|
|
45
|
+
|
|
46
|
+
## 4. Slides
|
|
47
|
+
|
|
48
|
+
Copy `<skill>/templates/carousel.html` into the post folder beside the pictures. Set
|
|
49
|
+
`--w`/`--h` to the size, keep one `<section data-slide>` per slide, and write each slide
|
|
50
|
+
from the pattern. Feed is 1080x1350 (4:5) or 1080x1080 (1:1); every slide of a carousel is
|
|
51
|
+
the same size.
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
node <skill>/scripts/render.mjs <post dir>/carousel.html --size 1080x1350 --out <post dir>/png \
|
|
55
|
+
&& node <skill>/scripts/sheet.mjs --out <post dir>/check.png <post dir>/png/*.png
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
`look_at` the check sheet once: text inside the slide, readable over the picture. Fonts come
|
|
59
|
+
from the network; a slide in a language the machine has no font for renders as boxes.
|
|
60
|
+
|
|
61
|
+
## 5. The composer
|
|
62
|
+
|
|
63
|
+
Write the caption to a file, then:
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
node <skill>/scripts/compose.mjs --ratio 4:5 --caption <post dir>/caption.txt --shot <post dir>/last.png <post dir>/png/*.png
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
It stops on the last screen with the draft open and prints what it set; `last.png` shows it.
|
|
70
|
+
The web composer has no music for a photo post. To post: snapshot, and click Share, the
|
|
71
|
+
link at the right of that screen's header. Not posting: `compose.mjs --discard`.
|
|
72
|
+
|
|
73
|
+
Once it is up, `profile.mjs <the account's handle> --posts 1` shows the new post's ratio and
|
|
74
|
+
size: a ratio other than the one built means it went up cut.
|