vlr-stats-scraper 1.5.0 → 2.0.0

package/README.md

@@ -1,189 +1,326 @@
-# vlr-stats-scraper
-
-Scrapes player stats from [vlr.gg](https://www.vlr.gg) and stores them in a local SQLite
-database, grouped by tournament or stage, so you can pull a backlog of players.
-
-## Setup
-
-Requires Node.js 18+ and a [Hyperbrowser](https://hyperbrowser.ai) API key for scraping.
-
-```bash
-npm install vlr-stats-scraper
-```
-
-Create a `.env` file in the directory you run it from:
-
-```
-HYPERBROWSER_API_KEY=your_key_here
-```
-
-## CLI usage
-
-The package installs a `valstats` command.
-
-| Command | What it does |
-| --- | --- |
-| `valstats <link> <year> <event-name>` | Scrape every player on an event page and save them under that event |
-| `valstats --player <name \| link>` | Print the stats of every saved player whose name contains `name`, or of one player scraped live from a link |
-| `valstats --clear-event <event-name>` | Remove one event and its players |
-| `valstats --clear` | Wipe the whole database (asks for confirmation) |
-
-### Scraping an event
-
-```bash
-npx valstats <link> <year> <event-name>
-```
-
-```bash
-npx valstats https://www.vlr.gg/event/2977/vct-2026-americas-stage-2/group-stage 2026 "Stage 2"
-```
-
-- `link` — the vlr.gg event/group-stage page to scrape.
-- `year` — the year the event took place.
-- `event-name` — a label for this tournament/stage (used to group players together later).
-
-What happens:
-
-1. If the event name contains the word **"stage"** (case-insensitive), you're prompted to
-   enter a region (e.g. `AMER`, `EMEA`). That region is saved on the event and on every
-   player scraped. Otherwise region is left empty.
-2. Every player profile linked on the event page is collected. Stats are pulled using
-   vlr.gg's **all-time** timespan.
-3. Each player is scraped, saved to `players.db` in your current directory, linked to the
-   event, and printed to the console.
-
-Running it again with an event name and year that are already saved replaces that event's
-players with the freshly scraped ones. A player already in the database is updated with the
-new data rather than duplicated.
-
-### Viewing a player's stats
-
-```bash
-npx valstats --player tenz                                    # every saved player with "tenz" in their name
-npx valstats --player ten                                     # partial names work too: tenz, tenten, ...
-npx valstats --player https://www.vlr.gg/player/9/tenz        # scrape a player live
-```
-
-- Passing a **name** searches `players.db` and prints every player whose name contains
-  those characters (case-insensitive). Names are the ones from the vlr.gg profile URL,
-  e.g. `tenz`. Nothing is scraped.
-- Passing a **vlr.gg player link** scrapes that profile live using the all-time timespan.
-  The result is printed but not saved to the database.
-
-For each player, their team, past teams, nationality, region, winnings, top agents and
-average K:D are printed, followed by a table of their stats on their top 3 agents.
-
-### Clearing data
-
-```bash
-npx valstats --clear-event "Stage 2"   # remove one event and its players
-npx valstats --clear                   # wipe the whole database (type "yes" to confirm)
-```
-
-Players that also belong to another event are kept when a single event is cleared.
-
-## What gets scraped
-
-Each player is stored as an object like this:
-
-```js
-{
-  name: 'tenz',                         // from the profile URL
-  team: 'Sentinels',                    // current team
-  past_teams: ['Cloud9', ...],          // previous teams
-  winnings: '$123,456',                 // total event winnings
-  most_played_agents: ['Jett', ...],    // top 3 agents by usage
-  avg_kda: 1.23,                        // average K:D across those top 3 agents
-  nationality: 'Canada',
-  region: 'AMER',                       // from the CLI prompt / argument, or null
-  stats: [                              // full agent stat rows for the top 3 agents,
-    { Agent: 'Jett', 'K:D': '1.25', ... } //   keyed by vlr.gg's column headers
-  ]
-}
-```
-
-## Library usage
-
-```js
-import {
-  scrapePlayer,
-  scrapeWebsite,
-  savePlayer,
-  saveEvent,
-  addPlayerToEvent,
-  getPlayersByEvent,
-  getRandomPlayerFromEvent,
-} from 'vlr-stats-scraper';
-
-const eventId = saveEvent({ name: 'Stage 2', region: 'AMER', year: 2026 });
-
-const links = await scrapeWebsite('https://www.vlr.gg/event/.../group-stage');
-for (const link of links) {
-  const player = await scrapePlayer(link, 'AMER');
-  if (player) {
-    savePlayer(player);
-    addPlayerToEvent(eventId, player.name);
-  }
-}
-
-const roster = getPlayersByEvent('Stage 2');
-const pick = getRandomPlayerFromEvent('Stage 2');
-```
-
-### Exports
-
-#### Scraping
-
-| Function | Returns | Description |
-| --- | --- | --- |
-| `scrapeWebsite(url)` | `Promise<string[]>` | Scrapes an event page and returns every player profile link on it (with `?timespan=all`). |
-| `scrapePlayer(url, region)` | `Promise<Player>` | Scrapes one player's profile page into a player object (see above). `region` is stored as-is. Returns `undefined` if scraping fails. |
-
-#### Players
-
-| Function | Returns | Description |
-| --- | --- | --- |
-| `savePlayer(player)` | — | Inserts a player, or updates them if one with the same name exists. |
-| `getPlayer(name)` | `Player \| null` | Looks up a player by name. |
-| `searchPlayers(query)` | `Player[]` | Every player whose name contains `query` (case-insensitive), sorted by name. |
-| `getAllPlayers()` | `Player[]` | Every player in the database. |
-| `getRandomPlayer()` | `Player \| null` | A random player from the whole database. |
-
-#### Events (tournaments / stages)
-
-| Function | Returns | Description |
-| --- | --- | --- |
-| `saveEvent({ name, url, region, year, stage })` | `number` | Creates or updates an event by name and returns its id. Only `name` is required. |
-| `addPlayerToEvent(eventId, playerName)` | — | Links a saved player to an event (ignored if already linked). |
-| `getEvents()` | `Event[]` | Every saved event. |
-| `getEventByName(name)` | `Event \| undefined` | Looks up an event by name. |
-| `getPlayersByEvent(eventName)` | `Player[]` | All players linked to an event. |
-| `getRandomPlayerFromEvent(eventName)` | `Player \| null` | A random player from one event. |
-
-#### Clearing
-
-| Function | Returns | Description |
-| --- | --- | --- |
-| `emptyEvent(eventName)` | `boolean` | Removes an event's players but keeps the event so it can be re-scraped. `false` if the event doesn't exist. |
-| `clearEvent(eventName)` | `boolean` | Removes an event and its players. `false` if the event doesn't exist. |
-| `clearDatabase()` | — | Deletes every event and player. |
-
-Players that still belong to another event are never deleted by `emptyEvent` or
-`clearEvent`.
-
-## Data storage
-
-Player and event data is stored in a `players.db` SQLite file created in whichever
-directory you run the tool from. It has three tables:
-
-- `players` — one row per player (`past_teams`, `most_played_agents` and `stats` are stored
-  as JSON and parsed back automatically when read).
-- `events` — one row per tournament/stage (`name`, `url`, `region`, `year`, `stage`).
-- `event_players` — links players to the events they were scraped from.
-
-Databases created by older versions (with `tournaments` / `tournament_players` tables) are
-migrated automatically.
-
-## License
-
-ISC
+# vlr-stats-scraper
+
+Scrapes player stats from [vlr.gg](https://www.vlr.gg) and stores them in a local SQLite
+database, grouped by tournament or stage, so you can pull a backlog of players.
+
+## Setup
+
+Requires Node.js 18+ and a [Hyperbrowser](https://hyperbrowser.ai) API key for scraping.
+
+```bash
+npm install vlr-stats-scraper
+```
+
+Create a `.env` file in the directory you run it from:
+
+```
+HYPERBROWSER_API_KEY=your_key_here
+```
+
+## CLI usage
+
+The package installs a `valstats` command.
+
+| Command | What it does |
+| --- | --- |
+| `valstats <link> <year> <event-name>` | Scrape every player on an event page and save them under that event |
+| `valstats --search <name \| link>` | Print the stats of every saved player whose name starts with `name`, or of one player scraped live from a link |
+| `valstats --player <link> <event-name> <year> [region]` | Scrape one player and add them to an event that's already saved |
+| `valstats --clear-event <event-name>` | Remove one event and its players |
+| `valstats --clear` | Wipe the whole database (asks for confirmation) |
+
+### Scraping an event
+
+```bash
+npx valstats <link> <year> <event-name>
+```
+
+```bash
+npx valstats https://www.vlr.gg/event/2977/vct-2026-americas-stage-2/group-stage 2026 "Stage 2"
+```
+
+- `link` — the vlr.gg event/group-stage page to scrape.
+- `year` — the year the event took place.
+- `event-name` — a label for this tournament/stage (used to group players together later).
+
+What happens:
+
+1. The event page is scraped for its title and every player profile linked on it.
+2. The region is read from the event title and saved on every player (not on the event):
+
+   | Title contains | Region |
+   | --- | --- |
+   | `America` (e.g. "VCT 2026: Americas Stage 2") | `AMER` |
+   | `EMEA` | `EMEA` |
+   | `Pacific` | `PAC` |
+   | `China` | `CN` |
+   | none of these (e.g. Masters, Champions) | `null` |
+
+3. Each player is scraped using vlr.gg's **all-time** timespan, saved to `players.db` in your
+   current directory under that event, and printed to the console.
+
+If the event page can't be scraped, nothing is saved or deleted.
+
+Running it again with an event name and year that are already saved replaces that event's
+players with the freshly scraped ones. A player who is already saved under a different event
+gets a new, separate row for this event, so their stats from each event are kept apart (see
+[Data storage](#data-storage)).
+
+### Viewing a player's stats
+
+```bash
+npx valstats --search tenz                                    # every saved player with "tenz" in their name
+npx valstats --search ten                                     # matches names starting with "ten": tenz, tenten, ... (not "kitten")
+npx valstats --search https://www.vlr.gg/player/9/tenz        # scrape a player live without saving
+```
+
+- `--search <name>` searches `players.db` and prints every player whose name starts with
+  those characters (case-insensitive). Names are the ones from the vlr.gg profile URL,
+  e.g. `tenz`. Nothing is scraped. A player saved under several events is printed once per
+  event, labelled with the event name and year, e.g. `tenz (Champs, 2026)`.
+- `--search <link>` scrapes that profile live using the all-time timespan. The result is
+  printed but not saved to the database.
+
+For each player, their team, past teams, nationality, region, winnings, top agents and
+average K:D are printed, followed by a table of their stats on their top 3 agents.
+
+### Adding a player to an event by hand
+
+```bash
+npx valstats --player <link> <event-name> <year> [region]
+```
+
+```bash
+npx valstats --player https://www.vlr.gg/player/9/tenz "Stage 2" 2026        # region copied from the event's other players
+npx valstats --player https://www.vlr.gg/player/9/tenz "Stage 2" 2026 AMER   # region set explicitly
+```
+
+Scrapes one player and saves them under an event that's already in `players.db`, e.g. a player
+the event page missed. The event is checked before scraping, so a mistyped name or year fails
+straight away without using a scrape. If you leave out `region`, the player gets the same
+region as the players already saved under that event.
+
+### Clearing data
+
+```bash
+npx valstats --clear-event "Stage 2"   # remove one event and its players
+npx valstats --clear                   # wipe the whole database (type "yes" to confirm)
+```
+
+Clearing one event only removes that event's player rows. The same player's saves under other
+events are kept.
+
+## What gets scraped
+
+Each player is stored as an object like this:
+
+```js
+{
+  name: 'tenz',                         // from the profile URL
+  team: 'Sentinels',                    // current team
+  past_teams: ['Cloud9', ...],          // previous teams
+  winnings: '$123,456',                 // total event winnings
+  most_played_agents: ['Jett', ...],    // top 3 agents by usage
+  avg_kda: 1.23,                        // average K:D across those top 3 agents
+  nationality: 'Canada',
+  region: 'AMER',                       // from the event title, or null
+  stats: [                              // full agent stat rows for the top 3 agents,
+    { Agent: 'Jett', 'K:D': '1.25', ... } //   keyed by vlr.gg's column headers
+  ]
+}
+```
+
+## Library usage
+
+```js
+import {
+  scrapePlayer,
+  scrapeWebsite,
+  savePlayer,
+  saveEvent,
+  getPlayersByEvent,
+  getRandomPlayerFromEvent,
+} from 'vlr-stats-scraper';
+
+const { playerLinks, region } = await scrapeWebsite('https://www.vlr.gg/event/.../group-stage');
+const eventId = saveEvent({ name: 'Stage 2', year: 2026 });
+
+for (const link of playerLinks) {
+  const player = await scrapePlayer(link, region);
+  if (player) {
+    savePlayer(eventId, player);
+  }
+}
+
+const roster = getPlayersByEvent('Stage 2');
+const pick = getRandomPlayerFromEvent('Stage 2');
+```
+
+### Upgrading from 1.x
+
+2.0.0 changes how players are stored and has breaking API changes:
+
+| 1.x | 2.0.0 |
+| --- | --- |
+| `savePlayer(player)` then `addPlayerToEvent(eventId, player.name)` | `savePlayer(eventId, player)`. `addPlayerToEvent` is removed. |
+| `scrapeWebsite(url)` returns `string[]` | Returns `{ playerLinks, region }` |
+| One row per player, shared across events | One row per player **per event** (see [Data storage](#data-storage)) |
+| `valstats --player <name>` | `valstats --search <name>`. `--player` now adds a player to an event. |
+| Region typed in at a prompt and saved on the event | Region read from the event title and saved on each player only |
+| `saveEvent({ name, url, region, year, stage })` | `saveEvent({ name, url, year, stage })` |
+
+Existing `players.db` files are migrated automatically the first time they're opened.
+
+### Exports
+
+#### Scraping
+
+| Function | Returns | Description |
+| --- | --- | --- |
+| `scrapeWebsite(url)` | `Promise<{ playerLinks, region }>` | Scrapes an event page. `playerLinks` is every player profile link on it (with `?timespan=all`); `region` is read from the event title (`'AMER'`, `'EMEA'`, `'PAC'`, `'CN'` or `null`). Returns `undefined` if scraping fails. |
+| `scrapePlayer(url, region)` | `Promise<Player>` | Scrapes one player's profile page into a player object (see above). `region` is stored as-is. Returns `undefined` if scraping fails. |
+
+#### Players
+
+A player is saved separately for each event, so the same player can have different stats in
+different events. Players read from the database also have `id`, `event_id`, `event_name` and
+`event_year`.
+
+| Function | Returns | Description |
+| --- | --- | --- |
+| `savePlayer(eventId, player)` | `number` | Saves a player's stats for one event and returns the row's id. Re-saving the same player in the same event updates them. |
+| `getPlayerById(id)` | `Player \| null` | Looks up one saved player row by id. |
+| `getPlayer(name)` | `Player \| null` | The player as they were in their most recent event. |
+| `getPlayerHistory(name)` | `Player[]` | Every saved version of a player, one per event, newest first. |
+| `searchPlayers(query)` | `Player[]` | Every player whose name starts with `query` (case-insensitive), sorted by name, one entry per event. |
+| `getAllPlayers()` | `Player[]` | Every player row in the database. |
+| `getRandomPlayer()` | `Player \| null` | A random player row from the whole database. |
+
+#### Events (tournaments / stages)
+
+| Function | Returns | Description |
+| --- | --- | --- |
+| `saveEvent({ name, url, year, stage })` | `number` | Creates or updates an event by name and returns its id. Only `name` is required. |
+| `getEvents()` | `Event[]` | Every saved event. |
+| `getEventByName(name)` | `Event \| undefined` | Looks up an event by name. |
+| `getEventId(name, year)` | `number \| undefined` | The id of the event with that name and year. |
+| `getPlayersByEvent(eventName)` | `Player[]` | Every player saved under an event. |
+| `getRandomPlayerFromEvent(eventName)` | `Player \| null` | A random player from one event. |
+
+#### Clearing
+
+| Function | Returns | Description |
+| --- | --- | --- |
+| `emptyEvent(eventName)` | `boolean` | Removes an event's players but keeps the event so it can be re-scraped. `false` if the event doesn't exist. |
+| `clearEvent(eventName)` | `boolean` | Removes an event and its players. `false` if the event doesn't exist. |
+| `clearDatabase()` | — | Deletes every event and player. |
+
+`emptyEvent` and `clearEvent` only delete that event's player rows, so the same player's saves in
+other events are kept.
+
+## Data storage
+
+Player and event data is stored in a `players.db` SQLite file created in whichever
+directory you run the tool from. It has two tables, and **every player row belongs to exactly
+one event**:
+
+```
+events                          players
+──────────────                  ────────────────────────────
+id (PK)  ◄───────────────────── event_id   (which event this row is from)
+name (unique)                   id (PK)    (unique per row)
+url                             name
+year                            team, past_teams, winnings,
+stage                           most_played_agents, avg_kda,
+                                nationality, region, stats
+```
+
+- `events` — one row per tournament/stage. Region isn't stored here; it's on each player.
+- `players` — one row per player **per event**. `past_teams`, `most_played_agents` and `stats`
+  are stored as JSON and parsed back automatically when read.
+
+A player's stats change over time, so the same player in two events is saved as two separate
+rows, each with the stats they had when that event was scraped.
+
+### Example
+
+**events**
+
+| id | name    | year |
+|----|---------|------|
+| 1  | Masters | 2025 |
+| 2  | Champs  | 2026 |
+
+**players**
+
+| id | event_id | name  | avg_kda |
+|----|----------|-------|---------|
+| 1  | 1        | tenz  | 1.10    |
+| 2  | 2        | tenz  | 1.35    |
+| 3  | 2        | aspas | 1.40    |
+
+`tenz` has two rows: one for how they played at Masters and one for Champs. Each has its own
+`id` and stats.
+
+### Rules the database enforces
+
+- **One copy of a player per event.** `(event_id, name)` is unique, so re-scraping Champs
+  updates tenz's Champs row instead of adding another one. Their Masters row isn't touched.
+- **Every player row must point to a real event.** Foreign keys are enforced, so saving a
+  player under an event id that doesn't exist fails.
+- **Deleting an event deletes its players.** Player rows are removed along with their event
+  (`ON DELETE CASCADE`).
+
+### Which function reads what
+
+| You want… | Function | What it does |
+|---|---|---|
+| Save a scraped player | `savePlayer(eventId, player)` | Inserts or updates that player's row for the event and returns its `id` |
+| An event's roster | `getPlayersByEvent('Champs')` | Every row whose `event_id` matches that event |
+| A player's latest version | `getPlayer('tenz')` | Their row from the newest event (highest year) |
+| Every version of a player | `getPlayerHistory('tenz')` | All their rows, newest first |
+| One specific row | `getPlayerById(2)` | Looks a row up by `id` |
+| Search by name | `searchPlayers('ten')` | Partial match, one result per player per event |
+| Re-scrape an event | `emptyEvent('Champs')` | Deletes that event's players but keeps the event |
+| Remove an event | `clearEvent('Champs')` | Deletes the event, and its players go with it |
+
+Every player read from the database also has `event_name` and `event_year`, so you can tell
+which event a row came from without a separate lookup.
+
+### What happens when you scrape an event
+
+```
+valstats <link> <year> <event-name>
+        │
+        ▼
+Already saved with this name + year? ──yes──► emptyEvent(name)
+        │                                     (delete its old player rows)
+        ▼                                            │
+saveEvent(...) ◄─────────────────────────────────────┘
+(create or update the event, get its id)
+        │
+        ▼
+For each player on the event page:
+  scrapePlayer(link) ──► savePlayer(eventId, player)
+```
+
+### Migrating older databases
+
+Older versions stored one shared row per player and linked players to events through an
+`event_players` table (or `tournaments` / `tournament_players` before that). These are
+migrated automatically the first time the database is opened:
+
+```
+Before                                   After
+players: tenz                            players: id 1, tenz, event_id 1
+event_players: (1, tenz), (2, tenz)  ──►          id 2, tenz, event_id 2
+```
+
+Each player is copied into a row for every event they were linked to, and `event_players` is
+removed. Older versions overwrote a player's stats on every scrape, so migrated rows for the
+same player all start with the same (most recent) stats. Players who weren't linked to any
+event are dropped.
+
+## License
+
+ISC

package/index.mjs

@@ -1,18 +1,20 @@
-export { default as scrapePlayer } from './scripts/player-info-scraper.mjs';
-export { scrapeWebsite } from './scripts/player-link-scraper.mjs';
-export {
-  savePlayer,
-  getPlayer,
-  searchPlayers,
-  getAllPlayers,
-  getRandomPlayer,
-  saveEvent,
-  addPlayerToEvent,
-  getEvents,
-  getEventByName,
-  getPlayersByEvent,
-  getRandomPlayerFromEvent,
-  emptyEvent,
-  clearEvent,
-  clearDatabase,
-} from './scripts/db.mjs';
+export { default as scrapePlayer } from './scripts/player-info-scraper.mjs';
+export { scrapeWebsite } from './scripts/player-link-scraper.mjs';
+export {
+  savePlayer,
+  getPlayerById,
+  getPlayer,
+  getPlayerHistory,
+  searchPlayers,
+  getAllPlayers,
+  getRandomPlayer,
+  saveEvent,
+  getEvents,
+  getEventByName,
+  getEventId,
+  getPlayersByEvent,
+  getRandomPlayerFromEvent,
+  emptyEvent,
+  clearEvent,
+  clearDatabase,
+} from './scripts/db.mjs';

package/main.mjs

@@ -1,138 +1,166 @@
-#!/usr/bin/env node
-import promptSync from 'prompt-sync'
-import { scrapeWebsite } from './scripts/player-link-scraper.mjs'
-import scrapePlayer from './scripts/player-info-scraper.mjs'
-import {
-  savePlayer,
-  saveEvent,
-  addPlayerToEvent,
-  getEventByName,
-  emptyEvent,
-  clearEvent,
-  clearDatabase,
-  searchPlayers,
-} from './scripts/db.mjs'
-
-const args = process.argv.slice(2)
-
-function printPlayerStats(player) {
-  console.log(`\n${player.name}`)
-  console.log(`  Team:        ${player.team ?? 'Free agent'}`)
-  console.log(`  Past teams:  ${player.past_teams?.length ? player.past_teams.join(', ') : 'None'}`)
-  console.log(`  Nationality: ${player.nationality || 'Unknown'}`)
-  console.log(`  Region:      ${player.region ?? 'Unknown'}`)
-  console.log(`  Winnings:    ${player.winnings ?? 'Unknown'}`)
-  // Older saves stored agents as raw <img> tags, so pull the name out of the alt text.
-  const agents = (player.most_played_agents ?? []).map(agent => agent?.match(/alt="([^"]*)"/)?.[1] ?? agent)
-  console.log(`  Top agents:  ${agents.join(', ') || 'None'}`)
-  console.log(`  Avg K:D:     ${Number(player.avg_kda).toFixed(2)}`)
-  if (player.stats?.length) {
-    console.table(player.stats)
-  }
-}
-
-if (args[0] === '--player') {
-  const query = args[1]
-  if (!query) {
-    console.error('Usage: valstats --player <name | vlr.gg player link>')
-    process.exit(1)
-  }
-
-  // A vlr.gg link is scraped live anything else is searched for in players.db.
-  if (query.startsWith('http')) {
-    const url = query.includes('timespan=') ? query : query.replace(/\/?$/, '/?timespan=all')
-    const player = await scrapePlayer(url, null)
-    if (!player) {
-      console.error(`Couldn't scrape a player from ${query}`)
-      process.exit(1)
-    }
-    printPlayerStats(player)
-    process.exit(0)
-  }
-
-  const players = searchPlayers(query)
-  if (!players.length) {
-    console.error(`No players matching "${query}" in players.db (pass a vlr.gg player link to scrape them live)`)
-    process.exit(1)
-  }
-  console.log(`${players.length} player${players.length === 1 ? '' : 's'} matching "${query}":`)
-  players.forEach(printPlayerStats)
-  process.exit(0)
-}
-
-if (args[0] === '--clear') {
-  const prompt = promptSync()
-  const answer = prompt('This deletes every event and player in players.db. Type "yes" to continue: ')
-  if (answer?.trim().toLowerCase() === 'yes') {
-    clearDatabase()
-    console.log('Cleared players.db')
-  } else {
-    console.log('Cancelled')
-  }
-  process.exit(0)
-}
-
-if (args[0] === '--clear-event') {
-  const name = args[1]
-  if (!name) {
-    console.error('Usage: valstats --clear-event <event-name>')
-    process.exit(1)
-  }
-  if (clearEvent(name)) {
-    console.log(`Removed "${name}" and its players from players.db`)
-  } else {
-    console.error(`No event named "${name}" in players.db`)
-    process.exit(1)
-  }
-  process.exit(0)
-}
-
-const [link, year, eventName] = args
-
-if (!link || !year || !eventName) {
-  console.error('Usage: valstats <link> <year> <event-name>')
-  console.error('       valstats --player <name | vlr.gg player link>')
-  console.error('       valstats --clear-event <event-name>')
-  console.error('       valstats --clear')
-  process.exit(1)
-}
-
-const isStage = eventName.toLowerCase().includes('stage')
-
-let region = null
-if (isStage) {
-  const prompt = promptSync()
-  region = prompt('Enter the region: ')
-}
-
-// Re-adding an event + year that's already saved replaces its old players.
-const existing = getEventByName(eventName)
-if (existing && existing.year === parseInt(year, 10)) {
-  emptyEvent(eventName)
-  console.log(`"${eventName}" (${year}) is already in players.db, replacing it`)
-}
-
-const eventId = saveEvent({
-  name: eventName,
-  url: link,
-  region,
-  year: parseInt(year, 10),
-  stage: null,
-})
-
-const links = await scrapeWebsite(link)
-
-if (!links) {
-  console.error(`Couldn't get the player list from ${link}`)
-  process.exit(1)
-}
-
-for (const playerLink of links) {
-  const player = await scrapePlayer(playerLink, region)
-  if (player) {
-    savePlayer(player)
-    addPlayerToEvent(eventId, player.name)
-    console.log(`Saved ${player.name} to players.db under "${eventName}"`)
-    console.log(player)
-  }
-}
+#!/usr/bin/env node
+import promptSync from 'prompt-sync'
+import { scrapeWebsite } from './scripts/player-link-scraper.mjs'
+import scrapePlayer from './scripts/player-info-scraper.mjs'
+import {
+  savePlayer,
+  saveEvent,
+  getEventByName,
+  getEventId,
+  getPlayersByEvent,
+  emptyEvent,
+  clearEvent,
+  clearDatabase,
+  searchPlayers,
+} from './scripts/db.mjs'
+
+const args = process.argv.slice(2)
+
+function printPlayerStats(player) {
+  // Players saved from the database belong to an event; ones scraped live don't.
+  const event = player.event_name ? ` (${player.event_name}${player.event_year ? `, ${player.event_year}` : ''})` : ''
+  console.log(`\n${player.name}${event}`)
+  console.log(`  Team:        ${player.team ?? 'Free agent'}`)
+  console.log(`  Past teams:  ${player.past_teams?.length ? player.past_teams.join(', ') : 'None'}`)
+  console.log(`  Nationality: ${player.nationality || 'Unknown'}`)
+  console.log(`  Region:      ${player.region ?? 'Unknown'}`)
+  console.log(`  Winnings:    ${player.winnings ?? 'Unknown'}`)
+  // Older saves stored agents as raw <img> tags, so pull the name out of the alt text.
+  const agents = (player.most_played_agents ?? []).map(agent => agent?.match(/alt="([^"]*)"/)?.[1] ?? agent)
+  console.log(`  Top agents:  ${agents.join(', ') || 'None'}`)
+  console.log(`  Avg K:D:     ${Number(player.avg_kda).toFixed(2)}`)
+  if (player.stats?.length) {
+    console.table(player.stats)
+  }
+}
+
+// Manually adds one player to an event that's already saved.
+if (args[0] === '--player') {
+  const [, query, eventName, eventYear, regionArg] = args
+  if (!query?.startsWith('http') || !eventName || !eventYear) {
+    console.error('Usage: valstats --player <vlr.gg player link> <event-name> <year> [region]')
+    process.exit(1)
+  }
+
+  // Look the event up before scraping so a typo doesn't waste a scrape.
+  const eventId = getEventId(eventName, parseInt(eventYear, 10))
+  if (!eventId) {
+    console.error(`No event named "${eventName}" (${eventYear}) in players.db`)
+    process.exit(1)
+  }
+
+  // Without a region argument, use the region the event's other players were saved with.
+  const region = regionArg ?? getPlayersByEvent(eventName)[0]?.region ?? null
+
+  const url = query.includes('timespan=') ? query : query.replace(/\/?$/, '/?timespan=all')
+  const player = await scrapePlayer(url, region)
+  if (!player) {
+    console.error(`Couldn't scrape a player from ${query}`)
+    process.exit(1)
+  }
+  savePlayer(eventId, player)
+  console.log(`Saved ${player.name} to players.db under "${eventName}" (${eventYear})`)
+  printPlayerStats(player)
+  process.exit(0)
+}
+
+if (args[0] === '--search') {
+  const query = args[1]
+  if (!query) {
+    console.error('Usage: valstats --search <name | vlr.gg player link>')
+    process.exit(1)
+  }
+
+  // A vlr.gg link is scraped live anything else is searched for in players.db.
+  if (query.startsWith('http')) {
+    const url = query.includes('timespan=') ? query : query.replace(/\/?$/, '/?timespan=all')
+    const player = await scrapePlayer(url, null)
+    if (!player) {
+      console.error(`Couldn't scrape a player from ${query}`)
+      process.exit(1)
+    }
+    printPlayerStats(player)
+    process.exit(0)
+  }
+
+  const players = searchPlayers(query)
+  if (!players.length) {
+    console.error(`No players matching "${query}" in players.db (pass a vlr.gg player link to scrape them live)`)
+    process.exit(1)
+  }
+  console.log(`${players.length} player${players.length === 1 ? '' : 's'} matching "${query}":`)
+  players.forEach(printPlayerStats)
+  process.exit(0)
+}
+
+if (args[0] === '--clear') {
+  const prompt = promptSync()
+  const answer = prompt('This deletes every event and player in players.db. Type "yes" to continue: ')
+  if (answer?.trim().toLowerCase() === 'yes') {
+    clearDatabase()
+    console.log('Cleared players.db')
+  } else {
+    console.log('Cancelled')
+  }
+  process.exit(0)
+}
+
+if (args[0] === '--clear-event') {
+  const name = args[1]
+  if (!name) {
+    console.error('Usage: valstats --clear-event <event-name>')
+    process.exit(1)
+  }
+  if (clearEvent(name)) {
+    console.log(`Removed "${name}" and its players from players.db`)
+  } else {
+    console.error(`No event named "${name}" in players.db`)
+    process.exit(1)
+  }
+  process.exit(0)
+}
+
+const [link, year, eventName] = args
+
+if (!link || !year || !eventName) {
+  console.error('Usage: valstats <link> <year> <event-name>')
+  console.error('       valstats --player <vlr.gg player link> <event-name> <year> [region]')
+  console.error('       valstats --search <name | vlr.gg player link>')
+  console.error('       valstats --clear-event <event-name>')
+  console.error('       valstats --clear')
+  process.exit(1)
+}
+
+// Scrape first so a failed scrape doesn't wipe an event that's already saved.
+const scraped = await scrapeWebsite(link)
+
+if (!scraped) {
+  console.error(`Couldn't get the player list from ${link}`)
+  process.exit(1)
+}
+
+const { playerLinks, region } = scraped
+console.log(`Found ${playerLinks.length} players, region: ${region ?? 'unknown'}`)
+
+// Re-adding an event + year that's already saved replaces its old players.
+const existing = getEventByName(eventName)
+if (existing && existing.year === parseInt(year, 10)) {
+  emptyEvent(eventName)
+  console.log(`"${eventName}" (${year}) is already in players.db, replacing it`)
+}
+
+const eventId = saveEvent({
+  name: eventName,
+  url: link,
+  year: parseInt(year, 10),
+  stage: null,
+})
+
+for (const playerLink of playerLinks) {
+  const player = await scrapePlayer(playerLink, region)
+  if (player) {
+    savePlayer(eventId, player)
+    console.log(`Saved ${player.name} to players.db under "${eventName}"`)
+    console.log(player)
+  }
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vlr-stats-scraper",
-  "version": "1.5.0",
+  "version": "2.0.0",
   "description": "Scrapes player stats from vlr.gg and stores them in a local SQLite database, grouped by tournament or stage.",
   "homepage": "https://github.com/OverYander/VLR-Webscraper#readme",
   "bugs": {

package/scripts/db.mjs

@@ -2,25 +2,7 @@ import Database from 'better-sqlite3'
 import path from 'path'
 
 const db = new Database(path.join(process.cwd(), 'players.db'))
-
-db.exec(`
-  CREATE TABLE IF NOT EXISTS players (
-    name TEXT PRIMARY KEY,
-    team TEXT,
-    past_teams TEXT,
-    winnings TEXT,
-    most_played_agents TEXT,
-    avg_kda REAL,
-    nationality TEXT,
-    region TEXT,
-    stats TEXT
-  )
-`)
-
-const existingPlayerColumns = db.prepare('PRAGMA table_info(players)').all().map(col => col.name)
-if (!existingPlayerColumns.includes('stats')) {
-  db.exec('ALTER TABLE players ADD COLUMN stats TEXT')
-}
+db.pragma('foreign_keys = ON')
 
 const existingTableNames = db.prepare("SELECT name FROM sqlite_master WHERE type='table'").all().map(t => t.name)
 if (existingTableNames.includes('tournaments') && !existingTableNames.includes('events')) {
@@ -36,7 +18,6 @@ db.exec(`
     id INTEGER PRIMARY KEY AUTOINCREMENT,
     name TEXT UNIQUE NOT NULL,
     url TEXT,
-    region TEXT,
     year INTEGER,
     stage INTEGER
   )
@@ -49,19 +30,60 @@ if (!existingEventColumns.includes('year')) {
 if (!existingEventColumns.includes('stage')) {
   db.exec('ALTER TABLE events ADD COLUMN stage INTEGER')
 }
+// Region is stored on each player, not on the event.
+if (existingEventColumns.includes('region')) {
+  db.exec('ALTER TABLE events DROP COLUMN region')
+}
 
-db.exec(`
-  CREATE TABLE IF NOT EXISTS event_players (
-    event_id INTEGER NOT NULL REFERENCES events(id),
-    player_name TEXT NOT NULL REFERENCES players(name),
-    PRIMARY KEY (event_id, player_name)
+// Each row is a player as they were in one event, so the same player can have a row per event
+// with different stats.
+const createPlayersTableSql = (tableName) => `
+  CREATE TABLE IF NOT EXISTS ${tableName} (
+    id INTEGER PRIMARY KEY AUTOINCREMENT,
+    event_id INTEGER NOT NULL REFERENCES events(id) ON DELETE CASCADE,
+    name TEXT NOT NULL,
+    team TEXT,
+    past_teams TEXT,
+    winnings TEXT,
+    most_played_agents TEXT,
+    avg_kda REAL,
+    nationality TEXT,
+    region TEXT,
+    stats TEXT,
+    UNIQUE (event_id, name)
   )
-`)
+`
+
+// Older databases kept one shared row per player and linked them to events through event_players.
+// Copy each of those links into its own per-event player row.
+const existingPlayerColumns = db.prepare('PRAGMA table_info(players)').all().map(col => col.name)
+if (existingPlayerColumns.length && !existingPlayerColumns.includes('event_id')) {
+  db.transaction(() => {
+    if (!existingPlayerColumns.includes('stats')) {
+      db.exec('ALTER TABLE players ADD COLUMN stats TEXT')
+    }
+    db.exec(createPlayersTableSql('players_new'))
+    db.exec(`
+      INSERT INTO players_new (event_id, name, team, past_teams, winnings, most_played_agents, avg_kda, nationality, region, stats)
+      SELECT event_players.event_id, players.name, players.team, players.past_teams, players.winnings,
+        players.most_played_agents, players.avg_kda, players.nationality, players.region, players.stats
+      FROM event_players
+      JOIN players ON players.name = event_players.player_name
+      JOIN events ON events.id = event_players.event_id
+    `)
+    db.exec('DROP TABLE event_players')
+    db.exec('DROP TABLE players')
+    db.exec('ALTER TABLE players_new RENAME TO players')
+  })()
+}
+
+db.exec(createPlayersTableSql('players'))
+db.exec('DROP TABLE IF EXISTS event_players')
 
 const upsertPlayerStmt = db.prepare(`
-  INSERT INTO players (name, team, past_teams, winnings, most_played_agents, avg_kda, nationality, region, stats)
-  VALUES (@name, @team, @past_teams, @winnings, @most_played_agents, @avg_kda, @nationality, @region, @stats)
-  ON CONFLICT(name) DO UPDATE SET
+  INSERT INTO players (event_id, name, team, past_teams, winnings, most_played_agents, avg_kda, nationality, region, stats)
+  VALUES (@event_id, @name, @team, @past_teams, @winnings, @most_played_agents, @avg_kda, @nationality, @region, @stats)
+  ON CONFLICT(event_id, name) DO UPDATE SET
     team = excluded.team,
     past_teams = excluded.past_teams,
     winnings = excluded.winnings,
@@ -73,19 +95,20 @@ const upsertPlayerStmt = db.prepare(`
 `)
 
 const upsertEventStmt = db.prepare(`
-  INSERT INTO events (name, url, region, year, stage)
-  VALUES (@name, @url, @region, @year, @stage)
+  INSERT INTO events (name, url, year, stage)
+  VALUES (@name, @url, @year, @stage)
   ON CONFLICT(name) DO UPDATE SET
     url = excluded.url,
-    region = excluded.region,
     year = excluded.year,
     stage = excluded.stage
 `)
 
-const linkPlayerToEventStmt = db.prepare(`
-  INSERT OR IGNORE INTO event_players (event_id, player_name)
-  VALUES (@eventId, @playerName)
-`)
+// Every player query also pulls in the event's name and year, newest event first.
+const selectPlayersSql = `
+  SELECT players.*, events.name AS event_name, events.year AS event_year FROM players
+  JOIN events ON events.id = players.event_id
+`
+const newestFirst = 'ORDER BY events.year DESC, players.id DESC'
 
 function rowToPlayer(row) {
   if (!row) return null
@@ -97,50 +120,60 @@ function rowToPlayer(row) {
   }
 }
 
-function savePlayer(player) {
+// Saves a player's stats for one event, replacing any earlier save of them in that same event.
+// Returns the player row's id.
+function savePlayer(eventId, player) {
   upsertPlayerStmt.run({
     ...player,
+    event_id: eventId,
     past_teams: JSON.stringify(player.past_teams),
     most_played_agents: JSON.stringify(player.most_played_agents),
     stats: JSON.stringify(player.stats),
   })
+  return db.prepare('SELECT id FROM players WHERE event_id = ? AND name = ?').get(eventId, player.name).id
+}
+
+function getPlayerById(id) {
+  return rowToPlayer(db.prepare(`${selectPlayersSql} WHERE players.id = ?`).get(id))
 }
 
+// The player as they were in their most recent event.
 function getPlayer(name) {
-  return rowToPlayer(db.prepare('SELECT * FROM players WHERE name = ?').get(name))
+  return rowToPlayer(db.prepare(`${selectPlayersSql} WHERE players.name = ? ${newestFirst} LIMIT 1`).get(name))
 }
 
-// Case-insensitive partial match, e.g. "ten" finds "tenz" and "tenten".
+// Every saved version of a player, one per event, newest first.
+function getPlayerHistory(name) {
+  return db.prepare(`${selectPlayersSql} WHERE players.name = ? ${newestFirst}`).all(name).map(rowToPlayer)
+}
+
+// Case-insensitive prefix match, e.g. "nat" finds "nats" but not "gnat". Returns one row per event.
 function searchPlayers(query) {
-  return db.prepare("SELECT * FROM players WHERE name LIKE ? ESCAPE '\\' ORDER BY name")
-    .all(`%${query.replace(/[\\%_]/g, '\\$&')}%`)
+  return db.prepare(`${selectPlayersSql} WHERE players.name LIKE ? ESCAPE '\\' ORDER BY players.name, events.year DESC, players.id DESC`)
+    .all(`${query.replace(/[\\%_]/g, '\\$&')}%`)
     .map(rowToPlayer)
 }
 
 function getAllPlayers() {
-  return db.prepare('SELECT * FROM players').all().map(rowToPlayer)
+  return db.prepare(selectPlayersSql).all().map(rowToPlayer)
 }
 
 function getRandomPlayer() {
-  return rowToPlayer(db.prepare('SELECT * FROM players ORDER BY RANDOM() LIMIT 1').get())
+  return rowToPlayer(db.prepare(`${selectPlayersSql} ORDER BY RANDOM() LIMIT 1`).get())
 }
 
+
 // Creates the event (tournament, stage, etc.) if it doesn't exist yet, and returns its id either way.
-function saveEvent({ name, url, region, year, stage }) {
+function saveEvent({ name, url, year, stage }) {
   upsertEventStmt.run({
     name,
     url: url ?? null,
-    region: region ?? null,
     year: year ?? null,
     stage: stage ?? null,
   })
   return db.prepare('SELECT id FROM events WHERE name = ?').get(name).id
 }
 
-function addPlayerToEvent(eventId, playerName) {
-  linkPlayerToEventStmt.run({ eventId, playerName })
-}
-
 function getEvents() {
   return db.prepare('SELECT * FROM events').all()
 }
@@ -149,62 +182,49 @@ function getEventByName(name) {
   return db.prepare('SELECT * FROM events WHERE name = ?').get(name)
 }
 
-function getPlayersByEvent(eventName) {
-  return db.prepare(`
-    SELECT players.* FROM players
-    JOIN event_players ON event_players.player_name = players.name
-    JOIN events ON events.id = event_players.event_id
-    WHERE events.name = ?
-  `).all(eventName).map(rowToPlayer)
+function getEventId(name, year) {
+  return db.prepare('SELECT id FROM events WHERE name = ? AND year = ?').get(name, year)?.id
 }
 
-function getRandomPlayerFromEvent(eventName) {
-  return rowToPlayer(db.prepare(`
-    SELECT players.* FROM players
-    JOIN event_players ON event_players.player_name = players.name
-    JOIN events ON events.id = event_players.event_id
-    WHERE events.name = ?
-    ORDER BY RANDOM() LIMIT 1
-  `).get(eventName))
+function getPlayersByEvent(eventName) {
+  return db.prepare(`${selectPlayersSql} WHERE events.name = ?`).all(eventName).map(rowToPlayer)
 }
 
-// Deletes players that no longer belong to any event.
-function deleteOrphanedPlayers() {
-  db.prepare('DELETE FROM players WHERE name NOT IN (SELECT player_name FROM event_players)').run()
+function getRandomPlayerFromEvent(eventName) {
+  return rowToPlayer(db.prepare(`${selectPlayersSql} WHERE events.name = ? ORDER BY RANDOM() LIMIT 1`).get(eventName))
 }
 
 // Removes every player from an event but keeps the event itself, so it can be re-scraped.
-// Players that are still in other events are kept.
-const emptyEvent = db.transaction((eventName) => {
+function emptyEvent(eventName) {
   const event = getEventByName(eventName)
   if (!event) return false
-  db.prepare('DELETE FROM event_players WHERE event_id = ?').run(event.id)
-  deleteOrphanedPlayers()
+  db.prepare('DELETE FROM players WHERE event_id = ?').run(event.id)
   return true
-})
+}
 
-// Removes an event and its players. Players that are still in other events are kept.
-const clearEvent = db.transaction((eventName) => {
-  if (!emptyEvent(eventName)) return false
-  db.prepare('DELETE FROM events WHERE name = ?').run(eventName)
-  return true
-})
+// Removes an event and its players.
+function clearEvent(eventName) {
+  // The event's players are deleted along with it (ON DELETE CASCADE).
+  return db.prepare('DELETE FROM events WHERE name = ?').run(eventName).changes > 0
+}
 
 // Wipes every event and player from the database.
 const clearDatabase = db.transaction(() => {
-  db.exec('DELETE FROM event_players DELETE FROM events DELETE FROM players')
+  db.exec('DELETE FROM players; DELETE FROM events;')
 })
 
 export {
   savePlayer,
+  getPlayerById,
   getPlayer,
+  getPlayerHistory,
   searchPlayers,
   getAllPlayers,
   getRandomPlayer,
   saveEvent,
-  addPlayerToEvent,
   getEvents,
   getEventByName,
+  getEventId,
   getPlayersByEvent,
   getRandomPlayerFromEvent,
   emptyEvent,

package/scripts/player-info-scraper.mjs

@@ -24,7 +24,7 @@ async function scrapePlayer(url, region) {
       }
     })
 
-    const $ = cheerio.load(scrapeData.data.html);
+    const $ = cheerio.load(scrapeData.data.html)
     const agentsPlayed = findAgents($)
 
     return {

package/scripts/player-link-scraper.mjs

@@ -1,3 +1,4 @@
+import * as cheerio from 'cheerio'
 import { Hyperbrowser } from "@hyperbrowser/sdk"
 import { config } from "dotenv";
 
@@ -19,24 +20,50 @@ async function scrapeWebsite(url) {
       url: url,
       scrapeOptions: {
         formats: ["html"],
-        includeTags: [".event-team-players-item"]
+        includeTags: [".event-header-main-title", ".event-team-players-item"]
       },
     })
+    const data = scrapeData.data.html
+    const $ = cheerio.load(data)
 
-    const players = scrapeData.data.html.split('\n').filter(line => line.includes('event-team-players-item'))
-    const playerLinks = players.map(player => {
-      const match = player.match(/href="(https:\/\/www\.vlr\.gg\/player\/[^"]*)"/)
-      return match ? match[1] + "/?timespan=all" : null;
-    }).filter(link => link !== null);
-    
-    return playerLinks
+    /*
+<h1 class="event-header-main-title">
+				VCT 2027: America Kickoff - North America Open Qualifier  
+			</h1>
+    */
+    const playerLinks = $('.event-team-players-item')
+    .map((i, el) => $(el).attr('href'))
+    .get()
+    .filter(href => href.includes('/player/'))
+    .map(href => href + '/?timespan=all')
+
+    let region = $('.event-header-main-title').first().text().trim()
+    console.log(region)
+
+    if (region.includes('America')) {
+      region = 'AMER'
+    } 
+    else if (region.includes('EMEA')) {
+      region = 'EMEA'
+    } 
+    else if (region.includes('Pacific')) {
+      region = 'PAC'
+    } 
+    else if (region.includes('China')) {
+      region = 'CN'
+    }
+    else {
+      region = null
+    }
+  
+    return { playerLinks, region }
   } 
   catch (error) {
     console.error("Error scraping website:", error)
   }
 }
 
-export { scrapeWebsite };
+export { scrapeWebsite }
 /*
 node teamscraper.mjs
 */