navcli 0.1.0__tar.gz → 0.2.1__tar.gz

navcli-0.2.1/PKG-INFO

@@ -0,0 +1,81 @@
+Metadata-Version: 2.4
+Name: navcli
+Version: 0.2.1
+Summary: 可交互、可探索的浏览器命令行工具，专为 AI Agent 设计
+Author: NavCLI Team
+License: MIT
+Project-URL: Homepage, https://github.com/navcli/navcli
+Project-URL: Repository, https://github.com/navcli/navcli
+Project-URL: Issues, https://github.com/navcli/navcli/issues
+Keywords: browser,cli,automation,ai-agent,playwright
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+Requires-Dist: playwright>=1.40.0
+Requires-Dist: cmd2>=2.4.0
+Requires-Dist: fastapi>=0.109.0
+Requires-Dist: uvicorn<1.0.0,>=0.25.0
+Requires-Dist: pydantic>=2.5.0
+Requires-Dist: cssify>=1.0.0
+Requires-Dist: aiohttp>=3.9.0
+Provides-Extra: dev
+Requires-Dist: pytest>=7.4.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Requires-Dist: mypy>=1.8.0; extra == "dev"
+
+# NavCLI - Goals & Vision
+
+## Core Goal
+
+**Enable AI Agents to browse the web like humans.**
+
+Existing solutions (HTTP APIs, headless browser scripts, Playwright MCP) have limitations:
+- No support for JS-rendered SPAs
+- No session persistence
+- Lack of interactive exploration
+
+NavCLI's positioning: **An interactive, explorable browser CLI**
+
+## Core Value
+
+| Feature | What NavCLI Solves |
+|---------|-------------------|
+| JS Rendering | Full SPA support |
+| Session Persistence | Cookies, session maintained |
+| Interactive CLI | Agent can explore while operating |
+| Token Optimization | Lightweight elements + on-demand text/html |
+
+## Typical Workflow
+
+```bash
+> g https://example.com          # Navigate
+> elements                       # Observe interactive elements
+> c .btn-login                   # Click login
+> t #email "test@example.com"    # Type email
+> t #password "123456"           # Type password
+> c button[type="submit"]       # Submit
+> text                           # Confirm result
+```
+
+Agent can: **Navigate → Observe → Interact → Feedback → Continue**
+
+## Vision
+
+Become the **standard browser interaction layer** for AI Agents, enabling any Agent to control browsers via command-line interface:
+- Form filling, login authentication
+- Information scraping, content exploration
+- Complex multi-step business processes
+
+## Related Documentation
+
+- [Skill File Download](https://github.com/wumu2013/navcli/blob/main/docs/skill.md)
+- [Project Homepage](https://make.datavoid.fun/navcli/)
+- [PRD Product Requirements](./NAVCLI_PRD.md)

navcli-0.2.1/README.md

@@ -0,0 +1,180 @@
+# NavCLI - Browser Automation CLI for AI Agents
+
+[![PyPI Version](https://img.shields.io/pypi/v/navcli)](https://pypi.org/project/navcli/)
+[![GitHub](https://img.shields.io/badge/GitHub-wumu2013/navcli-blue)](https://github.com/wumu2013/navcli)
+
+**Enable AI Agents to browse the web like humans.**
+
+## What is NavCLI?
+
+NavCLI is an interactive, explorable browser CLI designed for AI Agents. Unlike HTTP APIs or headless browser scripts, NavCLI provides:
+
+- **Full SPA Support** - JavaScript-rendered pages work seamlessly
+- **Session Persistence** - Cookies and login sessions maintained across requests
+- **Interactive CLI** - Agents can explore and interact dynamically
+- **Token Optimization** - Lightweight element queries with on-demand content
+
+## Quick Start
+
+```bash
+pip install navcli
+```
+
+### Start Server
+
+```bash
+nav-server
+```
+
+### Use CLI
+
+```bash
+nav
+```
+
+Or run commands directly:
+
+```bash
+nav g https://example.com
+nav tables
+nav paragraphs
+```
+
+## Data Extraction
+
+> **IMPORTANT**: For data extraction, focus on `tables` and `paragraphs`.
+
+| Command | Description |
+|---------|-------------|
+| `nav tables` | Extract all `<table>` elements with structured data |
+| `nav paragraphs` | Extract large text paragraphs (200+ chars) |
+| `nav g <url>` | Navigate + auto-fetch paragraphs on first call |
+
+## Core Commands
+
+### Navigation
+
+```bash
+nav g <url>       # Navigate to URL
+nav b             # Go back
+nav f             # Go forward
+nav r             # Reload
+```
+
+### Interaction
+
+```bash
+nav c <sel> [--force]       # Click element
+nav t <sel> <txt>            # Type text
+nav clear <sel>              # Clear input
+nav upload <sel> <path>     # Upload file
+```
+
+### Query
+
+```bash
+nav elements              # Interactive elements
+nav text                  # Page text content
+nav html                  # Page HTML
+nav screenshot            # Screenshot (base64)
+nav state                 # Full page state
+nav url                   # Current URL
+nav title                 # Page title
+nav tables                # All tables with data
+nav paragraphs [len]      # Large text paragraphs
+nav links                 # All hyperlinks
+nav forms                 # All forms
+```
+
+### Explore
+
+```bash
+nav find <text>           # Find element by text
+nav findall <text>        # Find all elements by text
+nav inspect <sel>         # Inspect element details
+nav wait_idle [sec]       # Wait for network idle
+nav scroll top|bottom|up|down  # Scroll page
+```
+
+### Session
+
+```bash
+nav cookies              # Show cookies
+nav save_session [name]  # Save session
+nav load_session [name]  # Load session
+```
+
+### Control
+
+```bash
+nav quit    # Close browser
+nav exit    # Exit CLI
+```
+
+## Agent Workflow
+
+```
+Navigate → Observe → Interact → Feedback → Continue
+```
+
+```bash
+# 1 Navigate - Go to target page
+nav g https://example.com/data-page
+
+# 2 Observe - Check what data is available
+nav tables         # Extract tabular data
+nav paragraphs     # Extract text content
+
+# 3 Interact - Navigate/paginate if needed
+nav find "Next"
+nav c a.next-page
+nav wait_idle
+
+# 4 Feedback - Verify data extraction
+nav tables
+nav paragraphs
+
+# 5 Continue - Proceed to next data source
+nav g https://example.com/page2
+```
+
+## API
+
+NavCLI uses a client-server architecture with HTTP API:
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/cmd/goto?url=<url>` | Navigate to URL |
+| POST | `/cmd/click?selector=<s>` | Click element |
+| GET | `/query/tables` | Get all tables |
+| GET | `/query/paragraphs` | Get large text paragraphs |
+| GET | `/query/elements` | Get interactive elements |
+| POST | `/cmd/session/save?name=<n>` | Save session |
+| POST | `/cmd/session/load?name=<n>` | Load session |
+
+See [docs/skill.md](docs/skill.md) for complete API reference.
+
+## Session Persistence
+
+Save login sessions for authenticated access:
+
+```bash
+# Save after login
+nav save_session jisilu
+
+# Load session for future headless access
+nav load_session jisilu
+```
+
+Sessions are stored in `~/.navcli/sessions/`.
+
+## Documentation
+
+- [CLI & API Reference](docs/skill.md)
+- [Skill File Download](https://github.com/wumu2013/navcli/blob/main/docs/skill.md)
+- [Project Homepage](https://make.datavoid.fun/navcli/)
+- [Product Requirements](docs/NAVCLI_PRD.md)
+
+## License
+
+MIT

navcli-0.2.1/docs/README.md

@@ -0,0 +1,48 @@
+# NavCLI - Goals & Vision
+
+## Core Goal
+
+**Enable AI Agents to browse the web like humans.**
+
+Existing solutions (HTTP APIs, headless browser scripts, Playwright MCP) have limitations:
+- No support for JS-rendered SPAs
+- No session persistence
+- Lack of interactive exploration
+
+NavCLI's positioning: **An interactive, explorable browser CLI**
+
+## Core Value
+
+| Feature | What NavCLI Solves |
+|---------|-------------------|
+| JS Rendering | Full SPA support |
+| Session Persistence | Cookies, session maintained |
+| Interactive CLI | Agent can explore while operating |
+| Token Optimization | Lightweight elements + on-demand text/html |
+
+## Typical Workflow
+
+```bash
+> g https://example.com          # Navigate
+> elements                       # Observe interactive elements
+> c .btn-login                   # Click login
+> t #email "test@example.com"    # Type email
+> t #password "123456"           # Type password
+> c button[type="submit"]       # Submit
+> text                           # Confirm result
+```
+
+Agent can: **Navigate → Observe → Interact → Feedback → Continue**
+
+## Vision
+
+Become the **standard browser interaction layer** for AI Agents, enabling any Agent to control browsers via command-line interface:
+- Form filling, login authentication
+- Information scraping, content exploration
+- Complex multi-step business processes
+
+## Related Documentation
+
+- [Skill File Download](https://github.com/wumu2013/navcli/blob/main/docs/skill.md)
+- [Project Homepage](https://make.datavoid.fun/navcli/)
+- [PRD Product Requirements](./NAVCLI_PRD.md)

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/app.py

@@ -54,6 +54,10 @@ class NavCLI(
             self.poutput("  state         Get full page state")
             self.poutput("  url           Get current URL")
             self.poutput("  title         Get page title")
+            self.poutput("  tables        Get all tables with data")
+            self.poutput("  paragraphs    Get large text paragraphs (200+ chars)")
+            self.poutput("  links         Get all links")
+            self.poutput("  forms         Get all forms")
 
             self.poutput("\nExplore Commands:")
             self.poutput("  find <text>        Find element by text")

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/client.py

@@ -123,6 +123,18 @@ class NavClient:
         """Get all forms on the page."""
         return await self._request("GET", "/query/forms")
 
+    async def get_tables(self):
+        """Get all tables on the page with their data."""
+        return await self._request("GET", "/query/tables")
+
+    async def get_paragraphs(self, min_length: int = 200):
+        """Get large text paragraphs (200+ chars) from the page.
+
+        Args:
+            min_length: Minimum paragraph length (default: 200)
+        """
+        return await self._request("GET", f"/query/paragraphs?min_length={min_length}")
+
     # Explore commands
     async def find(self, text: str):
         """Find element by text."""
@@ -234,25 +246,39 @@ class NavClient:
 
         return await self._request("POST", "/cmd/cookies/set", json=body)
 
-    async def save_session(self, path: str = ".navcli_session.json"):
+    async def save_session(self, name: str = None, path: str = None):
         """Save session to file.
 
         Args:
-            path: File path to save session
+            name: Session name (saves to ~/.navcli/sessions/<name>.json)
+            path: Full file path (alternative to name)
         """
         import urllib.parse
-        encoded_path = urllib.parse.quote(path)
-        return await self._request("POST", f"/cmd/session/save?path={encoded_path}")
 
-    async def load_session(self, path: str = ".navcli_session.json"):
+        if name:
+            return await self._request("POST", f"/cmd/session/save?name={name}")
+        elif path:
+            encoded_path = urllib.parse.quote(path)
+            return await self._request("POST", f"/cmd/session/save?path={encoded_path}")
+        else:
+            return await self._request("POST", "/cmd/session/save")
+
+    async def load_session(self, name: str = None, path: str = None):
         """Load session from file.
 
         Args:
-            path: File path to load session from
+            name: Session name (loads from ~/.navcli/sessions/<name>.json)
+            path: Full file path (alternative to name)
         """
         import urllib.parse
-        encoded_path = urllib.parse.quote(path)
-        return await self._request("POST", f"/cmd/session/load?path={encoded_path}")
+
+        if name:
+            return await self._request("POST", f"/cmd/session/load?name={name}")
+        elif path:
+            encoded_path = urllib.parse.quote(path)
+            return await self._request("POST", f"/cmd/session/load?path={encoded_path}")
+        else:
+            return await self._request("POST", "/cmd/session/load")
 
 
 async def main():

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/commands/control.py

@@ -21,7 +21,7 @@ class ControlCommands(BaseBrowserCommand):
         async def _quit():
             return await self.client.quit()
 
-        result = asyncio.run(self.run_async_client(_quit))
+        result = asyncio.run(self.run_async_client(_quit()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -35,7 +35,7 @@ class ControlCommands(BaseBrowserCommand):
         async def _shutdown():
             return await self.client.shutdown()
 
-        result = asyncio.run(self.run_async_client(_shutdown))
+        result = asyncio.run(self.run_async_client(_shutdown()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -50,41 +50,41 @@ class ControlCommands(BaseBrowserCommand):
     # Session commands
 
     save_parser = cmd2.Cmd2ArgumentParser()
-    save_parser.add_argument("path", nargs="?", default=".navcli_session.json",
-                              help="Path to save session file")
+    save_parser.add_argument("name", nargs="?", default=None,
+                             help="Session name (saves to ~/.navcli/sessions/<name>.json)")
 
     @cmd2.with_argparser(save_parser)
     def do_save_session(self, args):
         """Save session (cookies, storage) to file.
 
-        Usage: save_session [path]
-        Example: save_session my_session.json
+        Usage: save_session [name]
+        Example: save_session jisilu
         """
         import asyncio
 
         async def _save():
-            return await self.client.save_session(args.path)
+            return await self.client.save_session(name=args.name)
 
-        result = asyncio.run(self.run_async_client(_save))
+        result = asyncio.run(self.run_async_client(_save()))
         self.pretty_result(result)
 
     load_parser = cmd2.Cmd2ArgumentParser()
-    load_parser.add_argument("path", nargs="?", default=".navcli_session.json",
-                              help="Path to load session file from")
+    load_parser.add_argument("name", nargs="?", default=None,
+                             help="Session name (loads from ~/.navcli/sessions/<name>.json)")
 
     @cmd2.with_argparser(load_parser)
     def do_load_session(self, args):
         """Load session (cookies, storage) from file.
 
-        Usage: load_session [path]
-        Example: load_session my_session.json
+        Usage: load_session [name]
+        Example: load_session jisilu
         """
         import asyncio
 
         async def _load():
-            return await self.client.load_session(args.path)
+            return await self.client.load_session(name=args.name)
 
-        result = asyncio.run(self.run_async_client(_load))
+        result = asyncio.run(self.run_async_client(_load()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -98,7 +98,7 @@ class ControlCommands(BaseBrowserCommand):
         async def _get():
             return await self.client.get_cookies()
 
-        result = asyncio.run(self.run_async_client(_get))
+        result = asyncio.run(self.run_async_client(_get()))
         if result.get("success"):
             self.poutput("[OK] got cookies")
         else:
@@ -115,5 +115,5 @@ class ControlCommands(BaseBrowserCommand):
         async def _clear():
             return await self.client.clear_cookies()
 
-        result = asyncio.run(self.run_async_client(_clear))
+        result = asyncio.run(self.run_async_client(_clear()))
         self.pretty_result(result)

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/commands/explore.py

@@ -22,7 +22,7 @@ class ExploreCommands(BaseBrowserCommand):
         async def _find():
             return await self.client.find(args.text)
 
-        result = asyncio.run(self.run_async_client(_find))
+        result = asyncio.run(self.run_async_client(_find()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -37,7 +37,7 @@ class ExploreCommands(BaseBrowserCommand):
         async def _findall():
             return await self.client.findall(args.text)
 
-        result = asyncio.run(self.run_async_client(_findall))
+        result = asyncio.run(self.run_async_client(_findall()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -52,7 +52,7 @@ class ExploreCommands(BaseBrowserCommand):
         async def _inspect():
             return await self.client.inspect(args.selector)
 
-        result = asyncio.run(self.run_async_client(_inspect))
+        result = asyncio.run(self.run_async_client(_inspect()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -73,7 +73,7 @@ class ExploreCommands(BaseBrowserCommand):
             else:
                 return await self.client.wait()
 
-        result = asyncio.run(self.run_async_client(_wait))
+        result = asyncio.run(self.run_async_client(_wait()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -89,7 +89,7 @@ class ExploreCommands(BaseBrowserCommand):
             timeout = args.timeout if args.timeout else 3.0
             return await self.client.wait_idle(timeout=timeout)
 
-        result = asyncio.run(self.run_async_client(_wait_idle))
+        result = asyncio.run(self.run_async_client(_wait_idle()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -121,5 +121,5 @@ class ExploreCommands(BaseBrowserCommand):
             else:
                 return await self.client.scroll(direction="down")
 
-        result = asyncio.run(self.run_async_client(_scroll))
+        result = asyncio.run(self.run_async_client(_scroll()))
         self.pretty_result(result)

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/commands/interaction.py

@@ -33,7 +33,7 @@ class InteractionCommands(BaseBrowserCommand):
         async def _click():
             return await self.client.click(args.selector, force=args.force)
 
-        result = asyncio.run(self.run_async_client(_click))
+        result = asyncio.run(self.run_async_client(_click()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -48,7 +48,7 @@ class InteractionCommands(BaseBrowserCommand):
         async def _type():
             return await self.client.type(args.selector, args.text)
 
-        result = asyncio.run(self.run_async_client(_type))
+        result = asyncio.run(self.run_async_client(_type()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -63,7 +63,7 @@ class InteractionCommands(BaseBrowserCommand):
         async def _dblclick():
             return await self.client.dblclick(args.selector)
 
-        result = asyncio.run(self.run_async_client(_dblclick))
+        result = asyncio.run(self.run_async_client(_dblclick()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -78,7 +78,7 @@ class InteractionCommands(BaseBrowserCommand):
         async def _rightclick():
             return await self.client.rightclick(args.selector)
 
-        result = asyncio.run(self.run_async_client(_rightclick))
+        result = asyncio.run(self.run_async_client(_rightclick()))
         self.pretty_result(result)
 
     # Aliases
@@ -110,7 +110,7 @@ class InteractionCommands(BaseBrowserCommand):
         async def _clear():
             return await self.client.clear(args.selector)
 
-        result = asyncio.run(self.run_async_client(_clear))
+        result = asyncio.run(self.run_async_client(_clear()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -125,5 +125,5 @@ class InteractionCommands(BaseBrowserCommand):
         async def _upload():
             return await self.client.upload(args.selector, args.file_path)
 
-        result = asyncio.run(self.run_async_client(_upload))
+        result = asyncio.run(self.run_async_client(_upload()))
         self.pretty_result(result)

{navcli-0.1.0 → navcli-0.2.1}/navcli/cli/commands/navigation.py

@@ -17,7 +17,7 @@ class NavigationCommands(BaseBrowserCommand):
 
     @cmd2.with_argparser(_goto_parser)
     def do_goto(self, args):
-        """Navigate to a URL.
+        """Navigate to a URL and show page summary.
 
         Usage: goto <url>
         Example: goto https://example.com
@@ -28,9 +28,17 @@ class NavigationCommands(BaseBrowserCommand):
             url = "https://" + url
 
         async def _goto():
-            return await self.client.goto(url)
-
-        result = asyncio.run(self.run_async_client(_goto))
+            # Navigate first
+            result = await self.client.goto(url)
+            # Then get paragraphs and merge into state
+            para_result = await self.client.get_paragraphs(min_length=200)
+            if para_result.get("success") and result.get("success"):
+                # Extract paragraphs from state and merge
+                para_data = para_result.get("state", {}).get("paragraphs", [])
+                result["state"]["paragraphs"] = para_data
+            return result
+
+        result = asyncio.run(self.run_async_client(_goto()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -44,7 +52,7 @@ class NavigationCommands(BaseBrowserCommand):
         async def _back():
             return await self.client.back()
 
-        result = asyncio.run(self.run_async_client(_back))
+        result = asyncio.run(self.run_async_client(_back()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -58,7 +66,7 @@ class NavigationCommands(BaseBrowserCommand):
         async def _forward():
             return await self.client.forward()
 
-        result = asyncio.run(self.run_async_client(_forward))
+        result = asyncio.run(self.run_async_client(_forward()))
         self.pretty_result(result)
 
     @cmd2.with_argparser(cmd2.Cmd2ArgumentParser())
@@ -72,7 +80,7 @@ class NavigationCommands(BaseBrowserCommand):
         async def _reload():
             return await self.client.reload()
 
-        result = asyncio.run(self.run_async_client(_reload))
+        result = asyncio.run(self.run_async_client(_reload()))
         self.pretty_result(result)
 
     # Aliases
@@ -80,11 +88,25 @@ class NavigationCommands(BaseBrowserCommand):
     do_f = do_forward  # forward -> f
     do_r = do_reload  # reload -> r
 
-    # goto -> g (requires separate parser)
+    # goto -> g (shortcut with same logic as do_goto)
     @cmd2.with_argparser(_goto_parser)
     def do_g(self, args):
         """Navigate to a URL (shortcut).
 
         Usage: g <url>
         """
-        return self.do_goto(args)
+        import asyncio
+        url = args.url
+        if not url.startswith(("http://", "https://")):
+            url = "https://" + url
+
+        async def _goto():
+            result = await self.client.goto(url)
+            para_result = await self.client.get_paragraphs(min_length=200)
+            if para_result.get("success") and result.get("success"):
+                para_data = para_result.get("state", {}).get("paragraphs", [])
+                result["state"]["paragraphs"] = para_data
+            return result
+
+        result = asyncio.run(self.run_async_client(_goto()))
+        self.pretty_result(result)