cognitrix 0.2.0__tar.gz → 0.2.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. {cognitrix-0.2.0 → cognitrix-0.2.1}/PKG-INFO +15 -13
  2. cognitrix-0.2.1/README.md +77 -0
  3. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/agents/base.py +3 -3
  4. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/agents/templates.py +2 -15
  5. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/app.py +0 -6
  6. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/cli.py +9 -3
  7. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/config.py +1 -1
  8. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/__init__.py +3 -1
  9. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/misc.py +105 -11
  10. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/tool.py +7 -6
  11. {cognitrix-0.2.0 → cognitrix-0.2.1}/pyproject.toml +1 -1
  12. cognitrix-0.2.0/README.md +0 -75
  13. {cognitrix-0.2.0 → cognitrix-0.2.1}/LICENSE +0 -0
  14. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/__init__.py +0 -0
  15. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/agents/__init__.py +0 -0
  16. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/agents/assistant.py +0 -0
  17. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/agents/core.py +0 -0
  18. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/__init__.py +0 -0
  19. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/anthropic_llm.py +0 -0
  20. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/base.py +0 -0
  21. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/clarifai_llm.py +0 -0
  22. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/cohere_llm.py +0 -0
  23. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/google_llm.py +0 -0
  24. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/groq_llm.py +0 -0
  25. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/openai_llm.py +0 -0
  26. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/llms/together_llm.py +0 -0
  27. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tasks/__init__.py +0 -0
  28. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tasks/base.py +0 -0
  29. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/base.py +0 -0
  30. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/python.py +0 -0
  31. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/tools/serpapi.py +0 -0
  32. {cognitrix-0.2.0 → cognitrix-0.2.1}/cognitrix/utils/__init__.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: cognitrix
3
- Version: 0.2.0
3
+ Version: 0.2.1
4
4
  Summary: Package for creating AI Agents using llms
5
5
  Home-page: https://github.com/theonlyamos/cognitrix
6
6
  License: Apache2
@@ -47,19 +47,21 @@ Description-Content-Type: text/markdown
47
47
  Cognitrix is an open-source autonous AI agents orchestrator built in Python. It allows you to create and manage AI agents easily.
48
48
 
49
49
  ### Features
50
- - Create and manage AI agents.
51
- - Conversational interface using chat
52
- - Modular architecture for easy extensibility
53
- - Access to various tools like calculators, web - search, etc.
54
- - Support for multiple platforms
50
+ - **`Agent Creation and Management`:** Create, list, and load AI agents with customizable names, tasks, and configurations.
51
+ - **`LLM Integration`:** Seamlessly integrate with multiple LLM providers, including Anthropic (Claude), Cohere, Groq, Google, OpenAI, and Together.
52
+ - **`Modular Architecture`:** Easily extend the framework by adding new tools, agents, and LLM integrations.
53
+ - **`Conversational Interface`:** Interact with AI agents through a command-line interface, providing queries and receiving responses.
54
+ - **`Tool Integration`:** Agents can utilize a variety of tools, such as calculators, web searches, file system browsers, and more.
55
+ - **`Autonomous Agent Mode`:** Enable agents to operate autonomously, visually perceiving the screen, interacting with UI elements, and performing tasks.
56
+ - **`Multimodal Support`:** Agents can handle both text and image inputs/outputs, enabling multimodal interactions.
55
57
 
56
58
  ### Architecture
57
- The core components of Cognitrix are:
59
+ Cognitrix's architecture is designed to be modular and extensible, with core components including:
58
60
 
59
- - `Agent` - Base class for chatbot agents with - support for tools
60
- - `LLM` - Integration with large language models like - Cohere, GPT-3 etc.
61
- - `Tools` - Various utility tools like calculators, search etc.
62
- - `Memory` - For tracking context and state
61
+ 1. **`Agents`:** The base Agent class and specialized classes like AIAssistant for creating and managing AI agents.
62
+ 2. **`LLMs`:** A collection of classes for integrating with various LLM providers, such as Cohere, OpenAI, Claude, and more.
63
+ 3. **`Tools`:** A set of tools that agents can utilize, including calculators, web searches, file system browsers, and more.
64
+ 4. **`Templates`:** Customizable prompt templates for guiding the behavior and output formats of LLMs.
63
65
 
64
66
  The architecture is highly modular and extensible. New tools and capabilities can be easily added.
65
67
 
@@ -116,5 +118,5 @@ Cognitrix is open source and contributions are welcome. See [CONTRIBUTING.md](CO
116
118
  ### License
117
119
  This project is licensed under the MIT license. See [LICENSE.md](LICENSE.md) for more details.
118
120
 
119
- ### Credits
120
- Cognitrix was created by [Amos Amissah](https://github.com/theonlyamos).
121
+ ### Acknowledgments
122
+ Cognitrix was created by [Amos Amissah](https://github.com/theonlyamos) and is heavily inspired by projects like AutoGPT and EngineerGPT. Special thanks to the open-source community for their contributions and the AI companies providing LLM APIs.
@@ -0,0 +1,77 @@
1
+ ### Cognitrix
2
+ Cognitrix is an open-source autonous AI agents orchestrator built in Python. It allows you to create and manage AI agents easily.
3
+
4
+ ### Features
5
+ - **`Agent Creation and Management`:** Create, list, and load AI agents with customizable names, tasks, and configurations.
6
+ - **`LLM Integration`:** Seamlessly integrate with multiple LLM providers, including Anthropic (Claude), Cohere, Groq, Google, OpenAI, and Together.
7
+ - **`Modular Architecture`:** Easily extend the framework by adding new tools, agents, and LLM integrations.
8
+ - **`Conversational Interface`:** Interact with AI agents through a command-line interface, providing queries and receiving responses.
9
+ - **`Tool Integration`:** Agents can utilize a variety of tools, such as calculators, web searches, file system browsers, and more.
10
+ - **`Autonomous Agent Mode`:** Enable agents to operate autonomously, visually perceiving the screen, interacting with UI elements, and performing tasks.
11
+ - **`Multimodal Support`:** Agents can handle both text and image inputs/outputs, enabling multimodal interactions.
12
+
13
+ ### Architecture
14
+ Cognitrix's architecture is designed to be modular and extensible, with core components including:
15
+
16
+ 1. **`Agents`:** The base Agent class and specialized classes like AIAssistant for creating and managing AI agents.
17
+ 2. **`LLMs`:** A collection of classes for integrating with various LLM providers, such as Cohere, OpenAI, Claude, and more.
18
+ 3. **`Tools`:** A set of tools that agents can utilize, including calculators, web searches, file system browsers, and more.
19
+ 4. **`Templates`:** Customizable prompt templates for guiding the behavior and output formats of LLMs.
20
+
21
+ The architecture is highly modular and extensible. New tools and capabilities can be easily added.
22
+
23
+ ### Getting Started
24
+
25
+ ### Installation
26
+ ```bash
27
+ pip install cognitrix
28
+ ```
29
+ or
30
+ ```bash
31
+ pip install https://github.com/theonlyamos/cognitrix/archive/main.zip
32
+ ```
33
+
34
+ ### Usage
35
+ Run with default settings
36
+ ```bash
37
+ cognitrix
38
+ ```
39
+
40
+ List supported platforms
41
+ ```bash
42
+ cognitrix --platforms
43
+ ```
44
+
45
+ Run with specific platform
46
+ ```bash
47
+ cognitrix --platform <platform_name>
48
+ ```
49
+
50
+ Create a new agent
51
+ ```bash
52
+ cognitrix agents --new
53
+ ```
54
+
55
+ List created agents
56
+ ```bash
57
+ cognitrix agents
58
+ ```
59
+
60
+ Run with specific agent
61
+ ```bash
62
+ cognitrix --agent <agent_name>
63
+ ```
64
+
65
+ Print help message
66
+ ```bash
67
+ cognitrix --help
68
+ ```
69
+
70
+ ### Contributing
71
+ Cognitrix is open source and contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for more details.
72
+
73
+ ### License
74
+ This project is licensed under the MIT license. See [LICENSE.md](LICENSE.md) for more details.
75
+
76
+ ### Acknowledgments
77
+ Cognitrix was created by [Amos Amissah](https://github.com/theonlyamos) and is heavily inspired by projects like AutoGPT and EngineerGPT. Special thanks to the open-source community for their contributions and the AI companies providing LLM APIs.
@@ -12,7 +12,7 @@ from pydantic import BaseModel, Field
12
12
  from ..llms.base import LLM
13
13
  from ..tools.base import Tool
14
14
  from ..tasks.base import Task
15
- from ..agents.templates import PROMPT_TEMPLATE
15
+ from ..agents.templates import ASSISTANT_TEMPLATE
16
16
  from ..config import AGENTS_FILE
17
17
 
18
18
  logging.basicConfig(
@@ -35,7 +35,7 @@ class Agent(BaseModel):
35
35
  tools: List[Tool] = Field(default=[])
36
36
  """Tools to be used by agent"""
37
37
 
38
- prompt_template: str = Field(default=PROMPT_TEMPLATE)
38
+ prompt_template: str = Field(default=ASSISTANT_TEMPLATE)
39
39
  """Base system prompt template"""
40
40
 
41
41
  verbose: bool = Field(default=False)
@@ -206,7 +206,7 @@ class Agent(BaseModel):
206
206
  Initialize the llm
207
207
  """
208
208
  self.format_system_prompt()
209
-
209
+
210
210
  query: str|dict = input("\nUser (q to quit): ")
211
211
  while query:
212
212
  try:
@@ -74,20 +74,6 @@ Context: The AI assistant has access to the user's computer and the internet. Th
74
74
  The AI assistant will answer the user's question to the best of its ability, using its knowledge and access to tools provided by the user.
75
75
  The AI assistant has access to the following tools.
76
76
 
77
- You are an expert at world knowledge.
78
- Your task is to step back and paraphrase a question to a more generic
79
- step-back question, which is easier to answer.
80
-
81
- Here are a few examples:
82
- Original Question: Which position did Knox Cunningham hold from May 1955 to Apr 1956?
83
- Stepback Question: Which positions have Knox Cunning- ham held in his career?
84
-
85
- Original Question: Who was the spouse of Anna Karina from 1968 to 1974?
86
- Stepback Question: Who were the spouses of Anna Karina?
87
-
88
- Original Question: Which team did Thierry Audel play for from 2007 to 2008?
89
- Stepback Question: Which teams did Thierry Audel play for in his career?
90
-
91
77
  Remember, functions calls will be processed by the user and the result returned to the AI Asssistant as the next input.
92
78
  A function name is a name of a tool available to the AI Assistant.
93
79
  Whenever there is a function call, always wait for the answer from the user. Do not try to answer that query yourself.
@@ -130,7 +116,8 @@ if it's the final anwers or
130
116
  }
131
117
  if the assistant needs to use a tool to answer the user's query.
132
118
  Don't forget to present the answer to a function call to the user in an informative manner.
133
-
119
+ You should always break down complex tasks into smaller easier ones and perform them one by one.
120
+ You should always check if a task is complete by taking a screenshot of the screen.
134
121
  Begin:
135
122
  """
136
123
 
@@ -8,16 +8,10 @@ from tools import (
8
8
  InternetBrowser, tool
9
9
  )
10
10
 
11
- @tool
12
- def print_hello():
13
- """Returns hello world"""
14
- return 'hello world'
15
-
16
11
  if __name__ == "__main__":
17
12
  llm = Cohere()
18
13
  # llm = TogetherLLM()
19
14
  assistant = AIAssistant(llm=llm, name='Adam')
20
- assistant.add_tool(print_hello())
21
15
  assistant.add_tool(Calculator())
22
16
  assistant.add_tool(YoutubePlayer())
23
17
  assistant.add_tool(WorldNews())
@@ -14,8 +14,9 @@ from .tools import (
14
14
  Calculator, YoutubePlayer,
15
15
  WorldNews, PythonREPL,
16
16
  FSBrowser, SearchTool,
17
- InternetBrowser,
18
- take_screenshot
17
+ InternetBrowser, take_screenshot,
18
+ text_input, key_press, mouse_click,
19
+ mouse_double_click, mouse_right_click
19
20
  )
20
21
 
21
22
  from .config import VERSION
@@ -98,7 +99,12 @@ def start(args: Namespace):
98
99
  if assistant:
99
100
  assistant.llm = platform
100
101
  assistant.name = args.name
101
- assistant.add_tool(take_screenshot())
102
+ assistant.add_tool(take_screenshot)
103
+ assistant.add_tool(text_input)
104
+ assistant.add_tool(key_press)
105
+ assistant.add_tool(mouse_click)
106
+ assistant.add_tool(mouse_double_click)
107
+ assistant.add_tool(mouse_right_click)
102
108
  # assistant.add_tool(Calculator())
103
109
  # assistant.add_tool(YoutubePlayer())
104
110
  # assistant.add_tool(WorldNews())
@@ -1,7 +1,7 @@
1
1
  from pathlib import Path
2
2
  import json
3
3
 
4
- VERSION = '0.2.0'
4
+ VERSION = '0.2.1'
5
5
  SPIRAL_WORKDIR = Path('~').expanduser() / '.cognitrix'
6
6
 
7
7
  AGENTS_FILE = SPIRAL_WORKDIR / 'agents.json'
@@ -3,7 +3,9 @@ from ..tools.tool import tool
3
3
  from ..tools.misc import (
4
4
  Calculator, YoutubePlayer,
5
5
  WorldNews, FSBrowser,
6
- InternetBrowser, take_screenshot
6
+ InternetBrowser, take_screenshot,
7
+ text_input, key_press, mouse_click,
8
+ mouse_double_click, mouse_right_click
7
9
  )
8
10
 
9
11
  from ..tools.python import PythonREPL
@@ -395,20 +395,114 @@ def take_screenshot():
395
395
  }"""
396
396
  screenshot = pyautogui.screenshot()
397
397
 
398
- # Resize the image by 50% while maintaining the aspect ratio
399
- # width, height = screenshot.size
400
- # new_width = int(width * 0.5)
401
- # new_height = int(height * 0.5)
402
- # screenshot = screenshot.resize((new_width, new_height), Image.Resampling.LANCZOS)
398
+ return ['image', screenshot]
403
399
 
404
- # Convert the screenshot to a BytesIO object
405
- # screenshot_bytes = io.BytesIO()
406
- # screenshot.save(screenshot_bytes, format='JPEG')
400
+ @tool
401
+ def text_input(text: str):
402
+ """Use this tool to take make text inputs.
403
+ Args:
404
+ text (str): Text to input.
405
+
406
+ Returns:
407
+ str: Text input completed.
408
+
409
+ Example:
410
+ User: write hello world
411
+ AI Assistant: {
412
+ "type": "function_call",
413
+ "function": "Text Input",
414
+ "arguments": ["hello world"]
415
+ }
416
+ """
417
+ screenshot = pyautogui.write(text, 0.25)
418
+ return 'Text input completed'
407
419
 
408
- # # Convert the BytesIO object to base64
409
- # screenshot_base64 = base64.b64encode(screenshot_bytes.getvalue()).decode('utf-8')
420
+ @tool
421
+ def key_press(key: str):
422
+ """Use this tool to take make key presses.
423
+ Args:
424
+ key (str): Name of key to press.
410
425
 
411
- return ['image', screenshot]
426
+ Returns:
427
+ str: Keypress completed.
428
+
429
+ Example:
430
+ User: Press windows key
431
+ AI Assistant: {
432
+ "type": "function_call",
433
+ "function": "Key Press",
434
+ "arguments": ["win"]
435
+ }
436
+ """
437
+ screenshot = pyautogui.press(key.lower())
438
+
439
+ return 'Keypress completed'
440
+
441
+ @tool
442
+ def mouse_click(x: int, y: int):
443
+ """Use this tool to take make mouse clicks.
444
+ Args:
445
+ x (int): X coordinate of mouse click.
446
+ y (int): y coordinate of mouse click.
447
+
448
+ Returns:
449
+ str: Mouse click completed.
450
+
451
+ Example:
452
+ User: Click on Brave icon
453
+ AI Assistant: {
454
+ "type": "function_call",
455
+ "function": "Mouse Click",
456
+ "arguments": ["123", "456"]
457
+ }
458
+ """
459
+ screenshot = pyautogui.click(x, y)
460
+
461
+ return 'Mouse Click completed'
462
+
463
+ @tool
464
+ def mouse_double_click(x: int, y: int):
465
+ """Use this tool to take make mouse double clicks.
466
+ Args:
467
+ x (int): X coordinate of mouse click.
468
+ y (int): y coordinate of mouse click.
469
+
470
+ Returns:
471
+ str: Mouse double-click completed.
472
+
473
+ Example:
474
+ User: Click on Brave icon
475
+ AI Assistant: {
476
+ "type": "function_call",
477
+ "function": "Mouse Double Click",
478
+ "arguments": ["123", "456"]
479
+ }
480
+ """
481
+ screenshot = pyautogui.doubleClick(x, y)
482
+
483
+ return 'Mouse double-click completed.'
484
+
485
+ @tool
486
+ def mouse_right_click(x: int, y: int):
487
+ """Use this tool to take make mouse right clicks.
488
+ Args:
489
+ x (int): X coordinate of mouse click.
490
+ y (int): y coordinate of mouse click.
491
+
492
+ Returns:
493
+ str: Mouse right-click completed.
494
+
495
+ Example:
496
+ User: Right-click on Brave icon
497
+ AI Assistant: {
498
+ "type": "function_call",
499
+ "function": "Mouse Right Click",
500
+ "arguments": ["123", "456"]
501
+ }
502
+ """
503
+ screenshot = pyautogui.rightClick(x, y)
504
+
505
+ return 'Mouse double-click completed.'
412
506
 
413
507
  # @tool
414
508
  # def get_ui_elements_coordinates():
@@ -9,17 +9,18 @@ def tool(func: Callable, *args, **kwargs):
9
9
  return func(*args, **kwargs)
10
10
 
11
11
  class GenericTool(Tool):
12
- name: str = ' '.join(func.__name__.split('_')).capitalize()
13
- description: str = str(func.__doc__)
14
-
15
- def run(self):
12
+ def run(self, *args, **kwargs):
16
13
  if inspect.iscoroutinefunction(func):
17
14
  raise NotImplementedError("Asynchronous execution is not supported for coroutine functions")
18
15
  return wrapper(*args, **kwargs)
19
16
 
20
- async def arun(self):
17
+ async def arun(self, *args, **kwargs):
21
18
  if inspect.iscoroutinefunction(func):
22
19
  return await wrapper(*args, **kwargs)
23
20
  raise NotImplementedError("Synchronous execution is not supported for synchronous functions")
24
21
 
25
- return GenericTool
22
+ new_tool = GenericTool(
23
+ name=' '.join(func.__name__.split('_')).title(),
24
+ description=str(func.__doc__)
25
+ )
26
+ return new_tool
@@ -1,6 +1,6 @@
1
1
  [tool.poetry]
2
2
  name = "cognitrix"
3
- version = "0.2.0"
3
+ version = "0.2.1"
4
4
  description = "Package for creating AI Agents using llms"
5
5
  authors = ["theonlyamos <theonlyamos@gmail.com>"]
6
6
  license = "Apache2"
cognitrix-0.2.0/README.md DELETED
@@ -1,75 +0,0 @@
1
- ### Cognitrix
2
- Cognitrix is an open-source autonous AI agents orchestrator built in Python. It allows you to create and manage AI agents easily.
3
-
4
- ### Features
5
- - Create and manage AI agents.
6
- - Conversational interface using chat
7
- - Modular architecture for easy extensibility
8
- - Access to various tools like calculators, web - search, etc.
9
- - Support for multiple platforms
10
-
11
- ### Architecture
12
- The core components of Cognitrix are:
13
-
14
- - `Agent` - Base class for chatbot agents with - support for tools
15
- - `LLM` - Integration with large language models like - Cohere, GPT-3 etc.
16
- - `Tools` - Various utility tools like calculators, search etc.
17
- - `Memory` - For tracking context and state
18
-
19
- The architecture is highly modular and extensible. New tools and capabilities can be easily added.
20
-
21
- ### Getting Started
22
-
23
- ### Installation
24
- ```bash
25
- pip install cognitrix
26
- ```
27
- or
28
- ```bash
29
- pip install https://github.com/theonlyamos/cognitrix/archive/main.zip
30
- ```
31
-
32
- ### Usage
33
- Run with default settings
34
- ```bash
35
- cognitrix
36
- ```
37
-
38
- List supported platforms
39
- ```bash
40
- cognitrix --platforms
41
- ```
42
-
43
- Run with specific platform
44
- ```bash
45
- cognitrix --platform <platform_name>
46
- ```
47
-
48
- Create a new agent
49
- ```bash
50
- cognitrix agents --new
51
- ```
52
-
53
- List created agents
54
- ```bash
55
- cognitrix agents
56
- ```
57
-
58
- Run with specific agent
59
- ```bash
60
- cognitrix --agent <agent_name>
61
- ```
62
-
63
- Print help message
64
- ```bash
65
- cognitrix --help
66
- ```
67
-
68
- ### Contributing
69
- Cognitrix is open source and contributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for more details.
70
-
71
- ### License
72
- This project is licensed under the MIT license. See [LICENSE.md](LICENSE.md) for more details.
73
-
74
- ### Credits
75
- Cognitrix was created by [Amos Amissah](https://github.com/theonlyamos).
File without changes