scientific-writer 2.2.3__tar.gz → 2.3.0__tar.gz

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/CHANGELOG.md

@@ -6,6 +6,76 @@ All notable changes to the Scientific Writer project will be documented in this
 
 ---
 
+## [2.3.0] - 2025-11-04
+
+### ✨ Added
+
+#### Manuscript Editing Workflow
+
+- **Automatic Editing Mode Detection** - Smart file routing based on file type
+  - Manuscript files (`.tex`, `.md`, `.docx`, `.pdf`) automatically copied to `drafts/` folder
+  - Image files routed to `figures/` folder
+  - Data files routed to `data/` folder
+  - System recognizes manuscripts in drafts/ as editing tasks, not creation from scratch
+  
+- **EDITING MODE Context** - Clear feedback and instructions
+  - Prominent `⚠️ EDITING MODE` warning displayed when manuscripts detected
+  - Agent receives explicit instructions to edit existing manuscript
+  - Visual `[EDITING MODE]` indicators in CLI output
+  - Progress messages show manuscript file counts separately
+  
+- **Enhanced File Processing** - Improved data file handling
+  - New `get_manuscript_extensions()` function in `core.py`
+  - Updated `process_data_files()` to handle three file categories
+  - Updated `create_data_context_message()` with editing mode detection
+  - Manuscript files tracked separately in processed_info dictionary
+
+### 🔧 Improvements
+
+- **System Instructions (WRITER.md)** - Added comprehensive manuscript editing workflow section
+  - Clear instructions for handling manuscript files from data folder
+  - Defined file routing rules by file type
+  - Detailed editing workflow for the agent
+  - Example scenarios demonstrating the workflow
+  
+- **CLI User Experience** - Better visibility into file processing
+  - Welcome message explains manuscript file routing
+  - File processing shows separate counts for manuscripts, data, and images
+  - Help text updated with manuscript editing information
+  - Consistent `[EDITING MODE]` indicators throughout
+  
+- **API Progress Updates** - Enhanced feedback in programmatic mode
+  - Progress messages report manuscript files separately
+  - Clear indication when manuscripts are copied to drafts/
+  - Better tracking of file processing stages
+
+### 📝 Files Modified
+
+- `scientific_writer/.claude/WRITER.md` - Added "CRITICAL: Manuscript Editing Workflow" section
+- `scientific_writer/core.py` - Added manuscript detection and routing logic
+- `scientific_writer/cli.py` - Updated UI to show editing mode indicators
+- `scientific_writer/api.py` - Enhanced progress reporting for manuscript files
+
+### 🎯 Usage Example
+
+```bash
+# Place a manuscript file in the data folder
+cp my_paper.tex data/
+
+# Run scientific writer
+scientific-writer
+
+# The system will:
+# ✓ Detect my_paper.tex as a manuscript file
+# ✓ Copy it to drafts/ folder (not data/)
+# ✓ Display [EDITING MODE] indicator
+# ✓ Treat the task as editing, not creation
+
+> "Improve the introduction and add 5 more citations"
+```
+
+---
+
 ## [2.2.1] - 2025-11-04
 
 ### 🔧 Improvements

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scientific-writer
-Version: 2.2.3
+Version: 2.3.0
 Summary: AI-powered scientific writing with programmatic API and CLI - powered by Claude Sonnet 4.5 and the Claude Agents SDK
 License: MIT
 License-File: LICENSE

scientific_writer-2.3.0/RELEASE_NOTES_2.3.0.md

@@ -0,0 +1,208 @@
+# Scientific Writer v2.3.0 - Edit Papers Anywhere 📝
+
+## 🎉 Write & Edit Scientific Papers Anywhere
+
+This release introduces **Manuscript Editing Mode** - a powerful new workflow that lets you edit existing papers from anywhere! Simply place your manuscript file (`.tex`, `.md`, `.docx`, `.pdf`) in the `data/` folder, and the system automatically recognizes it as an editing task.
+
+## ✨ What's New
+
+### Automatic Editing Mode Detection
+
+The system now intelligently routes files based on their type:
+
+| 📄 File Type | 📁 Destination | 🎯 Purpose |
+|-------------|---------------|-----------|
+| **Manuscript files** (`.tex`, `.md`, `.docx`, `.pdf`) | `drafts/` | Editing existing manuscripts |
+| **Image files** (`.png`, `.jpg`, `.svg`, etc.) | `figures/` | Figures for your paper |
+| **Data files** (`.csv`, `.txt`, `.json`, etc.) | `data/` | Data for analysis |
+
+### How It Works
+
+```bash
+# 1. Place your manuscript in the data folder
+cp my_research_paper.tex data/
+
+# 2. Run scientific writer
+scientific-writer
+
+# The system automatically:
+# ✓ Detects it's a manuscript file
+# ✓ Copies it to drafts/ folder
+# ✓ Displays [EDITING MODE] indicator
+# ✓ Treats this as an editing task
+
+# 3. Request your edits
+> "Improve the introduction and add 5 more citations to the methods section"
+
+# Result:
+# - Original manuscript preserved
+# - New version created (v2_my_research_paper.tex)
+# - All changes documented in revision_notes.md
+```
+
+### Clear Visual Feedback
+
+When editing mode is active, you'll see clear indicators:
+
+```
+⚠️  EDITING MODE - Manuscript files detected!
+
+📦 Processing files...
+   ✓ Copied 1 manuscript file(s) to drafts/ [EDITING MODE]
+   ✓ Copied 2 image(s) to figures/
+   ✓ Deleted original files from data folder
+
+🔧 TASK: This is an EDITING task, not creating from scratch.
+```
+
+## 🚀 Key Features
+
+### 1. **Works with Multiple Formats**
+- LaTeX (`.tex`) - Your complete paper
+- Markdown (`.md`) - Research notes or drafts
+- Word (`.docx`) - Documents from collaborators
+- PDF (`.pdf`) - Existing publications to revise
+
+### 2. **Smart File Organization**
+- Manuscripts automatically go to `drafts/` folder
+- Images automatically go to `figures/` folder
+- Data files stay in `data/` folder
+- No manual organization needed!
+
+### 3. **Version Control Built-In**
+- Original manuscript preserved
+- New versions created (v2, v3, etc.)
+- Changes documented in `revision_notes.md`
+- Full audit trail of all edits
+
+### 4. **Works in Both CLI and API**
+- Same behavior in interactive CLI mode
+- Same behavior when using programmatic API
+- Consistent experience everywhere
+
+## 💡 Use Cases
+
+### Edit a Paper from a Collaborator
+```bash
+# Collaborator sends you draft.docx
+cp draft.docx data/
+scientific-writer
+> "Convert to LaTeX and add citations for all claims in the introduction"
+```
+
+### Improve an Existing LaTeX Paper
+```bash
+# Working on your submission
+cp neurips_submission.tex data/
+scientific-writer
+> "Address reviewer comments: strengthen methods section and add ablation study"
+```
+
+### Revise Based on Feedback
+```bash
+# Got feedback from advisor
+cp thesis_chapter.tex data/
+scientific-writer
+> "Rewrite the discussion section to address concerns about generalizability"
+```
+
+## 🔧 Technical Details
+
+### Files Modified
+- `scientific_writer/.claude/WRITER.md` - Added manuscript editing workflow instructions
+- `scientific_writer/core.py` - Added manuscript detection and routing logic
+- `scientific_writer/cli.py` - Updated UI with editing mode indicators
+- `scientific_writer/api.py` - Enhanced progress reporting for manuscripts
+
+### New Functions
+- `get_manuscript_extensions()` - Defines manuscript file types
+- Enhanced `process_data_files()` - Routes files intelligently
+- Enhanced `create_data_context_message()` - Provides editing mode context
+
+### Backward Compatibility
+✅ **100% backward compatible**
+- All existing functionality preserved
+- New behavior only activates for manuscript files
+- No breaking changes to API or CLI
+- Existing workflows continue to work
+
+## 📦 Installation
+
+```bash
+# Update to the latest version
+cd claude-scientific-writer
+git pull origin main
+uv sync
+
+# Verify installation
+scientific-writer --version  # Should show 2.3.0
+```
+
+## 📚 Documentation
+
+For detailed information, see:
+- [CHANGELOG.md](CHANGELOG.md) - Complete version history
+- [WRITER.md](scientific_writer/.claude/WRITER.md) - System instructions
+- [README.md](README.md) - Full documentation
+
+## 🎯 Example Workflow
+
+Here's a complete workflow showing the power of editing mode:
+
+```bash
+# You receive a paper draft from a colleague
+cp colleague_draft.docx data/
+
+# Start scientific writer
+scientific-writer
+
+# Request comprehensive edits
+> "Please:
+1. Convert this Word document to LaTeX
+2. Improve the introduction with better flow
+3. Add 10 more recent citations (2023-2024)
+4. Strengthen the methods section
+5. Create a new figure summarizing the results
+6. Format for Nature journal submission"
+
+# The system will:
+# ✓ Recognize this as an editing task
+# ✓ Read the existing document
+# ✓ Apply all requested changes
+# ✓ Create properly versioned outputs
+# ✓ Document all changes
+# ✓ Generate publication-ready LaTeX
+```
+
+## 🙏 What This Means for You
+
+### Before v2.3.0
+- Had to manually specify you're editing
+- Files could go to wrong folders
+- No clear indication of editing vs. creation
+- Manual file organization required
+
+### After v2.3.0
+- ✨ Automatic detection of editing tasks
+- ✨ Smart file routing by type
+- ✨ Clear visual feedback throughout
+- ✨ Zero manual organization needed
+- ✨ Edit papers from **anywhere**!
+
+## 🎊 Summary
+
+Version 2.3.0 makes it effortless to edit scientific papers from any source:
+- Drop manuscript files in `data/` folder
+- System automatically recognizes editing mode
+- Smart routing to correct folders
+- Clear feedback and version control
+- Works with LaTeX, Markdown, Word, and PDF
+
+**Write and edit scientific papers anywhere! 🚀**
+
+---
+
+**Full Changelog**: [CHANGELOG.md](CHANGELOG.md)
+
+**Questions or Issues?** Open an issue on GitHub or check the [documentation](README.md).
+

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "scientific-writer"
-version = "2.2.3"
+version = "2.3.0"
 description = "AI-powered scientific writing with programmatic API and CLI - powered by Claude Sonnet 4.5 and the Claude Agents SDK"
 readme = "README.md"
 requires-python = ">=3.10"

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/scientific_writer/.claude/WRITER.md

@@ -186,6 +186,43 @@ paper_outputs/
         └── supplementary.pdf
 ```
 
+### CRITICAL: Manuscript Editing Workflow
+
+**When manuscript files (.tex, .md, .docx, .pdf) are found in the data/ folder:**
+
+1. **ALWAYS copy manuscript files to the drafts/ folder** (not the data/ folder)
+   - File types to treat as manuscripts: `.tex`, `.md`, `.docx`, `.pdf`
+   - These files should go to `drafts/` folder in the new paper directory
+   - This signals the task is an EDITING task, not creating from scratch
+
+2. **Recognize this as an EDITING task:**
+   - When manuscript files are present in drafts/, your task is to EDIT the existing manuscript
+   - Print: `[HH:MM:SS] EDITING MODE: Found existing manuscript - [filename]`
+   - Print: `[HH:MM:SS] TASK: Editing and improving existing manuscript`
+   - Update progress.md to note this is an editing task
+
+3. **Editing Workflow:**
+   - Read the existing manuscript file(s) from drafts/
+   - Identify the format (.tex, .md, .docx, .pdf)
+   - Follow the user's editing instructions
+   - Create new version with incremented number (v2, v3, etc.)
+   - Document all changes in revision_notes.md
+   - Print: `[HH:MM:SS] EDITING: Reading existing manuscript from drafts/[filename]`
+   - Print: `[HH:MM:SS] EDITING: Creating [version] with requested changes`
+
+4. **What gets copied where:**
+   - **Manuscript files** (.tex, .md, .docx, .pdf) → `drafts/` folder
+   - **Image files** (.png, .jpg, .pdf figures, etc.) → `figures/` folder
+   - **Data files** (CSV, Excel, JSON, etc.) → `data/` folder
+
+5. **Example Scenario:**
+   - User places `my_paper.tex` in `data/` folder
+   - System creates: `paper_outputs/20241104_143000_edit_paper/`
+   - System copies: `my_paper.tex` → `drafts/my_paper.tex`
+   - System recognizes: "This is an editing task"
+   - System prints: `[HH:MM:SS] EDITING MODE: Found manuscript my_paper.tex in drafts/`
+   - System applies edits and creates: `drafts/v2_my_paper.tex` or `drafts/v1_draft.tex` (based on instructions)
+
 ### Naming Conventions
 
 - **Folders:** `lowercase_with_underscores`

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/scientific_writer/__init__.py

@@ -29,7 +29,7 @@ Example:
 from .api import generate_paper
 from .models import ProgressUpdate, PaperResult, PaperMetadata, PaperFiles
 
-__version__ = "2.2.3"
+__version__ = "2.3.0"
 __author__ = "K-Dense"
 __license__ = "MIT"
 

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/scientific_writer/api.py

@@ -77,8 +77,8 @@ async def generate_paper(
     if cwd:
         work_dir = Path(cwd).resolve()
     else:
-        # Default to user's current working directory
-        work_dir = Path.cwd()
+        # Default to user's current working directory (absolute path)
+        work_dir = Path.cwd().resolve()
     
     # Get package directory for copying skills to working directory
     package_dir = Path(__file__).parent.absolute()  # scientific_writer/ directory
@@ -100,7 +100,13 @@ async def generate_paper(
     system_instructions = load_system_instructions(work_dir)
     
     # Add conversation continuity instruction
-    system_instructions += "\n\n" + """
+    system_instructions += "\n\n" + f"""
+IMPORTANT - WORKING DIRECTORY:
+- Your working directory is: {work_dir}
+- ALWAYS create paper_outputs folder in this directory: {work_dir}/paper_outputs/
+- NEVER write to /tmp/ or any other temporary directory
+- All paper outputs MUST go to: {work_dir}/paper_outputs/<timestamp>_<description>/
+
 IMPORTANT - CONVERSATION CONTINUITY:
 - This is a NEW paper request - create a new paper directory
 - Create a unique timestamped directory in the paper_outputs folder
@@ -190,8 +196,12 @@ IMPORTANT - CONVERSATION CONTINUITY:
                     delete_originals=False  # Don't delete when using programmatic API
                 )
                 if processed_info:
+                    manuscript_count = len(processed_info.get('manuscript_files', []))
+                    message = f"Processed {len(processed_info['all_files'])} file(s)"
+                    if manuscript_count > 0:
+                        message += f" ({manuscript_count} manuscript(s) copied to drafts/)"
                     yield ProgressUpdate(
-                        message=f"Processed {len(processed_info['all_files'])} data file(s)",
+                        message=message,
                         stage="complete",
                         percentage=97,
                     ).to_dict()

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/scientific_writer/cli.py

@@ -35,7 +35,8 @@ async def main():
         sys.exit(1)
     
     # Get the current working directory (user's directory) and package directory
-    cwd = Path.cwd()  # User's current working directory
+    # Capture and resolve the current working directory at CLI invocation time
+    cwd = Path.cwd().resolve()  # User's current working directory (absolute path)
     package_dir = Path(__file__).parent.absolute()  # Package installation directory (scientific_writer/)
     
     # Set up Claude skills in the working directory (includes WRITER.md)
@@ -50,7 +51,13 @@ async def main():
     # Add conversation continuity instruction  
     # Note: The Python CLI handles session tracking via current_paper_path
     # These instructions only apply WITHIN a single CLI session, not across different chat sessions
-    system_instructions += "\n\n" + """
+    system_instructions += "\n\n" + f"""
+IMPORTANT - WORKING DIRECTORY:
+- Your working directory is: {cwd}
+- ALWAYS create paper_outputs folder in this directory: {cwd}/paper_outputs/
+- NEVER write to /tmp/ or any other temporary directory
+- All paper outputs MUST go to: {cwd}/paper_outputs/<timestamp>_<description>/
+
 IMPORTANT - CONVERSATION CONTINUITY:
 - The user will provide context in their prompt if they want to continue working on an existing paper
 - If the prompt includes [CONTEXT: You are currently working on a paper in: ...], continue editing that paper
@@ -89,11 +96,13 @@ IMPORTANT - CONVERSATION CONTINUITY:
     print("  2. I'll provide continuous updates during the process")
     print("  3. All outputs saved to: paper_outputs/<timestamp_description>/")
     print("  4. Progress tracked in real-time in progress.md")
-    print(f"\n📁 Output folder: {output_folder}")
+    print(f"\n📁 Working directory: {cwd}")
+    print(f"📁 Output folder: {output_folder}")
     print(f"\n📦 Data Files:")
     print("  • Place files in the 'data/' folder to include them in your paper")
-    print("  • Data files → copied to paper's data/ folder")
-    print("  • Images → copied to paper's figures/ folder")
+    print("  • Manuscript files (.tex, .md, .docx, .pdf) → copied to drafts/ for EDITING")
+    print("  • Data files (csv, txt, json, etc.) → copied to paper's data/ folder")
+    print("  • Images (png, jpg, svg, etc.) → copied to paper's figures/ folder")
     print("  • Original files are automatically deleted after copying")
     print("\n🤖 Intelligent Paper Detection:")
     print("  • I automatically detect when you're referring to a previous paper")
@@ -154,8 +163,11 @@ IMPORTANT - CONVERSATION CONTINUITY:
                 processed_info = process_data_files(cwd, data_files, current_paper_path)
                 if processed_info:
                     data_context = create_data_context_message(processed_info)
-                    data_count = len(processed_info['data_files'])
-                    image_count = len(processed_info['image_files'])
+                    manuscript_count = len(processed_info.get('manuscript_files', []))
+                    data_count = len(processed_info.get('data_files', []))
+                    image_count = len(processed_info.get('image_files', []))
+                    if manuscript_count > 0:
+                        print(f"   ✓ Copied {manuscript_count} manuscript file(s) to drafts/ [EDITING MODE]")
                     if data_count > 0:
                         print(f"   ✓ Copied {data_count} data file(s) to data/")
                     if image_count > 0:
@@ -212,8 +224,11 @@ User request: {user_input}"""
                                 print(f"\n📦 Processing {len(remaining_data_files)} data file(s)...")
                                 processed_info = process_data_files(cwd, remaining_data_files, current_paper_path)
                                 if processed_info:
-                                    data_count = len(processed_info['data_files'])
-                                    image_count = len(processed_info['image_files'])
+                                    manuscript_count = len(processed_info.get('manuscript_files', []))
+                                    data_count = len(processed_info.get('data_files', []))
+                                    image_count = len(processed_info.get('image_files', []))
+                                    if manuscript_count > 0:
+                                        print(f"   ✓ Copied {manuscript_count} manuscript file(s) to drafts/ [EDITING MODE]")
                                     if data_count > 0:
                                         print(f"   ✓ Copied {data_count} data file(s) to data/")
                                     if image_count > 0:
@@ -265,6 +280,7 @@ def _print_help():
     print("  - SUMMARY.md - Project summary and instructions")
     print("\n📦 Data Files:")
     print("  Place files in the 'data/' folder at project root:")
+    print("  • Manuscript files (.tex, .md, .docx, .pdf) → copied to drafts/ for EDITING")
     print("  • Data files (csv, txt, json, etc.) → copied to paper's data/")
     print("  • Images (png, jpg, svg, etc.) → copied to paper's figures/")
     print("  • Files are used as context for the paper")

{scientific_writer-2.2.3 → scientific_writer-2.3.0}/scientific_writer/core.py

@@ -106,6 +106,11 @@ def get_image_extensions() -> set:
     return {'.png', '.jpg', '.jpeg', '.gif', '.bmp', '.tiff', '.tif', '.svg', '.webp', '.ico'}
 
 
+def get_manuscript_extensions() -> set:
+    """Return a set of manuscript file extensions that should go to drafts/ folder."""
+    return {'.tex', '.md', '.docx', '.pdf'}
+
+
 def get_data_files(cwd: Path, data_files: Optional[List[str]] = None) -> List[Path]:
     """
     Get data files either from provided list or from data folder.
@@ -140,7 +145,8 @@ def process_data_files(
 ) -> Optional[Dict[str, Any]]:
     """
     Process data files by copying them to the paper output folder.
-    Images go to figures/, other files go to data/.
+    Manuscript files (.tex, .md, .docx, .pdf) go to drafts/, 
+    images go to figures/, other files go to data/.
     
     Args:
         cwd: Current working directory (project root).
@@ -157,15 +163,20 @@ def process_data_files(
     paper_output = Path(paper_output_path)
     data_output = paper_output / "data"
     figures_output = paper_output / "figures"
+    drafts_output = paper_output / "drafts"
     
     # Ensure output directories exist
     data_output.mkdir(parents=True, exist_ok=True)
     figures_output.mkdir(parents=True, exist_ok=True)
+    drafts_output.mkdir(parents=True, exist_ok=True)
     
     image_extensions = get_image_extensions()
+    manuscript_extensions = get_manuscript_extensions()
+    
     processed_info = {
         'data_files': [],
         'image_files': [],
+        'manuscript_files': [],
         'all_files': []
     }
     
@@ -174,7 +185,17 @@ def process_data_files(
         file_name = file_path.name
         
         # Determine destination based on file type
-        if file_ext in image_extensions:
+        # CRITICAL: Manuscript files go to drafts/ folder for editing workflow
+        if file_ext in manuscript_extensions:
+            destination = drafts_output / file_name
+            file_type = 'manuscript'
+            processed_info['manuscript_files'].append({
+                'name': file_name,
+                'path': str(destination),
+                'original': str(file_path),
+                'extension': file_ext
+            })
+        elif file_ext in image_extensions:
             destination = figures_output / file_name
             file_type = 'image'
             processed_info['image_files'].append({
@@ -225,12 +246,24 @@ def create_data_context_message(processed_info: Optional[Dict[str, Any]]) -> str
     
     context_parts = ["\n[DATA FILES AVAILABLE]"]
     
-    if processed_info['data_files']:
+    # CRITICAL: If manuscript files are present, this is an EDITING task
+    if processed_info.get('manuscript_files'):
+        context_parts.append("\n⚠️  EDITING MODE - Manuscript files detected!")
+        context_parts.append("\nManuscript files (in drafts/ folder for editing):")
+        for file_info in processed_info['manuscript_files']:
+            context_parts.append(f"  - {file_info['name']} ({file_info['extension']}): {file_info['path']}")
+        context_parts.append("\n🔧 TASK: This is an EDITING task, not creating from scratch.")
+        context_parts.append("   → Read the existing manuscript from drafts/")
+        context_parts.append("   → Apply the requested changes/improvements")
+        context_parts.append("   → Create new version following version numbering protocol")
+        context_parts.append("   → Document changes in revision_notes.md")
+    
+    if processed_info.get('data_files'):
         context_parts.append("\nData files (in data/ folder):")
         for file_info in processed_info['data_files']:
             context_parts.append(f"  - {file_info['name']}: {file_info['path']}")
     
-    if processed_info['image_files']:
+    if processed_info.get('image_files'):
         context_parts.append("\nImage files (in figures/ folder):")
         for file_info in processed_info['image_files']:
             context_parts.append(f"  - {file_info['name']}: {file_info['path']}")