abapgit-agent 1.13.2 → 1.13.3

package/abap/CLAUDE.md

@@ -359,7 +359,7 @@ edit src/zcl_auth_handler.clas.abap
 abapgit-agent syntax --files src/zcl_auth_handler.clas.abap
 git add . && git commit -m "feat: add authentication handler"
 git push origin main
-abapgit-agent pull --files src/zcl_auth_handler.clas.abap
+abapgit-agent pull --files src/zcl_auth_handler.clas.abap --sync-xml
 ```
 
 ### AI Tool Guidelines
@@ -425,6 +425,11 @@ abapgit-agent pull --files src/zcl_auth_handler.clas.abap
 1. ✗ Do not run `abapgit-agent transport release`
 2. ✓ Inform the user that transport release is disabled for this project
 
+**After every pull that creates or modifies ABAP objects:**
+1. ✓ Always pass `--sync-xml` — rewrites any XML metadata files that differ from the ABAP serializer output, amends the commit, and re-pulls so git and the ABAP system stay in sync
+2. ✓ If pull output shows `⚠️  X XML file(s) differ from serializer output`, re-run immediately with `--sync-xml`
+3. ✗ Never leave a pull without `--sync-xml` when you authored the objects — abapGit will show **M (modified)** permanently otherwise
+
 ---
 
 ### Quick Decision Tree for AI
@@ -435,11 +440,11 @@ abapgit-agent pull --files src/zcl_auth_handler.clas.abap
 Modified ABAP files?
 ├─ CLAS/INTF/PROG/DDLS files?
 │  ├─ Independent files (no cross-dependencies)?
-│  │  └─ ✅ Use: syntax → commit → push → pull
+│  │  └─ ✅ Use: syntax → commit → push → pull --sync-xml
 │  └─ Dependent files (interface + class, class uses class)?
-│     └─ ✅ Use: skip syntax → commit → push → pull
+│     └─ ✅ Use: skip syntax → commit → push → pull --sync-xml
 └─ Other types (DDLS, FUGR, TABL, etc.)?
-   └─ ✅ Use: skip syntax → commit → push → pull → (if errors: inspect)
+   └─ ✅ Use: skip syntax → commit → push → pull --sync-xml → (if errors: inspect)
 ```
 
 → See `guidelines/workflow-detailed.md` — run: `abapgit-agent ref --topic workflow-detailed`

package/abap/CLAUDE.slim.md

@@ -10,6 +10,8 @@ Read the full ABAP development guide by running:
 abapgit-agent guide
 ```
 
+**Never pipe `abapgit-agent guide` or `abapgit-agent ref` through `head`, `tail`, or any other truncation command. Always read the full output.**
+
 This guide covers: development workflow, ABAP syntax guidelines, object naming, unit testing, and debugging.
 
 > **Humans:** Full guide online at https://sylvoscai.github.io/abapgit-agent/pages/abap-coding-guidelines.html

package/abap/guidelines/abapgit.md

@@ -26,16 +26,12 @@ Structure          | z*.stru.abap                | z*.stru.xml
 Table Type         | z*.ttyp.abap                | z*.ttyp.xml
 ```
 
-```
-Key XML Settings:
-  Class EXPOSURE:   2=Public, 3=Protected, 4=Private
-  Class STATE:      1=Active
-  Table TABCLASS:   TRANSP, POOL, CLUSTER
-  Table DELIVERY:   A=Application, C=Customizing
-  CDS SOURCE_TYPE:  W=View Entity (modern), V=View (legacy)
-  Test Class XML:   <WITH_UNIT_TESTS>X</WITH_UNIT_TESTS>
-  Local Classes:    <CLSCCINCL>X</CLSCCINCL>
-```
+> **CRITICAL: Always write XML files with a UTF-8 BOM (`\ufeff`) as the very first character**, before `<?xml ...`.
+> Without the BOM, abapGit shows the object as **"M" (modified)** after every pull because the
+> serializer always produces XML with BOM — and every byte matters.
+
+> **CRITICAL: Only include fields that abapGit's serializer actually writes. Never add fields with
+> default values.** Extra fields cause a permanent "M" (modified) diff. Follow the exact templates below.
 
 **Searchable keywords**: class xml, interface xml, table xml, cds xml, test class, exposure, serializer, abapgit
 
@@ -46,14 +42,50 @@ abapGit needs XML files to:
 - Store object attributes (description, exposure, state, etc.)
 - Handle object-specific configurations
 
+## Field Presence Rules (CRITICAL)
+
+abapGit's serializer **omits fields that have their default value**. Writing extra fields causes permanent
+"M" (modified) status in abapGit UI. Follow these rules strictly:
+
+### CLAS — Field Presence Rules
+
+| Field | Include when | Omit when |
+|---|---|---|
+| `CLSNAME` | Always | — |
+| `LANGU` | Always | — |
+| `DESCRIPT` | Always | — |
+| `CATEGORY` | Non-standard class (`40`=exception, `05`=test double) | Standard class (default `00`) — **omit** |
+| `EXPOSURE` | **Never for CLAS** | **Always omit** — public (`2`) is the default and is never written |
+| `STATE` | Always (`1`) | — |
+| `CLSCCINCL` | `.clas.locals_def.abap` file exists | No local class files — **omit** |
+| `FIXPT` | Always (`X`) | — |
+| `UNICODE` | Always (`X`) | — |
+| `WITH_UNIT_TESTS` | `.clas.testclasses.abap` file exists | No test class file — **omit** |
+| `MSG_ID` | Class has a message class | No message class — **omit** |
+
+**Field order**: `CLSNAME → LANGU → DESCRIPT → [CATEGORY] → STATE → [CLSCCINCL] → FIXPT → UNICODE → [WITH_UNIT_TESTS] → [MSG_ID]`
+
+### INTF — Field Presence Rules
+
+| Field | Include when | Omit when |
+|---|---|---|
+| `CLSNAME` | Always | — |
+| `LANGU` | Always | — |
+| `DESCRIPT` | Always | — |
+| `EXPOSURE` | Always (`2`) | — (interfaces always have EXPOSURE, unlike classes) |
+| `STATE` | Always (`1`) | — |
+| `UNICODE` | Always (`X`) | — |
+
 ## Object Types and XML Templates
 
 ### Class (CLAS)
 
 **Filename**: `src/zcl_my_class.clas.xml`
 
+**Standard public class** (no local includes, no test class):
+
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_CLAS" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
@@ -61,29 +93,85 @@ abapGit needs XML files to:
     <CLSNAME>ZCL_MY_CLASS</CLSNAME>
     <LANGU>E</LANGU>
     <DESCRIPT>Description of the class</DESCRIPT>
-    <EXPOSURE>2</EXPOSURE>
     <STATE>1</STATE>
+    <FIXPT>X</FIXPT>
     <UNICODE>X</UNICODE>
+   </VSEOCLASS>
+  </asx:values>
+ </asx:abap>
+</abapGit>
+```
+
+**Class with test class** (`.clas.testclasses.abap` exists — add `WITH_UNIT_TESTS`):
+
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<abapGit version="v1.0.0" serializer="LCL_OBJECT_CLAS" serializer_version="v1.0.0">
+ <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
+  <asx:values>
+   <VSEOCLASS>
+    <CLSNAME>ZCL_MY_CLASS</CLSNAME>
+    <LANGU>E</LANGU>
+    <DESCRIPT>Description of the class</DESCRIPT>
+    <STATE>1</STATE>
     <FIXPT>X</FIXPT>
+    <UNICODE>X</UNICODE>
+    <WITH_UNIT_TESTS>X</WITH_UNIT_TESTS>
    </VSEOCLASS>
   </asx:values>
  </asx:abap>
 </abapGit>
 ```
 
-**Key Fields**:
-- `CLSNAME`: Class name (must match filename)
-- `DESCRIPT`: Class description
-- `EXPOSURE`: Exposure (2 = Public, 3 = Protected, 4 = Private)
-- `STATE`: State (1 = Active)
-- `UNICODE`: Unicode encoding (X = Yes)
-- `FIXPT`: Fixed-point arithmetic (X = Yes) - **Always include for correct decimal arithmetic**
+**Class with local includes** (`.clas.locals_def.abap` exists — add `CLSCCINCL` before `FIXPT`):
 
-**Note**: `<FIXPT>X</FIXPT>` is default for modern ABAP. Without it, decimals treated as integers.
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<abapGit version="v1.0.0" serializer="LCL_OBJECT_CLAS" serializer_version="v1.0.0">
+ <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
+  <asx:values>
+   <VSEOCLASS>
+    <CLSNAME>ZCL_MY_CLASS</CLSNAME>
+    <LANGU>E</LANGU>
+    <DESCRIPT>Description of the class</DESCRIPT>
+    <STATE>1</STATE>
+    <CLSCCINCL>X</CLSCCINCL>
+    <FIXPT>X</FIXPT>
+    <UNICODE>X</UNICODE>
+    <WITH_UNIT_TESTS>X</WITH_UNIT_TESTS>
+   </VSEOCLASS>
+  </asx:values>
+ </asx:abap>
+</abapGit>
+```
+
+**Exception class** (`CATEGORY>40` — add `CATEGORY` after `DESCRIPT`):
+
+```xml
+<?xml version="1.0" encoding="utf-8"?>
+<abapGit version="v1.0.0" serializer="LCL_OBJECT_CLAS" serializer_version="v1.0.0">
+ <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
+  <asx:values>
+   <VSEOCLASS>
+    <CLSNAME>ZCX_MY_EXCEPTION</CLSNAME>
+    <LANGU>E</LANGU>
+    <DESCRIPT>My exception</DESCRIPT>
+    <CATEGORY>40</CATEGORY>
+    <STATE>1</STATE>
+    <CLSCCINCL>X</CLSCCINCL>
+    <FIXPT>X</FIXPT>
+    <UNICODE>X</UNICODE>
+   </VSEOCLASS>
+  </asx:values>
+ </asx:abap>
+</abapGit>
+```
 
-**Local Classes**: If the class has local classes (e.g., test doubles), add:
-- `<WITH_UNIT_TESTS>X</WITH_UNIT_TESTS>` - for test classes
-- `<CLSCCINCL>X</CLSCCINCL>` - for local class definitions
+**Key rules**:
+- ❌ **Never include `<EXPOSURE>`** in a CLAS XML — public (2) is the default and abapGit omits it
+- ✅ Always include `<FIXPT>X</FIXPT>` and `<UNICODE>X</UNICODE>`
+- ✅ Add `<WITH_UNIT_TESTS>X</WITH_UNIT_TESTS>` only when `.clas.testclasses.abap` exists
+- ✅ Add `<CLSCCINCL>X</CLSCCINCL>` only when `.clas.locals_def.abap` exists
 
 **Local Class Files**:
 | File | Purpose |
@@ -98,7 +186,7 @@ abapGit needs XML files to:
 **Filename**: `src/zif_my_interface.intf.xml`
 
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_INTF" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
@@ -115,6 +203,9 @@ abapGit needs XML files to:
 </abapGit>
 ```
 
+**Key rules**:
+- ✅ `<EXPOSURE>2</EXPOSURE>` is **always present** for interfaces (unlike classes where it is omitted)
+
 ---
 
 ### Program (PROG)
@@ -122,13 +213,13 @@ abapGit needs XML files to:
 **Filename**: `src/zmy_program.prog.xml`
 
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_PROG" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
    <PROGDIR>
     <NAME>ZMY_PROGRAM</NAME>
-    <SUBC>I</SUBC>
+    <SUBC>1</SUBC>
     <RLOAD>E</RLOAD>
     <FIXPT>X</FIXPT>
     <UCCHECK>X</UCCHECK>
@@ -140,8 +231,12 @@ abapGit needs XML files to:
 
 **Key Fields**:
 - `NAME`: Program name
-- `SUBC`: Subc (I = Include, 1 = Executable, F = Function Group, M = Module Pool, S = Subroutine Pool)
-- `RLOAD`: Rload (E = External, I = Internal)
+- `SUBC`: Program type (`1`=Executable, `I`=Include, `F`=Function Group, `M`=Module Pool, `S`=Subroutine Pool)
+- `RLOAD`: `E`=External
+- `FIXPT`: Fixed-point arithmetic (`X`=Yes) — include for executables
+- `UCCHECK`: Unicode checks active (`X`=Yes)
+
+**Note**: The serializer may also write a `<TPOOL>` section after `<PROGDIR>` if the program has a title text. You do not need to write this when creating new programs — abapGit will add it on the next pull if needed.
 
 ---
 
@@ -150,7 +245,7 @@ abapGit needs XML files to:
 **Filename**: `src/zmy_table.tabl.xml`
 
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_TABL" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
@@ -168,9 +263,20 @@ abapGit needs XML files to:
 
 **Key Fields**:
 - `TABNAME`: Table name
-- `DDTEXT`: Description (NOT DESCRIPT)
-- `TABCLASS`: Table class (TRANSP = Transparent)
-- `CONTFLAG`: Delivery class (A = Application table)
+- `DDTEXT`: Description (**not** `DESCRIPT`)
+- `TABCLASS`: `TRANSP`=Transparent (most common), `POOL`, `CLUSTER`
+- `CONTFLAG`: Delivery class — `A`=Application, `C`=Customizing, `S`=System, `G`=Customizing protected
+
+**Note**: When abapGit serializes an existing table it also writes `<DD09L>` (technical settings) and `<DD03P_TABLE>` (field definitions). These sections are generated automatically from the ABAP Dictionary on pull — you only need the `<DD02V>` header when creating a new table. After the first pull the XML will be expanded with those sections.
+
+**`DD03P` field-level rules** (apply when editing an existing table XML that includes `<DD03P_TABLE>`):
+
+| Rule | Detail |
+|---|---|
+| `SHLPORIGIN` | Include `<SHLPORIGIN>D</SHLPORIGIN>` on fields where the Dictionary provides a value help (e.g. fields with a domain that has fixed values or a search help). Omit on fields with no value help. |
+| Field order for raw-type fields | For fields with no `ROLLNAME` (raw type, e.g. `CHAR`, `NUMC`), the serializer writes `<MASK>` **before** `<DDTEXT>`. For fields with a `ROLLNAME`, only `ROLLNAME` appears (no `MASK` or `DDTEXT`). |
+
+**Why this matters**: Missing `SHLPORIGIN` or wrong `MASK`/`DDTEXT` order causes a permanent diff between git and the system-serialized XML.
 
 ---
 
@@ -181,7 +287,7 @@ abapGit needs XML files to:
 The XML format is identical for both types — only `SOURCE_TYPE` differs:
 
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_DDLS" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
@@ -208,7 +314,7 @@ The XML format is identical for both types — only `SOURCE_TYPE` differs:
 **Filename**: `src/zmy_dtel.dtel.xml`
 
 ```xml
-<?xml version="1.0" encoding="utf-8"?>
+<?xml version="1.0" encoding="utf-8"?>
 <abapGit version="v1.0.0" serializer="LCL_OBJECT_DTEL" serializer_version="v1.0.0">
  <asx:abap xmlns:asx="http://www.sap.com/abapxml" version="1.0">
   <asx:values>
@@ -231,9 +337,9 @@ The XML format is identical for both types — only `SOURCE_TYPE` differs:
 
 **Key Fields**:
 - `ROLLNAME`: Data element name
-- `DDTEXT`: Description (NOT DESCRIPT)
-- `DATATYPE`: Data type (CHAR, NUMC, etc.)
-- `LENG`: Length (e.g., 000010 for 10 characters)
+- `DDTEXT`: Description (**not** `DESCRIPT`)
+- `DATATYPE`: Data type (`CHAR`, `NUMC`, `DATS`, `TIMS`, `INT4`, etc.)
+- `LENG`: Length padded to 6 digits (e.g. `000010` for 10 characters)
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "abapgit-agent",
-  "version": "1.13.2",
+  "version": "1.13.3",
   "description": "ABAP Git Agent - Pull and activate ABAP code via abapGit from any git repository",
   "files": [
     "bin/",
@@ -27,6 +27,7 @@
     "test:cmd:demo": "node tests/run-all.js --cmd --demo",
     "test:cmd:syntax": "node tests/run-all.js --cmd --command=syntax",
     "test:cmd:pull": "node tests/run-all.js --cmd --command=pull",
+    "test:sync-xml": "node tests/run-all.js --sync-xml",
     "test:cmd:inspect": "node tests/run-all.js --cmd --command=inspect",
     "test:cmd:unit": "node tests/run-all.js --cmd --command=unit",
     "test:cmd:view": "node tests/run-all.js --cmd --command=view",

package/src/commands/pull.js

@@ -5,6 +5,7 @@
 const { printHttpError } = require('../utils/format-error');
 const fs = require('fs');
 const pathModule = require('path');
+const { execSync } = require('child_process');
 
 // Calculate display width accounting for emoji (2 cells) vs ASCII (1 cell)
 function calcWidth(str) {
@@ -42,6 +43,7 @@ module.exports = {
   async execute(args, context) {
     const { loadConfig, AbapHttp, gitUtils, getTransport, getSafeguards, getConflictSettings, getTransportSettings } = context;
     const verbose = args.includes('--verbose');
+    const syncXml = args.includes('--sync-xml');
 
     // Check project-level safeguards
     const safeguards = getSafeguards();
@@ -198,13 +200,13 @@ module.exports = {
       }
     }
 
-    await this.pull(gitUrl, branch, files, transportRequest, loadConfig, AbapHttp, jsonOutput, undefined, conflictMode, verbose);
+    await this.pull(gitUrl, branch, files, transportRequest, loadConfig, AbapHttp, jsonOutput, undefined, conflictMode, verbose, syncXml);
   },
 
-  async pull(gitUrl, branch = 'main', files = null, transportRequest = null, loadConfig, AbapHttp, jsonOutput = false, gitCredentials = undefined, conflictMode = 'abort', verbose = false) {
+  async pull(gitUrl, branch = 'main', files = null, transportRequest = null, loadConfig, AbapHttp, jsonOutput = false, gitCredentials = undefined, conflictMode = 'abort', verbose = false, syncXml = false, isRepull = false) {
     const TERM_WIDTH = process.stdout.columns || 80;
 
-    if (!jsonOutput) {
+    if (!jsonOutput && !isRepull) {
       console.log(`\n🚀 Starting pull for: ${gitUrl}`);
       console.log(`   Branch: ${branch}`);
       if (files && files.length > 0) {
@@ -438,6 +440,71 @@ module.exports = {
         throw err;
       }
 
+      // --- Post-pull XML sync ---
+      // abapGit's status calculation already identified which XML files differ
+      // (match=false) — only those are returned in local_xml_files.
+      const localXmlFiles = result.local_xml_files || result.LOCAL_XML_FILES || [];
+
+      if (localXmlFiles.length > 0) {
+        const diffFiles = [];
+        for (const f of localXmlFiles) {
+          const relPath = ((f.path || f.PATH || '') + (f.filename || f.FILENAME || '')).replace(/^\//, '');
+          const absPath = pathModule.join(process.cwd(), relPath);
+          if (!fs.existsSync(absPath)) continue;
+          const incoming = Buffer.from(f.data || f.DATA, 'base64');
+          // Double-check: only write if bytes actually differ (guard against encoding quirks)
+          const current = fs.readFileSync(absPath);
+          if (!current.equals(incoming)) {
+            diffFiles.push({ relPath, absPath, incoming });
+          }
+        }
+
+        if (diffFiles.length > 0 && !syncXml) {
+          console.log(`\n⚠️  ${diffFiles.length} XML file(s) differ from serializer output:`);
+          for (const f of diffFiles) console.log(`   ${f.relPath}`);
+          console.log(`   Run with --sync-xml to accept serializer output and amend the last commit`);
+        } else if (diffFiles.length > 0 && syncXml) {
+          console.log(`\n🔄 Syncing ${diffFiles.length} XML file(s) to match serializer output:`);
+          for (const f of diffFiles) console.log(`   ${f.relPath}`);
+
+          // 1. Write serializer XML to disk
+          for (const f of diffFiles) fs.writeFileSync(f.absPath, f.incoming);
+
+          // 2. Stage changed XML files
+          const quotedPaths = diffFiles.map(f => `"${f.relPath}"`).join(' ');
+          execSync(`git add ${quotedPaths}`, { cwd: process.cwd() });
+
+          // 3. Amend last commit
+          execSync('git commit --amend --no-edit', { cwd: process.cwd() });
+
+          // 4. Push with force-with-lease; if no upstream, set it automatically
+          let pushed = false;
+          try {
+            execSync('git push --force-with-lease', { cwd: process.cwd(), stdio: 'pipe' });
+            pushed = true;
+          } catch (pushErr) {
+            const msg = (pushErr.stderr || pushErr.stdout || pushErr.message || '').toString();
+            if (msg.includes('no upstream branch') || msg.includes('has no upstream')) {
+              // Branch not yet pushed — set upstream and force push (amend requires force)
+              execSync(`git push --force-with-lease --set-upstream origin ${branch}`, { cwd: process.cwd(), stdio: 'pipe' });
+              pushed = true;
+            }
+            // Any other push error (no remote at all, auth failure, etc.) → skip silently
+          }
+
+          if (pushed) {
+            console.log(`   Re-pulling so ABAP system matches the amended commit...`);
+            // 5. Re-pull so ABAP system matches the amended commit (no sync loop)
+            await this.pull(gitUrl, branch, files, transportRequest, loadConfig, AbapHttp, jsonOutput, gitCredentials, conflictMode, verbose, false, true);
+            console.log(`\n✅ Synced ${diffFiles.length} XML file(s), amended commit, re-pulled`);
+          } else {
+            // No remote at all — files are written and committed locally
+            console.log(`\n✅ Synced ${diffFiles.length} XML file(s), amended commit`);
+            console.log(`   Push skipped (no remote). Push manually to sync with remote.`);
+          }
+        }
+      }
+
       return result;
     } catch (error) {
       if (error._isPullError) {