nind 2.2.1__tar.gz → 2.3.0__tar.gz

{nind-2.2.1 → nind-2.3.0}/CLAUDE.md

@@ -33,7 +33,7 @@ To run tests manually instead of via the full `gbuild.sh` cycle: `ctest --output
 
 C++ unit tests live in `tst/cpp/unit/<Module>/` (one executable per library, one ctest entry each) and use **doctest**, vendored as the single header `tst/cpp/unit/doctest.h` (v2.4.12) — no system package or download needed, so they are always built. Each executable accepts doctest options, e.g. `NindIndexUnitTests -tc='NindIndexRobustness*' -s`. Corrupt/hostile-input tests live in the `*RobustnessTest.cpp` files: they patch or byte-by-byte corrupt files produced by the writers and require every read to either succeed or throw a `FileException` — run them under `-a ON` so ASan/UBSan/LSan turn any memory error, UB or leak into a failure. (`-a ON` needs the GCC sanitizer runtimes — `libasan`, `libubsan`, `liblsan` packages on RHEL-like systems.)
 
-Memory-safety conventions in the C++ core (keep them when touching it): readers must never trust a length/offset/count read from a file — bound it against the buffer (`NindFile::get*` check before advancing) or the real file size (`NindFile::getCurrentFileSize()`); byte assembly is done in `uint32_t`/`uint64_t` (never `int`, which overflows); chains read from files (indirection blocks, compound words) must be provably finite; `NindFile`/`NindPadFile` are non-copyable; critical sections use the RAII `NindCriticalSection`, never raw `beginCriticalSection`/`endCriticalSection`.
+Memory-safety conventions in the C++ core (keep them when touching it): readers must never trust a length/offset/count read from a file — bound it against the buffer (`NindFile::get*` check before advancing) or the real file size (`NindFile::getCurrentFileSize()`); byte assembly is done in `uint32_t`/`uint64_t` (never `int`, which overflows); chains read from files (indirection blocks, compound words) must be provably finite; `NindFile`/`NindPadFile` are non-copyable; critical sections use the RAII `NindCriticalSection`, never raw `beginCriticalSection`/`endCriticalSection`; a file has at most one writer — `NindPadFile`/`NindLexiconFile` writers take a non-blocking exclusive `flock` (`NindFile::lockExclusive`, no-op on Windows) and create files with `"w+bx"` (never truncating a file another writer just created), so a second writer, even in-process, gets an exception instead of corrupting the file (readers never lock).
 
 `-DDEBUG_NIND` is added automatically in `Debug` and `RelWithDebInfo` modes (and in `Release` if configured with `-DWITH_DEBUG_MESSAGES=ON`) to enable conditional debug output.
 
@@ -80,9 +80,11 @@ NindIndex (deps: NindBasics, NindRetrolexicon)
    — NindTermIndex: the inverted file itself, as an index file
    — NindLocalIndex: per-document local index file
    — NindIndex: ties the above into a full inverted/local index file
+   — NindIndexMerge: offline merge of index shards (disjoint documents) into one standard index
 
 NindAmose (deps: NindIndex, NindRetrolexicon)
    — adapts nind to Amose's richer term-type model (see internal doc refs "LAT2015.JYS.448")
+   — NindAmoseMerge: NindIndexMerge for Amose indexes (retro lexicon, recomputed term counters)
 
 NindPy (deps: NindBasics, NindRetrolexicon, NindLexicon, NindIndex sources, compiled in directly)
    — pybind11 bindings, built only through the Python wheel (scikit-build-core), not gbuild.sh
@@ -105,7 +107,8 @@ Parallels the C++ layering with lowercase-suffixed module names for the index cl
 - `NindFile.py` — low-level binary codec. Historically read-only-oriented (the `litXxx` getters have long existed for every format), but now has a full writer counterpart too: `ejcritNombre1/3/4/5`, `ejcritNombreULat`/`ejcritNombreSLat` (both varint encodings, full tier range matching the readers and the C++ implementation), `ejcritChaine`, `ejcritZejros`.
 - `NindPadFile.py` — the shared "pad file" envelope (header, indirection block(s), specifics, identification trailer); read-only, no writer (mirrors `NindBasics::NindPadFile`, but nothing in Python subclasses it as a writer the way `NindIndex`/`NindRetrolexicon` do in C++).
 - `NindRetrolexicon.py`, `NindLexiconindex.py`, `NindTermindex.py`, `NindLocalindex.py` — ergonomic read-only Python wrappers around the corresponding `NindIndex`-family C++ classes; their hot-path lookups delegate to `nind._native`, as does the shared structural-diagnostic base (`analyseFichierPadFile`/`analyseFichierIndex`, inherited from `NindPadFile.py`/`NindIndex.py`). The remaining, per-format diagnostic/analysis methods (`dumpeFichier`, `debogueIndex`, `donneCollisions`, `donneMax`, `donneClef`, `afficheTerme`, `afficheDocument`, and each class's own `analyseFichierXxx` beyond the shared base) remain hand-rolled pure-Python parsing, since `nind._native` doesn't expose that introspection surface yet (see "Planned follow-up work" below). `NindLexiconindex` is hash-bucketed (`clefB(word) % nombreIndirection`), the other two are directly indexed by id.
-- `nind_engine.py` — `NindIndexer` (writer) and `NindEngine` (BM25-style search) built on top of the above. `NindIndexer` is the *only* Python writer for the index-family formats (`.nindlexiconindex`/`.nindtermindex`/`.nindlocalindex`) — it builds them via `nind._native`'s `is_writer=True` constructors. It does not write `.nindretrolexicon` (not needed by `NindEngine`, which only supports simple, non-compound words). Indexing is parallel but every file keeps a single writer: files are tokenized in worker processes (`workers` argument of `index_files`; in-process below `PARALLEL_MIN_FILES` files), then the term and local index files are written concurrently from two threads through the flat-buffer bulk bindings `NindTermIndex.set_term_defs_arrays`/`NindLocalIndex.set_local_defs_arrays`, which take `array('I')` buffers (building `Document`/`Term`/`Localisation` objects in Python cost ~20× the write itself) and release the GIL. The `*_local_def*_arrays` writers split a term's positions into entries of ≤255 (`<nbreLocalisations>` is one byte).
+- `nind_engine.py` — `NindIndexer` (writer) and `NindEngine` (BM25-style search) built on top of the above. `NindIndexer` is the *only* Python writer for the index-family formats (`.nindlexiconindex`/`.nindtermindex`/`.nindlocalindex`) — it builds them via `nind._native`'s `is_writer=True` constructors. It does not write `.nindretrolexicon` (not needed by `NindEngine`, which only supports simple, non-compound words). Indexing is parallel but every file keeps a single writer: files are tokenized in worker processes (`workers` argument of `index_files`; in-process below `PARALLEL_MIN_FILES` files), then the term and local index files are written concurrently from two threads through the flat-buffer bulk bindings `NindTermIndex.set_term_defs_arrays`/`NindLocalIndex.set_local_defs_arrays`, which take `array('I')` buffers (building `Document`/`Term`/`Localisation` objects in Python cost ~20× the write itself) and release the GIL. The `*_local_def*_arrays` writers split a term's positions into entries of ≤255 (`<nbreLocalisations>` is one byte). Every build goes into a new generation directory published atomically (see `_index_store.py` below), so rebuilding never disturbs readers. With `shards=N` it instead builds N independent indexes over contiguous parts of the files in worker processes (in a temporary directory inside the new generation), then merges them with `nind._native.merge_indexes`; opt-in because the merge costs about as much as writing the index again (see "Sharded indexing" below), so it only pays off when per-document work dominates.
+- `_index_store.py` — the on-disk layout that makes every `NindIndexer`/`NindEngine` operation safe without the caller doing anything: each build writes a fresh, never-modified generation directory `<prefix>.gen-<n>-<hex>/`, flushed to disk (`fsync`) then published by atomically replacing the pointer file `<prefix>.CURRENT` (a pointer file rather than a symlink: atomic on Windows too). Readers open every file of the generation `CURRENT` names under a shared `flock` on its `generation.lock` (only while opening: open files stay readable after deletion on POSIX); `NindEngine` checks `CURRENT` (one `stat`) at the start of each public call and switches to a newly published build, each call answering from one snapshot (`_Snapshot`), calls serialized by an `RLock` (the native readers aren't thread-safe). Writers of one prefix are serialized by `<prefix>.lock`; after publishing, the writer deletes older generations nobody is opening, crashed builds' leftovers, and flat files of indexes written before generations existed (still readable: no `CURRENT` = flat layout). A failed build never touches `CURRENT`. Lock fds are closed in forked children (`os.register_at_fork`) — otherwise a worker outliving a killed writer would keep the index locked forever — and `NindIndexer`'s pool workers exit when their parent dies. Windows: `msvcrt` locks are exclusive only, so readers don't lock and rely on retrying; generations still open can't be deleted and are retried at the next build.
 - `Nind_*.py` at the package root — standalone CLI scripts (dump a document, convert corpora, search, diagnostics)
 - `amose/` — Amose-specific conversion/parsing scripts (Lucene dump, XML-CLEF, sample corpora)
 
@@ -124,7 +127,11 @@ Binary file formats are specified in EBNF grammar (see README). The Python packa
 
 Not part of this: `.nindretrolexicon` writing (`NindIndexer` still doesn't write this format — unrelated, no change planned).
 
-**Sharded indexing (planned next, after the in-process parallelism above).** Nind files are single-writer by design (no locking, readers resync lazily; see `NindPadFile::getEntryPos`), and every write rewrites the shared specifics/identification tail at EOF, so concurrent writers on one file would serialize anyway — sharing a file between writers is explicitly *not* the plan. Instead, scale writes across N independent shards (each its own lexicon + term + local files, one single-threaded writer each, over a partition of the documents), then either:
-- merge offline: union the lexicons into a new one, remap every shard's term ids, concatenate posting lists with doc ids kept in increasing order (the term index stores relative doc ids), and remap term ids inside each shard's local index; or
-- search across shards: `NindEngine` fans out per shard, but BM25 needs corpus-wide statistics (N, df, avgdl) summed over all shards, not per-shard values.
-Expected cost ~1–2 weeks including tests; zero overhead on the write path.
+**Sharded indexing — offline merge implemented; follow-ups below.** Nind files are single-writer by design (no locking, readers resync lazily; see `NindPadFile::getEntryPos`), and every write rewrites the shared specifics/identification tail at EOF, so concurrent writers on one file would serialize anyway — sharing a file between writers is explicitly *not* the plan. Writes scale instead across N independent shards (each its own lexicon + term + local files, one single-threaded writer each, over a partition of the documents), merged offline into one standard index that every existing reader uses unchanged (search across shards was considered and rejected: every reader, Amose's included, would have to learn about shards, and BM25 would need corpus-wide statistics summed over them).
+
+How the merge works (`NindIndexMerge`, `src/cpp/NindIndex/`): shard lexicons are enumerated with `NindLexiconIndex::getEntries` (walks the hash buckets; no retro lexicon needed) and unioned with words canonicalised as (union id of the prefix, last component) — which also gives the exact merged size, used as the lexicon's hash modulo; documents are listed with `NindLocalIndex::getDocIdents` and checked disjoint *before anything is written*; each merged term's postings are gathered from every shard (per term, so memory stays small) and each document is copied with its term ids remapped (local term order is kept, so Amose's per-occurrence reading order survives). A failed merge removes the target files. `NindAmoseMerge` (`src/cpp/NindAmose/`) forces a retro lexicon and recomputes the 8 term counters, a term's type being derived from its components as in `NindLexiconAmose::getWord` (unique-term counts aren't additive across shards); they're written once at the end with `NindTermIndex::setSpecificWords`. The plain merge refuses shards with term specifics.
+
+Follow-ups:
+- **Merge cost is I/O-syscall bound, and so are all writes.** Merging 10k documents / 58k terms from 4 shards makes ~2.3M `lseek`, ~1.5M `read`, ~280k `write` and ~250k `rt_sigaction` calls: `NindIndex::getDefinition` discards the stdio buffer (`flush`) then seeks+reads twice (indirection, then data) on every read, `NindFile::writeBuffer` flushes every write, and every critical section installs/restores a signal handler. A direct build pays the same per definition, so a merge costs about a full rewrite. Reading with `pread` (one syscall per read, no buffer discard) and a bulk mode for a writer creating a fresh file (no per-write flush, tail written once) would speed up merges, direct builds and search alike — mind the reader-resync semantics above when touching this.
+- **CLI for Amose/LIMA** multi-process indexing: a small C++ program (like the legacy `tst/cpp/NindAmose/` tools) running `NindAmoseMerge` over shard prefixes. `NindAmoseMerge` isn't bound in `nind._native` (the wheel doesn't compile `NindAmose`).
+- `NindLexiconAmose::getWord` dereferences past the end of the component list for the lexicon word `§` alone (the first prefix of every named entity): undefined behaviour, seen as a `bad_alloc`. Normal lookups never ask for that id; the merge doesn't call `getWord`.

{nind-2.2.1 → nind-2.3.0}/CMakeLists.txt

@@ -113,7 +113,7 @@ endif ()
 
 ########### setting project verion number ###############
 
-set(NIND_VERSION "2.2.1")
+set(NIND_VERSION "2.3.0")
 
 string(REGEX REPLACE "^([0-9]+)\\.[0-9]+\\.[0-9]+$" "\\1" NIND_VERSION_MAJOR "${NIND_VERSION}")
 string(REGEX REPLACE "^[0-9]+\\.([0-9]+)\\.[0-9]+$" "\\1" NIND_VERSION_MINOR "${NIND_VERSION}")

{nind-2.2.1 → nind-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: nind
-Version: 2.2.1
+Version: 2.3.0
 Summary: Python frontend for the Nind Indexing Module, with its native C++ index stack built in
 Requires-Python: >=3.8
 Description-Content-Type: text/markdown

{nind-2.2.1 → nind-2.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "nind"
-version = "2.2.1"
+version = "2.3.0"
 description = "Python frontend for the Nind Indexing Module, with its native C++ index stack built in"
 readme = "README.md"
 requires-python = ">=3.8"
@@ -25,7 +25,7 @@ testpaths = ["tst/py"]
 pythonpath = ["src/py"]
 
 [tool.bumpver]
-current_version = "v2.2.1"
+current_version = "v2.3.0"
 version_pattern = "vMAJOR.MINOR.PATCH"
 commit_message = "Bump version {old_version} -> {new_version}"
 commit = true

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindAmose/CMakeLists.txt

@@ -8,6 +8,8 @@ SET(NindAmose_LIB_SRCS
   NindTermAmose.cpp
   NindLocalAmose.h
   NindLocalAmose.cpp
+  NindAmoseMerge.h
+  NindAmoseMerge.cpp
 )
 
 add_library(NindAmose SHARED ${NindAmose_LIB_SRCS})
@@ -22,4 +24,5 @@ install(FILES
     NindTermAmose.h 
     NindLocalAmose.h 
     NindLexiconAmose.h
+    NindAmoseMerge.h
   DESTINATION include/NindAmose)

nind-2.3.0/src/cpp/NindAmose/NindAmoseMerge.cpp

@@ -0,0 +1,77 @@
+//
+// C++ Implementation: NindAmoseMerge
+//
+// Description: Fusion hors ligne d'index Amose, voir NindAmoseMerge.h
+//
+// Author: Claude <noreply@anthropic.com>
+//
+// Copyright: 2014-2017 LATEJCON. See LICENCE.md file that comes with this distribution
+// This file is part of NIND (as "nouvelle indexation").
+// NIND is free software: you can redistribute it and/or modify it under the terms of the
+// GNU Less General Public License (LGPL) as published by the Free Software Foundation,
+// (see <http://www.gnu.org/licenses/>), either version 3 of the License, or any later version.
+// NIND is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
+// even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU Less General Public License for more details.
+////////////////////////////////////////////////////////////
+#include "NindAmoseMerge.h"
+using namespace latecon::nindex;
+using namespace std;
+////////////////////////////////////////////////////////////
+//mesme nombre et mesme ordre que NindTermAmose
+#define NOMBRE_COMPTEURS 8
+////////////////////////////////////////////////////////////
+static NindIndexMerge::Options forceRetrolexicon(const NindIndexMerge::Options &options)
+{
+    NindIndexMerge::Options result = options;
+    result.withRetrolexicon = true;      //NindLexiconAmose a toujours un retro lexique
+    return result;
+}
+////////////////////////////////////////////////////////////
+//brief Prepare the merge of the specified shards
+//param shardNames absolute path file names without extension of the shards
+//param options indirection block sizes of the merged files (withRetrolexicon is forced) */
+NindAmoseMerge::NindAmoseMerge(const vector<string> &shardNames,
+                               const Options &options):
+    NindIndexMerge(shardNames, forceRetrolexicon(options)),
+    m_uniqueTermCount(4, 0),
+    m_termOccurrences(4, 0)
+{
+}
+////////////////////////////////////////////////////////////
+NindAmoseMerge::~NindAmoseMerge()
+{
+}
+////////////////////////////////////////////////////////////
+unsigned int NindAmoseMerge::getTermSpecificsNumber() const
+{
+    return NOMBRE_COMPTEURS;
+}
+////////////////////////////////////////////////////////////
+//compte le terme selon son type, dejduit de ses composants comme NindLexiconAmose::getWord
+void NindAmoseMerge::countTerm(const list<string> &components,
+                               const list<NindTermIndex::TermCG> &termDef)
+{
+    AmoseTypes type = (components.size() == 1) ? SIMPLE_TERM : MULTI_TERM;
+    if (!components.empty() && components.front() == "§") type = NAMED_ENTITY;
+    unsigned int occurrences = 0;
+    for (list<NindTermIndex::TermCG>::const_iterator it = termDef.begin(); it != termDef.end(); it++)
+        occurrences += (*it).frequency;
+    m_uniqueTermCount[type] += 1;
+    m_uniqueTermCount[ALL] += 1;
+    m_termOccurrences[type] += occurrences;
+    m_termOccurrences[ALL] += occurrences;
+}
+////////////////////////////////////////////////////////////
+//mesme ordre que NindTermAmose::setCountsAsList
+list<unsigned int> NindAmoseMerge::getTermSpecifics() const
+{
+    list<unsigned int> result;
+    const AmoseTypes types[] = {ALL, SIMPLE_TERM, MULTI_TERM, NAMED_ENTITY};
+    for (unsigned int i = 0; i < 4; i++) {
+        result.push_back(m_uniqueTermCount[types[i]]);
+        result.push_back(m_termOccurrences[types[i]]);
+    }
+    return result;
+}
+////////////////////////////////////////////////////////////

nind-2.3.0/src/cpp/NindAmose/NindAmoseMerge.h

@@ -0,0 +1,58 @@
+//
+// C++ Interface: NindAmoseMerge
+//
+// Description: Fusion hors ligne d'index Amose (voir NindIndexMerge) : le lexique fusionnej a
+// toujours un retro lexique, et les compteurs du fichier inverse (termes uniques et occurrences
+// par type) sont recalculejs sur l'index fusionnej, le type d'un terme ejtant dejduit de ses
+// composants comme le fait NindLexiconAmose::getWord.
+//
+// Author: Claude <noreply@anthropic.com>
+//
+// Copyright: 2014-2017 LATEJCON. See LICENCE.md file that comes with this distribution
+// This file is part of NIND (as "nouvelle indexation").
+// NIND is free software: you can redistribute it and/or modify it under the terms of the
+// GNU Less General Public License (LGPL) as published by the Free Software Foundation,
+// (see <http://www.gnu.org/licenses/>), either version 3 of the License, or any later version.
+// NIND is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
+// even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU Less General Public License for more details.
+////////////////////////////////////////////////////////////
+#ifndef NindAmoseMerge_H
+#define NindAmoseMerge_H
+////////////////////////////////////////////////////////////
+#include "NindIndex/NindIndexMerge.h"
+#include "NindLexiconAmose.h"
+#include "NindCommonExport.h"
+#include <string>
+#include <list>
+#include <vector>
+////////////////////////////////////////////////////////////
+namespace latecon {
+    namespace nindex {
+////////////////////////////////////////////////////////////
+/**\brief Merges several Amose indexes (shards over disjoint sets of documents) into one */
+class DLLExportLexicon NindAmoseMerge : public NindIndexMerge {
+public:
+    /**\brief Prepare the merge of the specified shards
+    *\param shardNames absolute path file names without extension of the shards
+    *\param options indirection block sizes of the merged files (withRetrolexicon is forced) */
+    NindAmoseMerge(const std::vector<std::string> &shardNames,
+                   const Options &options = Options());
+
+    virtual ~NindAmoseMerge();
+
+protected:
+    virtual unsigned int getTermSpecificsNumber() const;
+    virtual void countTerm(const std::list<std::string> &components,
+                           const std::list<NindTermIndex::TermCG> &termDef);
+    virtual std::list<unsigned int> getTermSpecifics() const;
+
+private:
+    std::vector<unsigned int> m_uniqueTermCount;        //par AmoseTypes
+    std::vector<unsigned int> m_termOccurrences;        //par AmoseTypes
+};
+////////////////////////////////////////////////////////////
+    } // end namespace
+} // end namespace
+#endif
+////////////////////////////////////////////////////////////

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindBasics/NindFile.cpp

@@ -18,6 +18,9 @@
 #include "NindFile.h"
 #include <string.h>
 #include <sys/stat.h>
+#ifndef _WIN32
+#include <sys/file.h>
+#endif
 //#include <iostream>
 using namespace latecon::nindex;
 using namespace std;
@@ -124,6 +127,18 @@ bool NindFile::open(const string &accessMode)
     return true;
 }
 ////////////////////////////////////////////////////////////
+//brief Take an advisory exclusive lock on the open file, without waiting
+//return true if the lock was taken, false if another open of the file holds it */
+bool NindFile::lockExclusive()
+{
+#ifdef _WIN32
+    return true;
+#else
+    //flock est par description de fichier ouvert : refuse aussi un 2e ejcrivain du mesme processus
+    return flock(fileno(m_file), LOCK_EX | LOCK_NB) == 0;
+#endif
+}
+////////////////////////////////////////////////////////////
 //brief Close file associated with. Success in any case. */
 void NindFile::close()
 {

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindBasics/NindFile.h

@@ -46,6 +46,12 @@ public:
     *\return true if success, false otherwise */
     bool open(const std::string &accessMode);
 
+    /**\brief Take an advisory exclusive lock on the open file, without waiting: a second
+    * writer on the same file (in any process) is refused instead of silently corrupting it.
+    * The lock is released when the file is closed. No-op (always true) on Windows.
+    *\return true if the lock was taken, false if another open of the file holds it */
+    bool lockExclusive();
+
     /**\brief Close file associated with. Success in any case. */
     void close();
 

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindBasics/NindPadFile.cpp

@@ -84,6 +84,9 @@ NindPadFile::NindPadFile(const string &fileName,
     if (m_isWriter) {
         //si fichier ejcrivain, ouvre en ecriture + lecture
         bool isOpened = m_file.open("r+b");
+        //un seul ejcrivain par fichier : un 2e (de ce processus ou d'un autre) est refusej
+        if (isOpened && !m_file.lockExclusive())
+            throw NindPadFileException("NindPadFile::NindPadFile already opened by another writer " + m_fileName);
         if (isOpened) {
             //si le fichier existe, l'analyse pour trouver le mapping des blocs et l'identification
             //lit la taille d'une donneje indexeje at la taille des spejcifiques
@@ -112,8 +115,12 @@ NindPadFile::NindPadFile(const string &fileName,
             //la taille du bloc de dejfinitions doit estre spejcifieje diffejrente de 0
             if (dataEntriesBlocSize == 0)
                 throw NindPadFileException("NindPadFile::NindPadFile null dataEntriesBlocSize " + m_fileName);
-            isOpened = m_file.open("w+b");
+            //crejation exclusive ("x") : un autre ejcrivain qui crejerait le mesme fichier au mesme
+            //moment n'est pas tronquej, c'est cette ouverture-ci qui ejchoue
+            isOpened = m_file.open("w+bx");
             if (!isOpened) throw OpenFileException("NindPadFile::NindPadFile open error " + m_fileName);
+            if (!m_file.lockExclusive())
+                throw NindPadFileException("NindPadFile::NindPadFile already opened by another writer " + m_fileName);
             //taille d'une donneje indexeje et taille des spejcifiques en teste de fichier
             m_file.createBuffer(TAILLE_ENTETE_FIXE);
             m_file.putInt1(m_dataEntrySize);
@@ -267,6 +274,13 @@ string NindPadFile::getFileName()
     return m_fileName;
 }
 ////////////////////////////////////////////////////////////
+//brief get size of specific datas, as read from the file (or given to a neo writer)
+//return size in bytes of specific datas */
+unsigned int NindPadFile::getSpecificsSize() const
+{
+    return m_specificsSize;
+}
+////////////////////////////////////////////////////////////
 //brief write specifics header into write buffer */
 void NindPadFile::writeSpecificsHeader()
 {

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindBasics/NindPadFile.h

@@ -169,6 +169,10 @@ public:
     /**\brief get file name
     * \return file name */
     std::string getFileName();
+
+    /**\brief get size of specific datas, as read from the file (or given to a neo writer)
+    * \return size in bytes of specific datas */
+    unsigned int getSpecificsSize() const;
     NindFile m_file;                //pour l'ecrivain ou le lecteur
 
 protected:

{nind-2.2.1 → nind-2.3.0}/src/cpp/NindIndex/CMakeLists.txt

@@ -11,6 +11,8 @@ SET(NindIndex_LIB_SRCS
   NindTermIndex.cpp
   NindLocalIndex.h
   NindLocalIndex.cpp
+  NindIndexMerge.h
+  NindIndexMerge.cpp
 )
 
 add_library(NindIndex SHARED ${NindIndex_LIB_SRCS})
@@ -21,4 +23,4 @@ set_target_properties(NindIndex PROPERTIES VERSION ${NIND_LIB_VERSION} SOVERSION
 install(TARGETS NindIndex DESTINATION lib)
 
 ########### install files ###############
-install(FILES NindIndex.h NindLexiconIndex.h NindTermIndex.h NindLocalIndex.h DESTINATION include/NindIndex)
+install(FILES NindIndex.h NindLexiconIndex.h NindTermIndex.h NindLocalIndex.h NindIndexMerge.h DESTINATION include/NindIndex)