pyfock 0.2.2__tar.gz → 0.2.3__tar.gz

{pyfock-0.2.2/pyfock.egg-info → pyfock-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pyfock
-Version: 0.2.2
+Version: 0.2.3
 Summary: A simplistic and efficient pure-python quantum chemistry library from Phys Whiz.
 Home-page: https://github.com/manassharma07/pyfock
 Download-URL: 
@@ -204,6 +204,7 @@ Dynamic: summary
 - ✅ **ASE Calculator**: Optional ASE interface (geometry optimization and the wider ASE ecosystem), using analytical forces by default
 - ✅ **XC integration grids**: Treutler-Ahlrichs radial and Lebedev angular grids with region-wise angular pruning and Becke partitioning (levels 0-9), built by a parallel Numba kernel or on the GPU (~6x faster, same grid); energies agree with PySCF to the SCF convergence threshold in our benchmarks; numgrid grids remain available as an alternative scheme
 - ✅ **SANO initial guess**: superposition of atomic natural-orbital densities (ANO-RCC-MB natural orbitals projected onto the calculation basis, no atomic SCF needed) as the default SCF starting point; it roughly halves the number of SCF iterations relative to the core-Hamiltonian guess and prunes the XC grid accurately
+- ✅ **Extrapolated starting densities in geometry optimizations**: `density_guess='extrapolate'` in the ASE calculator carries the converged densities of the last steps along with the atoms and extrapolates them to the new geometry (adapting the density projection/extrapolation code contributed by Prof. Vincenzo Barone), for 16-18 % fewer SCF iterations along the same optimization path
 - ✅ **Cross-Platform**: Works on Linux, macOS, and Windows
 
 ## Installation
@@ -404,6 +405,11 @@ Available names are `skala-1.1` (recommended), `skala-1.1-rev1`, `skala-1.1-rev0
 See [`examples/ex44_Skala_neural_functional.py`](examples/ex44_Skala_neural_functional.py) for a runnable
 version, with and without the dispersion correction.
 
+Two Kaggle notebooks walk through the integration and can be run in the browser:
+[Skala in PyFock vs. PySCF](https://www.kaggle.com/code/ducktape07/skala-pyfock-vs-pyscf) runs Skala on the CPU, checks PyFock's results against PySCF,
+compares the timings (PyFock is considerably faster) and finishes with a geometry optimization;
+[PyFock + Skala on a Tesla T4](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4) shows how to run Skala on the GPU and benchmarks it.
+
 #### Getting the model
 
 `pip install "pyfock[skala]"` is all that is needed (see
@@ -451,8 +457,23 @@ Consequences worth knowing:
   grid-translation term); its features are integrals over each atomic grid, so a frozen grid would cost
   ~1e-2 Ha/Bohr rather than the ~1e-4 it costs a meta-GGA.
 - **Dispersion is off by default** — pass `dispersion=True` to include it, see below.
+- **Do not use Skala with ghost atoms on the default grid** (counterpoise corrections). Every atom that
+  owns grid points is part of Skala's functional — its non-local features are anchored at the atoms of
+  the grid — and PyFock gives ghost atoms grid points like any other atom. A ghost atom has no density of
+  its own, which is far outside what the model was trained on, and its potential can bind electrons on
+  the ghost basis functions: for CH4@(H2O)20 in def2-TZVPPD the SCF of the water cage with a ghost CH4
+  inside diverged, its first HOMO-LUMO gap 0.06 eV against 10.2 eV for the complex. Conventional
+  functionals are unaffected. What does work is evaluating each fragment on the grid of its real atoms
+  only, so that the ghost atoms contribute basis functions and nothing else — at grid level 4 or finer,
+  `grids=Grids(real_atoms_mol, level=4)`, since ghost functions in a cavity are then sampled only by the
+  outer shells of their neighbours' grids, and at level 3 that was too coarse for Skala in the (H2O)24
+  cage (a spurious bound ghost state again). Use the same level for the complex.
+- **Memory grows with the grid points per model call**: about 26 kB per point on the CPU, autograd graph
+  included, so the default 250 000 points per call take about 10 GB. Lower
+  `dftObj.skala_max_points_per_chunk` (also used by `DFT_Grad`) on a machine with little memory; only
+  the noise described next changes.
 - **The model carries about 1e-9 Ha of its own numerical noise.** Presenting the network with differently
-  shaped batches (a different `max_points_per_chunk`) shifts the energy at that level. Within one
+  shaped batches (a different `skala_max_points_per_chunk`) shifts the energy at that level. Within one
   calculation the chunking is fixed, so the shift is systematic rather than random and SCF convergence to
   `1e-8` is unaffected — but do not expect two runs with different chunk sizes to agree bit for bit.
 
@@ -466,7 +487,7 @@ conformer ranking.
 
 D3 depends only on the atomic numbers, the coordinates and those damping parameters — not on the
 density. It never enters the Kohn-Sham matrix and cannot change the SCF, so PyFock evaluates it once
-after convergence and adds it to the total energy:
+after convergence and adds it to the total energy — and `DFT_Grad` adds its gradient to the forces:
 
 ```bash
 pip install dftd3
@@ -476,24 +497,29 @@ pip install dftd3
 dftObj = DFT(mol, basis, auxbasis, xc='skala-1.1', dispersion=True)
 energy, dmat = dftObj.scf()      # dispersion-corrected total energy
 print(dftObj.Edisp)              # the correction on its own, in Hartree
+forces = DFT_Grad(dftObj).calculate()['forces']   # dispersion-corrected too
 ```
 
 `dispersion=True` uses whatever the functional declares. For any other functional, name the
 parametrisation yourself — `dispersion='pbe'`, `dispersion='b3lyp'` — and tune the damping through
 `dftObj.dispersion_version` (default `'d3bj'`) and `dftObj.dispersion_atm` (default `False`).
 [`pyfock.Dispersion`](pyfock/Dispersion.py) also exposes `d3_energy` and `d3_energy_and_gradient`
-directly, the latter giving `dE_disp/dR` for forces. See
+directly, the latter giving `dE_disp/dR` — which `DFT_Grad` forces already contain, so do not add it to
+them a second time. See
 [`examples/ex43_D3_dispersion_correction.py`](examples/ex43_D3_dispersion_correction.py).
 
-The ASE calculator uses the same backend by default:
+The ASE calculator uses the same backend by default, and like `DFT` it takes Skala's parametrisation
+from the checkpoint, so Skala needs no `dispersion_kwargs`; any other functional must name its own:
 
 ```python
+PyFockCalculator(functional='skala-1.1', dispersion=True)   # D3(BJ) with b3lyp5, as Skala declares
 PyFockCalculator(functional='PBE', dispersion=True, dispersion_kwargs={'xc': 'pbe'})
 ```
 
 `torch-dftd` (`pip install pyfock[dispersion-gpu]`) remains available there for GPU runs, where it
 evaluates the correction on the device. Select it with `backend='torch-dftd'`, or implicitly by asking
-for a non-CPU device:
+for a non-CPU device (with Skala, `xc` can be left out here too; it becomes torch-dftd's `'b3-lyp'` with
+BJ damping):
 
 ```python
 dispersion_kwargs={'xc': 'pbe', 'backend': 'torch-dftd', 'device': 'cuda'}
@@ -597,6 +623,9 @@ single points and for geometry optimizations. Forces run on the device too -- `D
 SCF -- so a `use_gpu=True` geometry optimization never returns to the host for a step; see
 [Forces on the GPU](#forces-on-the-gpu).
 
+The [PyFock + Skala on a Tesla T4](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4) Kaggle notebook shows the GPU setup end to end and
+benchmarks the PyFock + Skala integration on Kaggle's free T4.
+
 ### Initial Guess for the SCF
 
 The SCF starts by default from the **SANO** guess (superposition of atomic natural-orbital
@@ -716,6 +745,9 @@ forces = result["forces"]      # (natoms, 3) in Ha/Bohr
 gradient = result["gradient"]  # = -forces
 ```
 
+If the SCF applied a D3 correction (`dispersion=...`), the gradient includes its term, so it is always
+the gradient of the energy `scf()` returned, which is also `result["energy"]`.
+
 #### Forces on the GPU
 
 `DFT_Grad` follows the SCF: converge with `use_gpu=True` and the gradient runs on the device too, so
@@ -803,6 +835,17 @@ reads its thread count from the environment when numpy is first imported, so wit
 set `OMP_NUM_THREADS` at the top of your script rather than relying on `ncores` alone. See
 [`examples/ex45_ASE_geometry_optimization_with_Skala.py`](examples/ex45_ASE_geometry_optimization_with_Skala.py).
 
+#### Starting densities along an optimization path
+
+Every SCF of an optimization starts from the converged density of the previous step. With
+`density_guess='extrapolate'` the densities of the last five steps are instead carried along with
+the atoms — their occupied natural orbitals re-orthonormalized in the new overlap metric — and
+extrapolated to the new geometry with weights fitted to the geometries, then made N-representable.
+
+```python
+PyFockCalculator(functional="PBE", basis="def2-SVP", density_guess="extrapolate")   # default: 'previous'
+```
+
 #### Grid response
 
 By default the XC gradient treats the quadrature grid as fixed — its dependence on the nuclear
@@ -874,6 +917,8 @@ streamlit run app.py
 - [x] [Kaggle Notebook: Benchmarking PyFock against PySCF](https://www.kaggle.com/code/ducktape07/pyfock-vs-pyscf#10.-Timing-summary-table)
 - [x] [Kaggle Notebook: PyFock GPU Dynamic Precision](https://www.kaggle.com/code/ducktape07/pyfock-gpu-dynamic-precision)
 - [x] [Kaggle Notebook: PySCF vs. PyFock GPU Benchmark](https://www.kaggle.com/code/ducktape07/pyscf-vs-pyfock-gpu-benchmark)
+- [x] [Kaggle Notebook: Skala in PyFock vs. PySCF (CPU): accuracy, speed and geometry optimization](https://www.kaggle.com/code/ducktape07/skala-pyfock-vs-pyscf)
+- [x] [Kaggle Notebook: PyFock + Skala on the GPU (Tesla T4): setup and benchmark](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4)
 
 
 ## Documentation

{pyfock-0.2.2 → pyfock-0.2.3}/README.md

@@ -141,6 +141,7 @@
 - ✅ **ASE Calculator**: Optional ASE interface (geometry optimization and the wider ASE ecosystem), using analytical forces by default
 - ✅ **XC integration grids**: Treutler-Ahlrichs radial and Lebedev angular grids with region-wise angular pruning and Becke partitioning (levels 0-9), built by a parallel Numba kernel or on the GPU (~6x faster, same grid); energies agree with PySCF to the SCF convergence threshold in our benchmarks; numgrid grids remain available as an alternative scheme
 - ✅ **SANO initial guess**: superposition of atomic natural-orbital densities (ANO-RCC-MB natural orbitals projected onto the calculation basis, no atomic SCF needed) as the default SCF starting point; it roughly halves the number of SCF iterations relative to the core-Hamiltonian guess and prunes the XC grid accurately
+- ✅ **Extrapolated starting densities in geometry optimizations**: `density_guess='extrapolate'` in the ASE calculator carries the converged densities of the last steps along with the atoms and extrapolates them to the new geometry (adapting the density projection/extrapolation code contributed by Prof. Vincenzo Barone), for 16-18 % fewer SCF iterations along the same optimization path
 - ✅ **Cross-Platform**: Works on Linux, macOS, and Windows
 
 ## Installation
@@ -341,6 +342,11 @@ Available names are `skala-1.1` (recommended), `skala-1.1-rev1`, `skala-1.1-rev0
 See [`examples/ex44_Skala_neural_functional.py`](examples/ex44_Skala_neural_functional.py) for a runnable
 version, with and without the dispersion correction.
 
+Two Kaggle notebooks walk through the integration and can be run in the browser:
+[Skala in PyFock vs. PySCF](https://www.kaggle.com/code/ducktape07/skala-pyfock-vs-pyscf) runs Skala on the CPU, checks PyFock's results against PySCF,
+compares the timings (PyFock is considerably faster) and finishes with a geometry optimization;
+[PyFock + Skala on a Tesla T4](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4) shows how to run Skala on the GPU and benchmarks it.
+
 #### Getting the model
 
 `pip install "pyfock[skala]"` is all that is needed (see
@@ -388,8 +394,23 @@ Consequences worth knowing:
   grid-translation term); its features are integrals over each atomic grid, so a frozen grid would cost
   ~1e-2 Ha/Bohr rather than the ~1e-4 it costs a meta-GGA.
 - **Dispersion is off by default** — pass `dispersion=True` to include it, see below.
+- **Do not use Skala with ghost atoms on the default grid** (counterpoise corrections). Every atom that
+  owns grid points is part of Skala's functional — its non-local features are anchored at the atoms of
+  the grid — and PyFock gives ghost atoms grid points like any other atom. A ghost atom has no density of
+  its own, which is far outside what the model was trained on, and its potential can bind electrons on
+  the ghost basis functions: for CH4@(H2O)20 in def2-TZVPPD the SCF of the water cage with a ghost CH4
+  inside diverged, its first HOMO-LUMO gap 0.06 eV against 10.2 eV for the complex. Conventional
+  functionals are unaffected. What does work is evaluating each fragment on the grid of its real atoms
+  only, so that the ghost atoms contribute basis functions and nothing else — at grid level 4 or finer,
+  `grids=Grids(real_atoms_mol, level=4)`, since ghost functions in a cavity are then sampled only by the
+  outer shells of their neighbours' grids, and at level 3 that was too coarse for Skala in the (H2O)24
+  cage (a spurious bound ghost state again). Use the same level for the complex.
+- **Memory grows with the grid points per model call**: about 26 kB per point on the CPU, autograd graph
+  included, so the default 250 000 points per call take about 10 GB. Lower
+  `dftObj.skala_max_points_per_chunk` (also used by `DFT_Grad`) on a machine with little memory; only
+  the noise described next changes.
 - **The model carries about 1e-9 Ha of its own numerical noise.** Presenting the network with differently
-  shaped batches (a different `max_points_per_chunk`) shifts the energy at that level. Within one
+  shaped batches (a different `skala_max_points_per_chunk`) shifts the energy at that level. Within one
   calculation the chunking is fixed, so the shift is systematic rather than random and SCF convergence to
   `1e-8` is unaffected — but do not expect two runs with different chunk sizes to agree bit for bit.
 
@@ -403,7 +424,7 @@ conformer ranking.
 
 D3 depends only on the atomic numbers, the coordinates and those damping parameters — not on the
 density. It never enters the Kohn-Sham matrix and cannot change the SCF, so PyFock evaluates it once
-after convergence and adds it to the total energy:
+after convergence and adds it to the total energy — and `DFT_Grad` adds its gradient to the forces:
 
 ```bash
 pip install dftd3
@@ -413,24 +434,29 @@ pip install dftd3
 dftObj = DFT(mol, basis, auxbasis, xc='skala-1.1', dispersion=True)
 energy, dmat = dftObj.scf()      # dispersion-corrected total energy
 print(dftObj.Edisp)              # the correction on its own, in Hartree
+forces = DFT_Grad(dftObj).calculate()['forces']   # dispersion-corrected too
 ```
 
 `dispersion=True` uses whatever the functional declares. For any other functional, name the
 parametrisation yourself — `dispersion='pbe'`, `dispersion='b3lyp'` — and tune the damping through
 `dftObj.dispersion_version` (default `'d3bj'`) and `dftObj.dispersion_atm` (default `False`).
 [`pyfock.Dispersion`](pyfock/Dispersion.py) also exposes `d3_energy` and `d3_energy_and_gradient`
-directly, the latter giving `dE_disp/dR` for forces. See
+directly, the latter giving `dE_disp/dR` — which `DFT_Grad` forces already contain, so do not add it to
+them a second time. See
 [`examples/ex43_D3_dispersion_correction.py`](examples/ex43_D3_dispersion_correction.py).
 
-The ASE calculator uses the same backend by default:
+The ASE calculator uses the same backend by default, and like `DFT` it takes Skala's parametrisation
+from the checkpoint, so Skala needs no `dispersion_kwargs`; any other functional must name its own:
 
 ```python
+PyFockCalculator(functional='skala-1.1', dispersion=True)   # D3(BJ) with b3lyp5, as Skala declares
 PyFockCalculator(functional='PBE', dispersion=True, dispersion_kwargs={'xc': 'pbe'})
 ```
 
 `torch-dftd` (`pip install pyfock[dispersion-gpu]`) remains available there for GPU runs, where it
 evaluates the correction on the device. Select it with `backend='torch-dftd'`, or implicitly by asking
-for a non-CPU device:
+for a non-CPU device (with Skala, `xc` can be left out here too; it becomes torch-dftd's `'b3-lyp'` with
+BJ damping):
 
 ```python
 dispersion_kwargs={'xc': 'pbe', 'backend': 'torch-dftd', 'device': 'cuda'}
@@ -534,6 +560,9 @@ single points and for geometry optimizations. Forces run on the device too -- `D
 SCF -- so a `use_gpu=True` geometry optimization never returns to the host for a step; see
 [Forces on the GPU](#forces-on-the-gpu).
 
+The [PyFock + Skala on a Tesla T4](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4) Kaggle notebook shows the GPU setup end to end and
+benchmarks the PyFock + Skala integration on Kaggle's free T4.
+
 ### Initial Guess for the SCF
 
 The SCF starts by default from the **SANO** guess (superposition of atomic natural-orbital
@@ -653,6 +682,9 @@ forces = result["forces"]      # (natoms, 3) in Ha/Bohr
 gradient = result["gradient"]  # = -forces
 ```
 
+If the SCF applied a D3 correction (`dispersion=...`), the gradient includes its term, so it is always
+the gradient of the energy `scf()` returned, which is also `result["energy"]`.
+
 #### Forces on the GPU
 
 `DFT_Grad` follows the SCF: converge with `use_gpu=True` and the gradient runs on the device too, so
@@ -740,6 +772,17 @@ reads its thread count from the environment when numpy is first imported, so wit
 set `OMP_NUM_THREADS` at the top of your script rather than relying on `ncores` alone. See
 [`examples/ex45_ASE_geometry_optimization_with_Skala.py`](examples/ex45_ASE_geometry_optimization_with_Skala.py).
 
+#### Starting densities along an optimization path
+
+Every SCF of an optimization starts from the converged density of the previous step. With
+`density_guess='extrapolate'` the densities of the last five steps are instead carried along with
+the atoms — their occupied natural orbitals re-orthonormalized in the new overlap metric — and
+extrapolated to the new geometry with weights fitted to the geometries, then made N-representable.
+
+```python
+PyFockCalculator(functional="PBE", basis="def2-SVP", density_guess="extrapolate")   # default: 'previous'
+```
+
 #### Grid response
 
 By default the XC gradient treats the quadrature grid as fixed — its dependence on the nuclear
@@ -811,6 +854,8 @@ streamlit run app.py
 - [x] [Kaggle Notebook: Benchmarking PyFock against PySCF](https://www.kaggle.com/code/ducktape07/pyfock-vs-pyscf#10.-Timing-summary-table)
 - [x] [Kaggle Notebook: PyFock GPU Dynamic Precision](https://www.kaggle.com/code/ducktape07/pyfock-gpu-dynamic-precision)
 - [x] [Kaggle Notebook: PySCF vs. PyFock GPU Benchmark](https://www.kaggle.com/code/ducktape07/pyscf-vs-pyfock-gpu-benchmark)
+- [x] [Kaggle Notebook: Skala in PyFock vs. PySCF (CPU): accuracy, speed and geometry optimization](https://www.kaggle.com/code/ducktape07/skala-pyfock-vs-pyscf)
+- [x] [Kaggle Notebook: PyFock + Skala on the GPU (Tesla T4): setup and benchmark](https://www.kaggle.com/code/ducktape07/pyfock-skala-on-gpu-tesla-t4)
 
 
 ## Documentation

{pyfock-0.2.2 → pyfock-0.2.3}/pyfock/DFT.py

@@ -395,7 +395,8 @@ class DFT:
         self.mo_energies = None
         """ Molecular orbital energies. Will be computed during SCF. """
         self.Total_energy = None
-        """ Total energy of the system. Will be computed during SCF. """
+        """ Total energy of the system. Will be computed during SCF. Once the SCF has finished it is the
+        energy :meth:`scf` returns, D3 correction (:attr:`Edisp`) included when one is applied. """
         self.J_energy = None
         """ Coulomb energy contribution. Will be computed during SCF. """
         self.XC_energy = None
@@ -424,9 +425,18 @@ class DFT:
         device gives most of the speedup without needing PyFock's full CuPy path. Requires a CUDA build
         of PyTorch; ignored for conventional functionals."""
 
+        self.skala_max_points_per_chunk = 250000
+        """Upper bound on the grid points the Skala model evaluates in one call, in the SCF and in
+        `DFT_Grad`. Peak memory grows linearly with it -- about 26 kB per point on the CPU, autograd graph
+        included (skala-1.1: 9.9 GB of process commit for 250 000 points, 3.2 GB for 50 000) -- while the
+        result does not change: a chunk holds whole atomic grids, so only the float32 summation order
+        inside the model differs (~1e-9 Ha). Lower it when the model runs out of memory, e.g. next to a
+        large `max_memory_ints3c2e` on a machine with little RAM; ignored for conventional functionals."""
+
         self.dispersion = dispersion
         """DFT-D3 dispersion correction (:mod:`pyfock.Dispersion`), added to the total energy after the
-        SCF has converged. It is a purely geometric, additive term and never enters the Kohn-Sham matrix.
+        SCF has converged, and its gradient to the forces of `DFT_Grad`. It is a purely geometric, additive
+        term and never enters the Kohn-Sham matrix.
         ``None``/``False`` (default) leaves it out; ``True`` uses the parametrisation the functional
         itself declares, which currently means Skala (``'b3lyp5'``); a string names the functional whose
         D3 parameters to use, e.g. ``'pbe'`` or ``'b3lyp'``. Needs ``pip install dftd3``."""
@@ -442,6 +452,11 @@ class DFT:
         self.Edisp = 0.0
         """Dispersion energy of the last :meth:`scf` call in Hartree, 0 when no correction was applied."""
 
+        self.dispersion_method = None
+        """D3 parametrisation the last :meth:`scf` call applied -- the functional whose damping parameters
+        were used, e.g. ``'b3lyp5'`` for Skala 1.1 -- or None when no correction was applied. Set by the
+        SCF from :attr:`dispersion`; `DFT_Grad` reads it to add the matching gradient."""
+
         self.direct_scf = False 
         """ Only relevant for calculations without DF. If True, the 4c2e integrals are recalculated at every SCF iteration.
         Extremely memory efficient. The 4c2e tensor is never stored in memory and is contracted with the density matrix
@@ -2047,6 +2062,7 @@ class DFT:
                             list_nonzero_indices=list_nonzero_indices,
                             count_nonzero_indices=count_nonzero_indices,
                             list_ao_values=list_ao_values, list_ao_grad_values=list_ao_grad_values,
+                            max_points_per_chunk=self.skala_max_points_per_chunk,
                             debug=debug, threads_per_block=threads_per_block)
                         Vxc = cp.asarray(Vxc, dtype=cp.float64)
                     else:
@@ -2056,6 +2072,7 @@ class DFT:
                                                            count_nonzero_indices=count_nonzero_indices,
                                                            list_ao_values=list_ao_values,
                                                            list_ao_grad_values=list_ao_grad_values,
+                                                           max_points_per_chunk=self.skala_max_points_per_chunk,
                                                            debug=debug)
                 elif not self.use_gpu:
                     if XC_algo==1:
@@ -2305,16 +2322,20 @@ class DFT:
 
         
         self.converged = scf_converged
-        self.niter = itr-1
+        # A converged SCF leaves the loop through `break`, before `itr` is incremented.
+        self.niter = itr if scf_converged else itr-1
 
         # D3 dispersion is a function of the geometry alone, so it is evaluated once here rather than
         # inside the SCF loop; it cannot affect the converged density (see pyfock.Dispersion).
         self.Edisp = 0.0
+        self.dispersion_method = dispersion_method
         if dispersion_method is not None:
             from . import Dispersion
             self.Edisp = Dispersion.d3_energy(mol, dispersion_method, version=self.dispersion_version,
                                               atm=self.dispersion_atm)
             Etot = Etot + self.Edisp
+            # DFT_Grad and DFT_NumGrad read the energy here, and must see the one returned below
+            self.Total_energy = Etot
             print('\n\n------------------------------------------------------', flush=True)
             print('DFT-D3 dispersion correction', flush=True)
             print('------------------------------------------------------', flush=True)

{pyfock-0.2.2 → pyfock-0.2.3}/pyfock/DFT_Grad.py

@@ -11,6 +11,7 @@ from threadpoolctl import threadpool_limits
 import pyfock.Integrals as Integrals
 from pyfock import XC
 from pyfock import Data
+from pyfock import Dispersion
 from pyfock.Basis import Basis
 from pyfock.Mol import Mol
 
@@ -60,11 +61,18 @@ class DFT_Grad:
               - 0.5 sum_PQ c_P c_Q d(P|Q)/dR          (DF Coulomb)
               + sum_ij D_ij dV_ecp_ij/dR              (ECP, if present)
               + dExc/dR                               (XC, fixed grid)
+              + dE_disp/dR                            (DFT-D3, if the SCF applied it)
 
     where W is the energy-weighted density matrix and c_P are the density
     fitting coefficients of the converged density. The grid-weight response
     of the XC term is neglected (same approximation as PySCF's default).
 
+    The D3 term is present whenever the SCF added a dispersion correction
+    (``DFT(..., dispersion=...)``, e.g. ``dispersion=True`` for Skala), with
+    the parametrisation it used, so the gradient is that of the corrected
+    energy ``scf()`` returns. D3 depends on the nuclear positions alone, so
+    the term is exact; simple-dftd3 evaluates it on the CPU in either mode.
+
     When effective core potentials (ECPs) are present, ``mol.Zcharges`` already
     holds the reduced (Z - n_core) charges, so the nuclear-repulsion and
     nuclear-attraction gradient terms above are automatically consistent. The
@@ -161,6 +169,8 @@ class DFT_Grad:
         # functional: the converged DFT object already holds the loaded model, and the gradient goes
         # through its own driver.
         self.skala = getattr(dft_obj, 'skala', None)
+        # the model's peak memory scales with the points per call; use the SCF's setting
+        self.skala_max_points_per_chunk = int(getattr(dft_obj, 'skala_max_points_per_chunk', 250000))
         if self.grid_response is None:
             self.grid_response = self.skala is not None
         if self.skala is not None and not self.grid_response:
@@ -264,8 +274,10 @@ class DFT_Grad:
         Returns
         -------
         dict
-            Dictionary with `energy`, `gradient` (natoms, 3) in Ha/Bohr,
-            `forces` (= -gradient) and per-term `timings`.
+            Dictionary with `energy` (the energy ``scf()`` returned, D3
+            included when applied), `gradient` (natoms, 3) in Ha/Bohr,
+            `forces` (= -gradient), the per-term `gradient_components` and
+            per-term `timings`.
         """
         if not self.use_gpu:
             self._cp_stream = None
@@ -454,7 +466,8 @@ class DFT_Grad:
             dexc_dbf, explicit_xc_grad = skala_grad(
                 basis, dmat, self.grids, self.skala, ncores=ncores, blocksize=blocksize,
                 list_nonzero_indices=list_nonzero_indices,
-                count_nonzero_indices=count_nonzero_indices, **skala_kwargs,
+                count_nonzero_indices=count_nonzero_indices,
+                max_points_per_chunk=self.skala_max_points_per_chunk, **skala_kwargs,
             )
         elif use_gpu:
             xc_result = Integrals.eval_xc_grad_2_cupy(
@@ -496,7 +509,19 @@ class DFT_Grad:
                 grad_ecp = self._ecp_grad(dmat)
             timings['ecp'] = timer() - start
 
-        gradient = grad_nn + grad_T + grad_V + grad_S + grad_J + grad_xc + grad_ecp
+        # ---------------- DFT-D3 dispersion (if the SCF applied it) ----------------
+        # scf() adds E_disp to the energy it returns, so the gradient must carry dE_disp/dR as well.
+        # Its net force vanishes on its own, so translational invariance would not notice it missing.
+        grad_disp = np.zeros((natoms, 3))
+        dispersion_method = getattr(dft_obj, 'dispersion_method', None)
+        if dispersion_method is not None:
+            start = timer()
+            _, grad_disp = Dispersion.d3_energy_and_gradient(
+                mol, dispersion_method, version=dft_obj.dispersion_version,
+                atm=dft_obj.dispersion_atm)
+            timings['dispersion'] = timer() - start
+
+        gradient = grad_nn + grad_T + grad_V + grad_S + grad_J + grad_xc + grad_ecp + grad_disp
         forces = -gradient
 
         if self.verbose:
@@ -521,6 +546,7 @@ class DFT_Grad:
                 'coulomb_df': grad_J,
                 'xc': grad_xc,
                 'ecp': grad_ecp,
+                'dispersion': grad_disp,
             },
             'timings': timings,
         }

{pyfock-0.2.2 → pyfock-0.2.3}/pyfock/Dispersion.py

@@ -5,20 +5,26 @@ coordinates and a set of functional-specific damping parameters, and on nothing
 the electron density, does not enter the Kohn-Sham matrix and does not change the SCF in any way. That is
 why PyFock evaluates it once, after the SCF has converged, and simply adds it to the total energy.
 
+Only real atoms take part, each with the atomic number of its element: ghost atoms (counterpoise
+calculations) are left out, and an atom with an ECP enters as its element rather than with the effective
+charge ``mol.Zcharges`` holds once an ECP basis is loaded. A counterpoise fragment with ghost partners
+therefore gets exactly the dispersion energy of its real atoms.
+
 The numbers come from `simple-dftd3 <https://github.com/dftd3/simple-dftd3>`_ (``pip install dftd3``),
 through its ``dftd3.interface`` module. Note that the ``dftd3.pyscf`` submodule of the same package needs
 PySCF, but ``dftd3.interface`` does not, so PyFock stays free of that dependency.
 
-This module is the core-level dispersion API, used by ``DFT(..., dispersion=...)``. PyFock's ASE
-calculator has a **separate** dispersion path of its own, reached through
-``PyFockCalculator(dispersion=True, dispersion_kwargs=...)``, which is built on ``torch-dftd`` and works
-in ASE's eV/Angstrom units. The two are independent; this one is the reference implementation from the
-Grimme group and is what reproduces the published Skala energies.
+This module is the core-level dispersion API, used by ``DFT(..., dispersion=...)`` for the energy and
+by ``DFT_Grad`` for its gradient. PyFock's ASE calculator applies the correction itself instead, after
+an SCF that has none, through ``PyFockCalculator(dispersion=True, dispersion_kwargs=...)`` in ASE's
+eV/Angstrom units: with this module by default, or with ``torch-dftd`` on a GPU. simple-dftd3 is the
+Grimme group's reference implementation and is what reproduces the published Skala energies.
 
 The main consumer is the Skala neural functional, which is parametrised together with D3(BJ) using the
 B3LYP5 damping parameters and no three-body term -- exactly the defaults of
 ``dftd3.interface.RationalDampingParam``. :func:`pyfock.XC.SkalaFunctional.d3_settings` reports the
-parametrisation the checkpoint declares, and ``DFT(..., dispersion=True)`` picks it up automatically.
+parametrisation the checkpoint declares, and ``DFT(..., dispersion=True)`` and
+``PyFockCalculator(dispersion=True)`` pick it up automatically.
 
 References
 ----------
@@ -72,19 +78,39 @@ def _damping_param(method, version, atm, param):
     return cls(**param)
 
 
-def _model(mol):
-    """A ``DispersionModel`` for a PyFock :class:`~pyfock.Mol.Mol`, or for a bare geometry.
+def _geometry(mol):
+    """The atoms D3 sees: ``(atomic_numbers, positions_in_bohr, their indices in mol, natoms of mol)``.
+
+    ``mol`` is a PyFock :class:`~pyfock.Mol.Mol` or a bare ``(atomic_numbers, positions_in_bohr)`` pair,
+    which is how callers that do not have a ``Mol`` -- the ASE calculator, for instance -- reach the same
+    code. Two kinds of atom need care:
 
-    ``mol`` may also be a ``(atomic_numbers, positions_in_bohr)`` pair, which is how callers that do
-    not have a ``Mol`` -- the ASE calculator, for instance -- reach the same code.
+    * ghost atoms carry basis functions but no nucleus, so they have no dispersion. They are left out
+      rather than handed to the library with Z = 0, where they would still count towards the coordination
+      numbers of nearby atoms -- shifting those atoms' C6 coefficients -- and receive gradients of their
+      own. In a bare pair, atomic number 0 marks a ghost;
+    * an atom with an ECP enters with the atomic number of its element (``Mol.element_numbers``). Once an
+      ECP basis is loaded ``mol.Zcharges`` holds the effective charge -- iodine with a 28-electron ECP
+      reads 25, manganese -- and D3's parameters belong to the element.
     """
-    from dftd3.interface import DispersionModel
     if hasattr(mol, 'Zcharges'):
-        numbers, positions = mol.Zcharges, mol.coordsBohrs
+        numbers = mol.element_numbers() if hasattr(mol, 'element_numbers') else mol.Zcharges
+        ghost = mol.ghost_mask() if hasattr(mol, 'ghost_mask') else None
+        positions = mol.coordsBohrs
     else:
         numbers, positions = mol
-    numbers = np.asarray(numbers, dtype=np.int64)
+        ghost = None
+    numbers = np.asarray(numbers, dtype=np.int64).reshape(-1)
     positions = np.ascontiguousarray(np.asarray(positions, dtype=np.float64).reshape(-1, 3))
+    if ghost is None:
+        ghost = numbers == 0
+    keep = np.nonzero(~np.asarray(ghost, dtype=bool))[0]
+    return numbers[keep], np.ascontiguousarray(positions[keep]), keep, numbers.shape[0]
+
+
+def _model(numbers, positions):
+    """A ``DispersionModel`` for the atoms :func:`_geometry` selected."""
+    from dftd3.interface import DispersionModel
     return DispersionModel(numbers, positions)
 
 
@@ -93,8 +119,10 @@ def d3_energy(mol, method, version='d3bj', atm=False, param=None):
 
     Parameters
     ----------
-    mol : Mol
-        Molecule; only ``mol.Zcharges`` and ``mol.coordsBohrs`` are used.
+    mol : Mol or tuple
+        Molecule, or an ``(atomic_numbers, positions_in_bohr)`` pair. Ghost atoms are left out and atoms
+        with an ECP enter as their element (see :func:`_geometry`), so the energy of a counterpoise
+        fragment with ghost partners is exactly that of the fragment's real atoms.
     method : str
         Functional name whose D3 parameters to use, e.g. ``'b3lyp5'`` (what Skala 1.1 expects),
         ``'pbe'``, ``'b3lyp'``. Ignored when ``param`` is given.
@@ -113,7 +141,11 @@ def d3_energy(mol, method, version='d3bj', atm=False, param=None):
     float
         Dispersion energy in Hartree (negative).
     """
-    result = _model(mol).get_dispersion(_damping_param(method, version, atm, param), grad=False)
+    damping = _damping_param(method, version, atm, param)
+    numbers, positions, _, _ = _geometry(mol)
+    if numbers.shape[0] == 0:
+        return 0.0      # nothing but ghost atoms
+    result = _model(numbers, positions).get_dispersion(damping, grad=False)
     return float(np.asarray(result['energy']).item())
 
 
@@ -125,11 +157,17 @@ def d3_energy_and_gradient(mol, method, version='d3bj', atm=False, param=None):
     energy : float
         Dispersion energy in Hartree.
     gradient : (natm, 3) ndarray
-        dE_disp/dR in Hartree/Bohr, ready to be added to the SCF forces.
+        dE_disp/dR in Hartree/Bohr (the forces are minus this). One row per atom of ``mol``; the rows
+        of ghost atoms are zero. ``DFT_Grad`` adds it itself after an SCF with ``dispersion``.
     """
-    result = _model(mol).get_dispersion(_damping_param(method, version, atm, param), grad=True)
-    return (float(np.asarray(result['energy']).item()),
-            np.ascontiguousarray(np.asarray(result['gradient'], dtype=np.float64)))
+    damping = _damping_param(method, version, atm, param)
+    numbers, positions, keep, natm = _geometry(mol)
+    gradient = np.zeros((natm, 3), dtype=np.float64)
+    if numbers.shape[0] == 0:
+        return 0.0, gradient      # nothing but ghost atoms
+    result = _model(numbers, positions).get_dispersion(damping, grad=True)
+    gradient[keep] = np.asarray(result['gradient'], dtype=np.float64).reshape(-1, 3)
+    return float(np.asarray(result['energy']).item()), gradient
 
 
 def citation():