nbdev 3.3.27__tar.gz → 3.3.29__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. {nbdev-3.3.27/nbdev.egg-info → nbdev-3.3.29}/PKG-INFO +4 -4
  2. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/__init__.py +1 -1
  3. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/_modidx.py +1 -8
  4. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/clean.py +2 -1
  5. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/config.py +2 -3
  6. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/diff.py +40 -98
  7. nbdev-3.3.29/nbdev/diff_core.py +148 -0
  8. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/doclinks.py +12 -8
  9. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/processors.py +14 -5
  10. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/serve.py +4 -6
  11. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/serve_drv.py +11 -1
  12. nbdev-3.3.29/nbdev/skill.py +60 -0
  13. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/test.py +2 -4
  14. {nbdev-3.3.27 → nbdev-3.3.29/nbdev.egg-info}/PKG-INFO +4 -4
  15. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev.egg-info/SOURCES.txt +2 -0
  16. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev.egg-info/requires.txt +2 -4
  17. {nbdev-3.3.27 → nbdev-3.3.29}/pyproject.toml +4 -4
  18. nbdev-3.3.29/tests/test_diff.py +93 -0
  19. nbdev-3.3.27/nbdev/skill.py +0 -63
  20. {nbdev-3.3.27 → nbdev-3.3.29}/CONTRIBUTING.md +0 -0
  21. {nbdev-3.3.27 → nbdev-3.3.29}/LICENSE +0 -0
  22. {nbdev-3.3.27 → nbdev-3.3.29}/README.md +0 -0
  23. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/cli.py +0 -0
  24. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/export.py +0 -0
  25. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/extract_attachments.py +0 -0
  26. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/frontmatter.py +0 -0
  27. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/imports.py +0 -0
  28. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/maker.py +0 -0
  29. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/merge.py +0 -0
  30. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/migrate.py +0 -0
  31. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/moddocs.py +0 -0
  32. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/process.py +0 -0
  33. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/qmd.py +0 -0
  34. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/quarto.py +0 -0
  35. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/quarto_render.py +0 -0
  36. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/readme_filter.py +0 -0
  37. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/release.py +0 -0
  38. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/scrubmagics.py +0 -0
  39. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/showdoc.py +0 -0
  40. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev/sync.py +0 -0
  41. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev.egg-info/dependency_links.txt +0 -0
  42. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev.egg-info/entry_points.txt +0 -0
  43. {nbdev-3.3.27 → nbdev-3.3.29}/nbdev.egg-info/top_level.txt +0 -0
  44. {nbdev-3.3.27 → nbdev-3.3.29}/setup.cfg +0 -0
  45. {nbdev-3.3.27 → nbdev-3.3.29}/tests/test_quarto_render.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: nbdev
3
- Version: 3.3.27
3
+ Version: 3.3.29
4
4
  Summary: Create delightful software with Jupyter Notebooks
5
5
  Author-email: "Jeremy Howard and the fast.ai community" <j@fast.ai>
6
6
  License-Expression: Apache-2.0
@@ -12,10 +12,10 @@ Classifier: Intended Audience :: Developers
12
12
  Classifier: Natural Language :: English
13
13
  Classifier: Programming Language :: Python :: 3
14
14
  Classifier: Programming Language :: Python :: 3 :: Only
15
- Requires-Python: >=3.10
15
+ Requires-Python: >=3.11
16
16
  Description-Content-Type: text/markdown
17
17
  License-File: LICENSE
18
- Requires-Dist: fastcore>=2.2.28
18
+ Requires-Dist: fastcore>=2.2.39
19
19
  Requires-Dist: execnb>=0.3.3
20
20
  Requires-Dist: astunparse
21
21
  Requires-Dist: ghapi>=2.0.2
@@ -25,7 +25,6 @@ Requires-Dist: setuptools
25
25
  Requires-Dist: build
26
26
  Requires-Dist: fastgit>=0.0.7
27
27
  Requires-Dist: pyyaml
28
- Requires-Dist: tomli; python_version < "3.11"
29
28
  Provides-Extra: dev
30
29
  Requires-Dist: ipywidgets; extra == "dev"
31
30
  Requires-Dist: nbdev-numpy; extra == "dev"
@@ -40,6 +39,7 @@ Requires-Dist: sphinx; extra == "dev"
40
39
  Requires-Dist: plum-dispatch; extra == "dev"
41
40
  Requires-Dist: nbclient; extra == "dev"
42
41
  Requires-Dist: ipykernel; extra == "dev"
42
+ Requires-Dist: pytest; extra == "dev"
43
43
  Dynamic: license-file
44
44
 
45
45
  # Getting Started
@@ -6,7 +6,7 @@ Modules:
6
6
  - `nbdev.moddocs`: # Module docs: creating them from existing notebooks
7
7
  - `nbdev.skill`: Write, read and review nbdev notebooks as source, documentation, examples and tests. You MUST read this before any work on a project with notebooks."""
8
8
 
9
- __version__ = "3.3.27"
9
+ __version__ = "3.3.29"
10
10
 
11
11
  from .doclinks import nbdev_export
12
12
  from .showdoc import show_doc
@@ -81,18 +81,12 @@ d = { 'settings': { 'branch': 'main',
81
81
  'nbdev.config.write_cells': ('api/config.html#write_cells', 'nbdev/config.py')},
82
82
  'nbdev.diff': { 'nbdev.diff._cell_changes': ('api/diff.html#_cell_changes', 'nbdev/diff.py'),
83
83
  'nbdev.diff._diff_order': ('api/diff.html#_diff_order', 'nbdev/diff.py'),
84
- 'nbdev.diff._dline': ('api/diff.html#_dline', 'nbdev/diff.py'),
85
84
  'nbdev.diff._dtag': ('api/diff.html#_dtag', 'nbdev/diff.py'),
86
85
  'nbdev.diff._file_cells': ('api/diff.html#_file_cells', 'nbdev/diff.py'),
87
86
  'nbdev.diff._nb_srcdict': ('api/diff.html#_nb_srcdict', 'nbdev/diff.py'),
88
87
  'nbdev.diff._out_lines': ('api/diff.html#_out_lines', 'nbdev/diff.py'),
89
- 'nbdev.diff._refine': ('api/diff.html#_refine', 'nbdev/diff.py'),
90
88
  'nbdev.diff._src': ('api/diff.html#_src', 'nbdev/diff.py'),
91
89
  'nbdev.diff._srcdict': ('api/diff.html#_srcdict', 'nbdev/diff.py'),
92
- 'nbdev.diff._tokens': ('api/diff.html#_tokens', 'nbdev/diff.py'),
93
- 'nbdev.diff._trunc': ('api/diff.html#_trunc', 'nbdev/diff.py'),
94
- 'nbdev.diff._wline': ('api/diff.html#_wline', 'nbdev/diff.py'),
95
- 'nbdev.diff._word_diff': ('api/diff.html#_word_diff', 'nbdev/diff.py'),
96
90
  'nbdev.diff.cell_diffs': ('api/diff.html#cell_diffs', 'nbdev/diff.py'),
97
91
  'nbdev.diff.changed_cells': ('api/diff.html#changed_cells', 'nbdev/diff.py'),
98
92
  'nbdev.diff.nb_diff': ('api/diff.html#nb_diff', 'nbdev/diff.py'),
@@ -100,8 +94,7 @@ d = { 'settings': { 'branch': 'main',
100
94
  'nbdev.diff.nbdev_diff_driver': ('api/diff.html#nbdev_diff_driver', 'nbdev/diff.py'),
101
95
  'nbdev.diff.nbs_pair': ('api/diff.html#nbs_pair', 'nbdev/diff.py'),
102
96
  'nbdev.diff.read_nb_from_git': ('api/diff.html#read_nb_from_git', 'nbdev/diff.py'),
103
- 'nbdev.diff.render_diff': ('api/diff.html#render_diff', 'nbdev/diff.py'),
104
- 'nbdev.diff.source_diff': ('api/diff.html#source_diff', 'nbdev/diff.py')},
97
+ 'nbdev.diff.render_diff': ('api/diff.html#render_diff', 'nbdev/diff.py')},
105
98
  'nbdev.doclinks': { 'nbdev.doclinks.NbdevLookup': ('api/doclinks.html#nbdevlookup', 'nbdev/doclinks.py'),
106
99
  'nbdev.doclinks.NbdevLookup.__getitem__': ( 'api/doclinks.html#nbdevlookup.__getitem__',
107
100
  'nbdev/doclinks.py'),
@@ -62,12 +62,13 @@ def _clean_cell_output_id(lines):
62
62
 
63
63
  # %% ../nbs/api/11_clean.ipynb #b4cde615
64
64
  def _clean_cell_output(cell, clean_ids, allowed_out_meta_keys):
65
- "Remove `cell` output execution count and optionally ids from text reprs"
65
+ "Remove `cell` output execution count, optionally ids from text reprs, and representations listed under `nbdev.strip`"
66
66
  outputs = cell.get('outputs', [])
67
67
  for o in outputs:
68
68
  if 'execution_count' in o: o['execution_count'] = None
69
69
  data = o.get('data', {})
70
70
  data.pop("application/vnd.google.colaboratory.intrinsic+json", None)
71
+ for k in nested_idx(o, 'metadata', 'nbdev', 'strip') or []: data.pop(k, None)
71
72
  for k in data:
72
73
  if k.startswith('text') and clean_ids: data[k] = _clean_cell_output_id(data[k])
73
74
  if k.startswith('image') and "svg" not in k: data[k] = data[k].rstrip()
@@ -22,8 +22,7 @@ import ast,importlib,re
22
22
  from IPython.display import Markdown
23
23
  from fastcore.nbio import read_nb,NbCell
24
24
 
25
- try: import tomllib
26
- except ImportError: import tomli as tomllib
25
+ import tomllib
27
26
 
28
27
  # %% ../nbs/api/01_config.ipynb #117128e6
29
28
  pyproject_nm = 'pyproject.toml'
@@ -171,7 +170,7 @@ def _find_nbdev_pyproject(path=None):
171
170
  # %% ../nbs/api/01_config.ipynb #3dac70e0
172
171
  nbdev_defaults = dict(nbs_path='nbs', doc_path='_docs', tst_flags='notest', recursive=True, readme_nb='index.ipynb',
173
172
  clean_ids=True, clear_all=False, put_version_in_init=True, jupyter_hooks=False, custom_sidebar=False, branch='main',
174
- doc_procs=[], export_procs=[], exec_profile=True, cell_timeout=600)
173
+ doc_procs=[], export_procs=[], exec_profile=True, exec_all=False, cell_timeout=600)
175
174
 
176
175
  _path_keys = 'lib_path', 'nbs_path', 'doc_path'
177
176
 
@@ -5,21 +5,21 @@ Docs: https://nbdev.fast.ai/api/diff.html.md"""
5
5
  # AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/api/19_diff.ipynb.
6
6
 
7
7
  # %% auto #0
8
- __all__ = ['MAXLEN', 'OUT_MAXLEN', 'MIN_RATIO', 'read_nb_from_git', 'nbs_pair', 'changed_cells', 'source_diff', 'cell_diffs',
9
- 'render_diff', 'nb_diff', 'nbdev_diff_driver', 'nbdev_diff']
8
+ __all__ = ['OUT_MAXLEN', 'read_nb_from_git', 'nbs_pair', 'changed_cells', 'cell_diffs', 'render_diff', 'nb_diff',
9
+ 'nbdev_diff_driver', 'nbdev_diff']
10
10
 
11
11
  # %% ../nbs/api/19_diff.ipynb #10c8ad0b
12
12
  import json
13
13
  from fastcore.utils import *
14
14
  from fastcore.meta import delegates
15
15
  from fastcore.script import *
16
- from difflib import unified_diff, SequenceMatcher
17
16
  from subprocess import CalledProcessError
18
17
  from typing import Annotated
19
18
  from fastgit import Git
20
19
  from fastcore.nbio import *
21
20
  from fastcore.nbio import _directive
22
21
  from .doclinks import nbglob
22
+ from .diff_core import *
23
23
 
24
24
  # %% ../nbs/api/19_diff.ipynb #a8981115
25
25
  def read_nb_from_git(
@@ -99,94 +99,16 @@ def changed_cells(nb_path, **kwargs):
99
99
  def f(cid,o,n): return cid
100
100
  return set(_cell_changes(nb_path, f, **kwargs).keys())
101
101
 
102
- # %% ../nbs/api/19_diff.ipynb #079eedb0
103
- def source_diff(
104
- old_source, # Original source string
105
- new_source # New source string
106
- ): # Unified diff string
107
- "Return unified diff string for source change"
108
- return '\n'.join(unified_diff(old_source.splitlines(), new_source.splitlines(), lineterm=''))
109
-
110
102
  # %% ../nbs/api/19_diff.ipynb #38216afa
111
103
  @delegates(_cell_changes)
112
104
  def cell_diffs(nb_path, **kwargs):
113
105
  "{cell_id:diff} for changed/added/deleted cells between two refs"
114
- def f(cid,o,n): return source_diff(o,n)
106
+ def f(cid,o,n): return source_diff(o,n,final_newline=False)
115
107
  return _cell_changes(nb_path, f, **kwargs)
116
108
 
117
109
  # %% ../nbs/api/19_diff.ipynb #d670e58e
118
- MAXLEN = 180 # Most characters shown per displayed line
119
110
  OUT_MAXLEN = 512 # Most characters shown of a cell's stored output
120
- MIN_RATIO = 0.5 # Least similar (by shared words) a changed line pair can be and still merge into one word-diff line
121
- _dcolors = {'## …':'\x1b[34m', '# ':'\x1b[1;7m', '#':'\x1b[1m', '-':'\x1b[31m', '+':'\x1b[32m', '@':'\x1b[36m'}
122
-
123
- def _trunc(s, maxlen):
124
- "`s` cut to `maxlen`, ending `…[n]` with `n` the number of characters missing"
125
- if not maxlen or len(s)<=maxlen: return s
126
- return truncstr(s, maxlen, suf=lambda n: f'…[{humanize(n)}]')
127
-
128
- def _dline(l, maxlen=MAXLEN, color=False):
129
- "Truncate `l` with `_trunc`, optionally colored by its leading diff marker; word-diff spans render via `_wline`"
130
- if not isinstance(l, str): return _wline(l, maxlen, color)
131
- l = _trunc(l, maxlen)
132
- c = first(v for k,v in _dcolors.items() if l.startswith(k)) if color else None
133
- return f'{c}{l}\x1b[0m' if c else l
134
111
 
135
- # %% ../nbs/api/19_diff.ipynb #7a1d7b28
136
- _wcolors = {'-':'\x1b[31;9m', '+':'\x1b[32m'}
137
- _wmarks = {'-':('[-','-]'), '+':('{+','+}')}
138
-
139
- def _tokens(s): return re.findall(r'\w+|\s+|[^\w\s]', s)
140
-
141
- def _word_diff(a, b, maxlen=MAXLEN):
142
- "`(kind, text)` spans merging lines `a` and `b` (' ' shared, '-' only in `a`, '+' only in `b`); None when too different, or longer than `maxlen`"
143
- if maxlen and max(len(a),len(b))>maxlen: return None
144
- if SequenceMatcher(None, re.findall(r'\w+', a), re.findall(r'\w+', b), autojunk=False).ratio()<MIN_RATIO: return None
145
- ta,tb = _tokens(a),_tokens(b)
146
- res = []
147
- for tag,i1,i2,j1,j2 in SequenceMatcher(None, ta, tb, autojunk=False).get_opcodes():
148
- if tag=='equal': res.append((' ', ''.join(ta[i1:i2])))
149
- else:
150
- if i2>i1: res.append(('-', ''.join(ta[i1:i2])))
151
- if j2>j1: res.append(('+', ''.join(tb[j1:j2])))
152
- return res
153
-
154
- def _wline(spans, maxlen=MAXLEN, color=False):
155
- "One `!` line from word-diff `spans`: deletions red and struck out, additions green, or `[-…-]`/`{+…+}` markers without `color`"
156
- if not color: return _dline('!'+''.join(t if k==' ' else _wmarks[k][0]+t+_wmarks[k][1] for k,t in spans), maxlen)
157
- full = '!'+''.join(t for _,t in spans)
158
- cut = _trunc(full, maxlen)
159
- suf = cut[cut.rfind('…['):] if cut!=full else ''
160
- res,room = '!',len(cut)-len(suf)-1
161
- for k,t in spans:
162
- t = t[:room]
163
- room -= len(t)
164
- res += t if k==' ' else f'{_wcolors[k]}{t}\x1b[0m'
165
- if not room: break
166
- return res+suf
167
-
168
- # %% ../nbs/api/19_diff.ipynb #2bf82340
169
- def _refine(lines, maxlen=MAXLEN):
170
- "Diff `lines` with each positionally paired `-`/`+` line merged into word-diff spans where `_word_diff` allows"
171
- res,i = [],0
172
- while i<len(lines):
173
- if not lines[i].startswith('-'):
174
- res.append(lines[i])
175
- i += 1
176
- continue
177
- j = i
178
- while j<len(lines) and lines[j].startswith('-'): j += 1
179
- k = j
180
- while k<len(lines) and lines[k].startswith('+'): k += 1
181
- olds,news = lines[i:j],lines[j:k]
182
- bo,bn = [],[]
183
- for o,n in zip(olds,news):
184
- w = _word_diff(o[1:], n[1:], maxlen)
185
- if w: res,bo,bn = res+bo+bn+[w],[],[]
186
- else: bo,bn = bo+[o],bn+[n]
187
- res += bo+olds[len(news):]+bn+news[len(olds):]
188
- i = k
189
- return res
190
112
 
191
113
  # %% ../nbs/api/19_diff.ipynb #342fc231
192
114
  def _diff_order(old, new):
@@ -201,7 +123,7 @@ def _diff_order(old, new):
201
123
  # %% ../nbs/api/19_diff.ipynb #eb3fc128
202
124
  def _out_lines(c):
203
125
  "Stored outputs of cell `c` rendered as text, capped at `OUT_MAXLEN` chars, each line prefixed `| `"
204
- r = _trunc(render_text(c.get('outputs', [])), OUT_MAXLEN)
126
+ r = truncate(render_text(c.get('outputs', [])), OUT_MAXLEN)
205
127
  return [f'| {l}' for l in r.splitlines()]
206
128
 
207
129
  def render_diff(
@@ -210,6 +132,7 @@ def render_diff(
210
132
  maxlen:int=MAXLEN, # Truncate diff lines to this width (falsy: no limit)
211
133
  color:bool=False, # Add ANSI colors?
212
134
  context:int=1, # Unchanged cells to show either side of each change
135
+ line_context:int=3, # Unchanged source lines to show either side of each change
213
136
  show_out:bool=True # Show the stored outputs of changed and added cells?
214
137
  ): # One section per shown cell, in notebook order
215
138
  "Render cell-level changes between two notebooks as truncated unified diffs, with context cells and outputs"
@@ -220,16 +143,19 @@ def render_diff(
220
143
  res,prev = [],-1
221
144
  for i,cid in enumerate(order):
222
145
  if i not in show: continue
223
- if 0<=prev<i-1: res.append(_dline(f'## … {i-prev-1} cells', maxlen, color))
146
+ if 0<=prev<i-1: res.append(render_line(f'## … {i-prev-1} cells', maxlen, color))
224
147
  prev = i
225
148
  c = new.get(cid) or old[cid]
226
- if not _changed(cid): lines = [f'## {cid}{_dtag(c)}:'] + [' '+l for l in _src(c).splitlines()]
149
+ if not _changed(cid):
150
+ lines = [render_line(l, maxlen, color) for l in [f'## {cid}{_dtag(c)}:'] + [' '+l for l in _src(c).splitlines()]]
227
151
  else:
228
152
  o,n = (_src(old[cid]) if cid in old else ''), (_src(new[cid]) if cid in new else '')
229
153
  kind = 'modified' if cid in old and cid in new else 'added' if cid in new else 'deleted'
230
- lines = [f'## {kind} {cid}{_dtag(c)}:'] + _refine(source_diff(o,n).splitlines()[2:], maxlen)
231
- if show_out and cid in new: lines += _out_lines(new[cid])
232
- res.append('\n'.join(_dline(l, maxlen, color) for l in lines))
154
+ lines = [render_line(f'## {kind} {cid}{_dtag(c)}:', maxlen, color)]
155
+ source = render_source_diff(o, n, maxlen=maxlen, color=color, context=line_context, final_newline=False)
156
+ if source: lines.append(source)
157
+ if show_out and cid in new: lines += [render_line(l, maxlen, color) for l in _out_lines(new[cid])]
158
+ res.append('\n'.join(lines))
233
159
  return '\n\n'.join(res)
234
160
 
235
161
  # %% ../nbs/api/19_diff.ipynb #7f90d675
@@ -240,11 +166,12 @@ def nb_diff(
240
166
  maxlen:int=MAXLEN, # Truncate diff lines to this width (falsy: no limit)
241
167
  color:bool=False, # Add ANSI colors?
242
168
  context:int=1, # Unchanged cells to show either side of each change
169
+ line_context:int=3, # Unchanged source lines to show either side of each change
243
170
  show_out:bool=True # Show the stored outputs of changed and added cells?
244
171
  ): # Rendered diff of changed cells
245
172
  "Rendered cell diff for `nb_path` between two refs"
246
173
  a,b = nbs_pair(nb_path, ref_a, ref_b)
247
- return render_diff(a, b, maxlen=maxlen, color=color, context=context, show_out=show_out)
174
+ return render_diff(a, b, maxlen=maxlen, color=color, context=context, line_context=line_context, show_out=show_out)
248
175
 
249
176
  # %% ../nbs/api/19_diff.ipynb #ab9f0b26
250
177
  def _file_cells(path):
@@ -257,7 +184,7 @@ _pos = Annotated[str, {'opt':False, 'nargs':'?'}] # optional positional CLI arg
257
184
 
258
185
  @call_parse
259
186
  def nbdev_diff_driver(
260
- path:str, # Repo path of the notebook being diffed
187
+ path:str, # Repo path of the file being diffed
261
188
  old_file:_pos=None, # Pre-change file (git temp file, or /dev/null)
262
189
  old_hex:_pos=None, # Pre-change blob hash
263
190
  old_mode:_pos=None, # Pre-change file mode
@@ -267,16 +194,30 @@ def nbdev_diff_driver(
267
194
  rename_to:_pos=None, # New repo path, when git detected a rename
268
195
  similarity:_pos=None, # Similarity score, when git detected a rename
269
196
  maxlen:int=MAXLEN, # Truncate diff lines to this width (0 for no limit)
270
- context:int=1, # Unchanged cells to show either side of each change
197
+ context:int=1, # Unchanged notebook cells to show either side of each change
198
+ line_context:int=3, # Unchanged source lines to show either side of each change
271
199
  show_out:bool=True # Show the stored outputs of changed and added cells?
272
200
  ):
273
- "Git external diff driver for notebooks; installed by `nbdev-install-hooks`"
201
+ "Git external diff driver for notebooks and plain text files"
274
202
  if not new_file: return print(f'unmerged: {path}')
275
203
  color = sys.stdout.isatty() or os.environ.get('GIT_PAGER_IN_USE')=='true'
276
- res = render_diff(_file_cells(old_file), _file_cells(new_file), maxlen=maxlen, color=color, context=context, show_out=show_out)
277
- if not res: return
278
- print(_dline(f'# {path}', 0, color))
279
- print(res, end='\n\n')
204
+ if Path(path).suffix=='.ipynb':
205
+ res = render_diff(_file_cells(old_file), _file_cells(new_file), maxlen=maxlen, color=color,
206
+ context=context, line_context=line_context, show_out=show_out)
207
+ else:
208
+ old,new = Path(old_file).read_bytes(),Path(new_file).read_bytes()
209
+ res = 'Binary files differ' if old!=new else ''
210
+ if b'\0' not in old and b'\0' not in new:
211
+ try: res = render_source_diff(old.decode('utf-8'), new.decode('utf-8'), maxlen=maxlen, color=color, context=line_context)
212
+ except UnicodeDecodeError: pass
213
+ meta = []
214
+ if rename_to: meta.append(f'rename to {rename_to}')
215
+ if old_mode and new_mode and old_mode!=new_mode: meta.append(f'mode {old_mode} -> {new_mode}')
216
+ if not res and not meta: return
217
+ print(render_line(f'# {path}', 0, color))
218
+ for line in meta: print(render_line('# '+line, 0, color))
219
+ if res: print(res)
220
+ print()
280
221
 
281
222
  # %% ../nbs/api/19_diff.ipynb #2604c719
282
223
  @call_parse
@@ -287,10 +228,11 @@ def nbdev_diff(
287
228
  maxlen:int=MAXLEN, # Truncate diff lines to this width (0 for no limit)
288
229
  color:bool_arg=None, # Add ANSI colors? (default: only if stdout is a tty)
289
230
  context:int=1, # Unchanged cells to show either side of each change
231
+ line_context:int=3, # Unchanged source lines to show either side of each change
290
232
  show_out:bool=True # Show the stored outputs of changed and added cells?
291
233
  ):
292
234
  "Cell-level diffs for changed notebooks between two git refs"
293
235
  if color is None: color = sys.stdout.isatty()
294
236
  for p in sorted(nbglob(path)):
295
- res = nb_diff(p, ref_a, ref_b, maxlen=maxlen, color=color, context=context, show_out=show_out)
296
- if res: print(_dline(f'# {p}', 0, color), res, sep='\n', end='\n\n')
237
+ res = nb_diff(p, ref_a, ref_b, maxlen=maxlen, color=color, context=context, line_context=line_context, show_out=show_out)
238
+ if res: print(render_line(f'# {p}', 0, color), res, sep='\n', end='\n\n')
@@ -0,0 +1,148 @@
1
+ "Word-level terminal diffs of plain text, shared by file and notebook rendering."
2
+
3
+ from difflib import unified_diff, SequenceMatcher
4
+ from fastcore.utils import *
5
+
6
+ __all__ = ['MAXLEN', 'MIN_RATIO', 'truncate', 'render_line', 'source_diff', 'render_source_diff']
7
+
8
+ MAXLEN = 180 # Most visible characters per displayed line; 0 means unlimited
9
+ MIN_RATIO = 0.5 # Minimum shared-word similarity for merging a changed line pair
10
+ _dcolors = {'## …':'\x1b[34m', '# ':'\x1b[1;7m', '#':'\x1b[1m', '-':'\x1b[31m', '+':'\x1b[32m', '@':'\x1b[36m'}
11
+ _wcolors = {'-':'\x1b[31;9m', '+':'\x1b[32m'}
12
+ _wmarks = {'-':('[-','-]'), '+':('{+','+}')}
13
+
14
+
15
+ def truncate(s, maxlen):
16
+ "Cut `s` to `maxlen`, marking the number of omitted characters; 0 means unlimited."
17
+ if not maxlen or len(s)<=maxlen: return s
18
+ return truncstr(s, maxlen, suf=lambda n: f'…[{humanize(n)}]')[:maxlen]
19
+
20
+
21
+ def render_line(
22
+ line, # A display line, including its diff or heading marker
23
+ maxlen=MAXLEN, # Maximum visible length; 0 means unlimited
24
+ color=False # Add ANSI colors?
25
+ ):
26
+ "Truncate and optionally color a line by its leading display marker."
27
+ line = truncate(line, maxlen)
28
+ c = next((v for k,v in _dcolors.items() if line.startswith(k)), '') if color else ''
29
+ return f'{c}{line}\x1b[0m' if c else line
30
+
31
+
32
+ def _source_lines(old, new, context, final_newline):
33
+ def lines(s): return s.splitlines(keepends=True) if final_newline else [l+'\n' for l in s.splitlines()]
34
+ return list(unified_diff(lines(old), lines(new), n=context))
35
+
36
+
37
+ def source_diff(
38
+ old_source, # Original source string
39
+ new_source, # New source string
40
+ context=3, # Unchanged lines shown either side of a change
41
+ final_newline=True # Distinguish a missing final newline (unnecessary for notebook cells)?
42
+ ):
43
+ "Unified source diff, including missing-final-newline markers."
44
+ lines = _source_lines(old_source, new_source, context, final_newline)
45
+ return '\n'.join(l.removesuffix('\n') + ('' if l.endswith('\n') else '\n\') for l in lines)
46
+
47
+
48
+ def _word_diff(a, b):
49
+ if a==b: return None
50
+ if SequenceMatcher(None, re.findall(r'\w+', a), re.findall(r'\w+', b), autojunk=False).ratio()<MIN_RATIO: return None
51
+ ta,tb = (re.findall(r'\w+|\s+|[^\w\s]', s) for s in (a,b))
52
+ ops = SequenceMatcher(lambda t: t.isspace(), ta, tb, autojunk=False).get_opcodes()
53
+ def width(op): return max(sum(map(len, ta[op[1]:op[2]])), sum(map(len, tb[op[3]:op[4]])))
54
+ groups = []
55
+ for i,(tag,i1,i2,j1,j2) in enumerate(ops):
56
+ # Short coincidences inside a rewrite should not split it into alternating words.
57
+ if tag=='equal' and 0<i<len(ops)-1 and width(ops[i])<min(width(ops[i-1]),width(ops[i+1])): tag = 'replace'
58
+ if tag!='equal' and groups and groups[-1][0]!='equal':
59
+ _,i1,_,j1,_ = groups.pop()
60
+ tag = 'replace'
61
+ groups.append((tag,i1,i2,j1,j2))
62
+ res = []
63
+ for tag,i1,i2,j1,j2 in groups:
64
+ if tag=='equal': res.append((' ', ''.join(ta[i1:i2])))
65
+ else:
66
+ if i2>i1: res.append(('-', ''.join(ta[i1:i2])))
67
+ if j2>j1: res.append(('+', ''.join(tb[j1:j2])))
68
+ return res
69
+
70
+
71
+ def _refine(lines):
72
+ res,i = [],0
73
+ while i<len(lines):
74
+ if not lines[i].startswith('-'):
75
+ res.append(lines[i])
76
+ i += 1
77
+ continue
78
+ j = i
79
+ while j<len(lines) and lines[j].startswith('-'): j += 1
80
+ k = j
81
+ while k<len(lines) and lines[k].startswith('+'): k += 1
82
+ olds,news = lines[i:j],lines[j:k]
83
+ bo,bn = [],[]
84
+ for o,n in zip(olds,news):
85
+ w = _word_diff(o[1:], n[1:])
86
+ if w: res,bo,bn = res+bo+bn+[w],[],[]
87
+ else: bo,bn = bo+[o],bn+[n]
88
+ res += bo+olds[len(news):]+bn+news[len(olds):]
89
+ i = k
90
+ return res
91
+
92
+
93
+ def _slice_spans(spans, start, end):
94
+ res,pos = [],0
95
+ for k,t in spans:
96
+ a,b = max(0,start-pos), min(len(t),end-pos)
97
+ if a<b: res.append((k,t[a:b]))
98
+ pos += len(t)
99
+ if pos>=end: break
100
+ return res
101
+
102
+
103
+ def _span_width(spans, color): return sum(len(t)+(4 if k!=' ' and not color else 0) for k,t in spans)
104
+
105
+
106
+ def _render_spans(spans, maxlen, color):
107
+ total = sum(len(t) for _,t in spans)
108
+ start,end = 0,total
109
+ if maxlen and _span_width(spans, color)+1>maxlen:
110
+ first = int(spans[0][0]==' ')
111
+ prefix = len(spans[0][1]) if first else 0
112
+ last = first
113
+ while last<len(spans) and spans[last][0]!=' ': last += 1
114
+ room = maxlen-1-_span_width(spans[first:last], color)-2*len(f'…[{total}]')
115
+ start = max(0,prefix-min(20,maxlen//4,max(0,room)))
116
+ end = min(total,start+maxlen-1)
117
+ while True:
118
+ shown = _slice_spans(spans, start, end)
119
+ head,tail = (f'…[{start}]' if start else ''), (f'…[{total-end}]' if end<total else '')
120
+ excess = 1+len(head)+_span_width(shown, color)+len(tail)-maxlen
121
+ if excess<=0: break
122
+ if end==start: return truncate('!'+head+tail, maxlen)
123
+ end = max(start,end-excess)
124
+ else: shown,head,tail = spans,'',''
125
+ def fmt(k,t):
126
+ if k==' ': return t
127
+ if color: return f'{_wcolors[k]}{t}\x1b[0m'
128
+ return _wmarks[k][0]+t+_wmarks[k][1]
129
+ return '!'+head+''.join(fmt(k,t) for k,t in shown)+tail
130
+
131
+
132
+ def render_source_diff(
133
+ old_source, # Original source string
134
+ new_source, # New source string
135
+ maxlen=MAXLEN, # Maximum visible length; 0 means unlimited
136
+ color=False, # Add ANSI colors instead of [-removed-]/{+added+} markers?
137
+ context=3, # Unchanged lines shown either side of a change
138
+ final_newline=True # Distinguish a missing final newline (unnecessary for notebook cells)?
139
+ ):
140
+ "Render source changes, cropping long word-diff lines around the first change."
141
+ lines = _source_lines(old_source, new_source, context, final_newline)
142
+ if not lines: return ''
143
+ body = [l.removesuffix('\n') for l in lines[2:]]
144
+ res = [render_line(l, maxlen, color) if isinstance(l,str) else _render_spans(l, maxlen, color) for l in _refine(body)]
145
+ if final_newline:
146
+ for name,text in (('old',old_source), ('new',new_source)):
147
+ if text and not text.endswith('\n'): res.append(render_line(fr'\ No newline at end of {name} file', maxlen, color))
148
+ return '\n'.join(res)
@@ -230,6 +230,7 @@ def _qual_syms(entries):
230
230
 
231
231
  # %% ../nbs/api/05_doclinks.ipynb #53505fa4
232
232
  _re_backticks = re.compile(r'`([^`\s]+?)(?:\(\))?`')
233
+ _re_linkify = re.compile(r'\[(?:`[^`]*`|[^\]`])*\](?:\((?:[^()]|\([^()]*\))*\)|\[[^\]]*\])?|' + _re_backticks.pattern)
233
234
 
234
235
  # %% ../nbs/api/05_doclinks.ipynb #3a24b883
235
236
  @lru_cache(None)
@@ -291,19 +292,22 @@ class NbdevLookup:
291
292
 
292
293
  def _link_sym(self, m):
293
294
  l = m.group(1)
295
+ if l is None: return m.group(0)
294
296
  s = self.doc(l)
295
297
  if s is None: return m.group(0)
296
298
  l = l.replace('\\', r'\\')
297
299
  if m.group(0).endswith('()`'): l += '()'
298
300
  return rf"[`{l}`]({s})"
299
301
 
300
- def link_line(self, l): return _re_backticks.sub(self._link_sym, l)
302
+ def link_line(self, l): return _re_linkify.sub(self._link_sym, l)
301
303
 
302
304
  def linkify(self, md):
303
- if md:
304
- in_fence=False
305
- lines = md.splitlines()
306
- for i,l in enumerate(lines):
307
- if l.startswith("```"): in_fence=not in_fence
308
- elif not l.startswith(' ') and not in_fence: lines[i] = self.link_line(l)
309
- return '\n'.join(lines)
305
+ if not md: return md
306
+ fence = None
307
+ lines = md.splitlines(keepends=True)
308
+ for i,l in enumerate(lines):
309
+ if fence:
310
+ if re.fullmatch(rf' {{0,3}}{fence[0]}{{{len(fence)},}}\s*', l): fence = None
311
+ elif m := re.match(r' {0,3}(`{3,}|~{3,})', l): fence = m[1]
312
+ elif not l.startswith((' ', '\t')): lines[i] = self.link_line(l)
313
+ return ''.join(lines)
@@ -228,11 +228,15 @@ def _ast_contains(trees, types):
228
228
 
229
229
  def _tl_contains(trees, types): return any(isinstance(t, types) for t in trees)
230
230
 
231
- def _do_eval(cell):
231
+ def _do_eval(cell, all_cells=False, dflt=True, flags=frozenset()):
232
232
  if cell_lang(cell) != 'python': return
233
233
  if not cell.source or 'nbdev_export'+'()' in cell.source: return
234
234
  trees = cell.parsed_()
235
235
  if cell.cell_type != 'code' or not trees: return
236
+ if all_cells:
237
+ v = cell.directives_.get('eval')
238
+ run = dflt if v is None else v.lower() != 'false'
239
+ return run and not (cell.directives_.keys() & flags) and 'skip-execution' not in cell.metadata.get('tags', [])
236
240
  if cell.directives_.get('eval', '').lower() == 'false': return
237
241
 
238
242
  _show_dirs = {'export','exports','exporti','exec_doc'}
@@ -245,17 +249,22 @@ def _do_eval(cell):
245
249
 
246
250
  # %% ../nbs/api/10_processors.ipynb #867cf721
247
251
  class exec_show_docs(Processor):
248
- "Execute cells needed for `show_docs` output, including exported cells and imports"
252
+ "Execute cells needed for `show_docs` output, including exported cells and imports, or every cell under `exec_all`"
249
253
  def begin(self):
250
254
  if nb_lang(self.nb) != 'python': return
251
- self.k = CaptureShell(profile=bool(get_config().exec_profile))
255
+ cfg,fm = get_config(),getattr(self.nb, 'frontmatter_', {})
256
+ self.all_cells = str2bool(fm.get('exec_all', cfg.exec_all))
257
+ self.dflt = fm_default_eval(fm)
258
+ flags = cfg.tst_flags
259
+ self.flags = set(flags.split() if isinstance(flags, str) else flags)
260
+ self.k = CaptureShell(profile=bool(cfg.exec_profile))
252
261
  self.k.run_cell('from nbdev.showdoc import show_doc')
253
262
 
254
263
  def __call__(self, cell):
255
- if not self.nb.has_docs_ or not hasattr(self, 'k'): return
264
+ if not hasattr(self, 'k') or not (self.nb.has_docs_ or self.all_cells): return
256
265
  fm = getattr(self.nb, 'frontmatter_', {})
257
266
  if str2bool(fm.get('skip_showdoc', False)): return
258
- if _do_eval(cell): self.k.cell(cell)
267
+ if _do_eval(cell, self.all_cells, self.dflt, self.flags): self.k.cell(cell)
259
268
  title = fm.get('title', '')
260
269
  if (e := self.k.exc):
261
270
  raise Exception(f"Error{' in notebook: '+title if title else ''} in cell {cell.idx_} :\n{cell.source}\n{type(e).__name__}: {e}") from e
@@ -1,4 +1,4 @@
1
- """A parallel ipynb processor (experimental)
1
+ """A parallel notebook and Quarto processor (experimental)
2
2
 
3
3
  Docs: https://nbdev.fast.ai/api/serve.html.md"""
4
4
 
@@ -8,13 +8,12 @@ Docs: https://nbdev.fast.ai/api/serve.html.md"""
8
8
  __all__ = ['proc_nbs']
9
9
 
10
10
  # %% ../nbs/api/17_serve.ipynb #6899a335
11
- import ast,subprocess,threading,sys
11
+ import ast
12
12
  from shutil import rmtree,copy2
13
13
  from fnmatch import fnmatch
14
14
 
15
15
  from fastcore.utils import *
16
16
  from fastcore.parallel import parallel
17
- from fastcore.script import call_parse
18
17
  from fastcore.meta import delegates
19
18
 
20
19
  from .config import get_config
@@ -46,7 +45,7 @@ def _proc_file(s, cache, path):
46
45
  if d.exists() and s.stat().st_mtime<=d.stat().st_mtime: return
47
46
 
48
47
  d.parent.mkdir(parents=True, exist_ok=True)
49
- if s.suffix=='.ipynb': return s,d,FilterDefaults
48
+ if s.suffix in ('.ipynb','.qmd'): return s,d,FilterDefaults
50
49
  md = _is_qpy(s)
51
50
  if md is not None: return s,d,md.strip()
52
51
  else: copy2(s,d)
@@ -95,7 +94,6 @@ def proc_nbs(
95
94
  _rm_stale(cache, path)
96
95
 
97
96
  files = files.map(_proc_file, cache=cache, path=path).filter()
98
- kw = {} if in_notebook() else {'method':'spawn'}
99
- parallel(nbdev.serve_drv.main, files, n_workers=n_workers, pause=0.01, **kw)
97
+ parallel(nbdev.serve_drv.main, files, n_workers=n_workers, pause=0.01, max_tasks_per_child=1)
100
98
  if cache.exists(): cache.touch()
101
99
  return cache
@@ -1,15 +1,24 @@
1
1
  import os
2
2
  from fastcore.nbio import read_nb,write_nb
3
+ from nbdev.doclinks import NbdevLookup
4
+ from nbdev.frontmatter import _re_fm_md
3
5
  from io import StringIO
4
6
  from contextlib import redirect_stdout
5
7
 
8
+ def link_qmd(text):
9
+ "Linkify a Quarto document without changing its YAML frontmatter."
10
+ fm = _re_fm_md.match(text)
11
+ start = fm.end() if fm else 0
12
+ return text[:start] + NbdevLookup().linkify(text[start:])
13
+
6
14
  def exec_scr(src, dst, md):
7
15
  f = StringIO()
8
16
  g = {}
9
17
  with redirect_stdout(f): exec(compile(src.read_text(), src, 'exec'), g)
10
18
  res = ""
11
19
  if md: res += "---\n" + md + "\n---\n\n"
12
- dst.write_text(res + f.getvalue())
20
+ text = res + f.getvalue()
21
+ dst.write_text(link_qmd(text) if dst.suffix=='.qmd' else text)
13
22
 
14
23
  def exec_nb(src, dst, cb):
15
24
  nb = read_nb(src)
@@ -21,6 +30,7 @@ def main(o):
21
30
  os.environ["IN_TEST"] = "1"
22
31
  if src.suffix=='.ipynb': exec_nb(src, dst, x)
23
32
  elif src.suffix=='.py': exec_scr(src, dst, x)
33
+ elif src.suffix=='.qmd': dst.write_text(link_qmd(src.read_text()))
24
34
  else: raise Exception(src)
25
35
  del os.environ["IN_TEST"]
26
36
 
@@ -0,0 +1,60 @@
1
+ # NB: this file is designed for an LLM reader, not a human. Add it to your LLM's context and ask it to explain.
2
+ """Write, read and review nbdev notebooks as source, documentation, examples and tests. You MUST read this before any work on a project with notebooks.
3
+
4
+ notation: /why
5
+
6
+ ## Source
7
+
8
+ - published library notebooks: implementation, explanation, examples, outputs, failure demos interleaved /read top to bottom; internal project: match neighbours
9
+ - per file: `# AUTOGENERATED! DO NOT EDIT!` → edit its notebook (`# %% ../nbs/04_usage.ipynb #a45f753a` names the cell), never module, `__all__`, `_modidx.py`, module docstring /`nbdev-export` regenerates them; unmarked: hand-written
10
+ - understand and review through notebooks and their git diff, not generated modules /no prose, examples, tests there
11
+ - notebooks teach APIs, lessons are the tests; plain `.py`, tested by `tests/*.py`: rare dense mechanism iterated under pytest; never add pytests to a project lacking them unless asked
12
+ - project questions: `nbs/index.ipynb`, README's source; after editing it: `nbdev-readme`
13
+
14
+ ## Authoring
15
+
16
+ - before editing: whole notebook or every cell's summary
17
+ - new notebook: first read a role model such as claudette's `00_core.ipynb` for literate form; one of tutorial, how-to, explanation, reference; H1 title cell, `>` subtitle, intro; H2 per major concept, short narratives unsplit; each heading its own markdown cell /Jupyter collapse
18
+ - one idea at a time: small implementation, explanation, lesson; every substantive API gets a lesson; basic functionality outside the API with little logic: usually none
19
+ - lesson: markdown intro, split when complicated; extend examples, no near-duplicates: explain the differing argument, no repeated setup; classes: `@patch` each method beside its explanation unless that obscures the API
20
+ - new behaviour worth teaching (often not, for trivial logic): choose its narrative position; lesson first, ideally by changing an existing one; run preceding cells, see it fail; intro and implementation above it, see it pass; any changed assertion likewise fails on old code, passes on new
21
+ - bug fix: make an existing example hit the failing path only for subtle logic likely to recur or a tiny readable change, else add an assertion to it; never a cell just to witness a fix; no documented behaviour change: no lesson change
22
+ - explore in the notebook, not a separate kernel, after reading existing examples; keep useful findings as lessons, delete the rest; try to leave the notebook better
23
+ - `_name`: machinery without independent contract, not everything used internally; helper useful only internally: often awkward design
24
+
25
+ ## Lessons
26
+
27
+ - readability first: code and displayed output guide readers through what code does; displayed value worth testing: assign, test, show; test the point, not incidentals like formatting; errors only where readers are likely to hit them
28
+ - realistic values, minimal setup, reused objects, direct supporting assertions; `fastcore.test`, `expect_fail`, in ordinary code cells
29
+ - evidence: compact `_repr_markdown_`, structured summary, plot, table, image, HTML; large output: say what matters, store focused results
30
+ - test-serving cells, e.g. safety-only setup, glue checks: off the page; usually remove; substantive logic check → `#| hide` or pytest; discuss such changes carefully
31
+ - reader-facing examples: no plumbing, mocks, dense comprehensions, long setup → small helper or hidden check
32
+ - rewrites to consider: commented example → markdown and code, keeping constraint comments; numbered steps → markdown/code pairs; `print` inside a sentence → prose or displayed value; final `print(x)` → `x`; reassignment stages, blank-line groups → introduced cells; `try`/`except` → `expect_fail`; long docstring → markdown; TODO, FIXME in example comments → resolve or issue
33
+
34
+ ## Prose and exports
35
+
36
+ - markdown: rationale, distinctions, guarantees, extended explanation, examples, warnings; not code narration or filler; normally after definitions /generated page reads correctly
37
+ - symbol docstrings short; parameters, return: docments beside signature; backticked `symbol` → nbdev link
38
+ - module docstring: `nbdev-export` joins title's `>` description and exported markdown after H1; long: one exported markdown cell per major section; shared constraints, workflows only /`doc(module)` lists functions
39
+ - docstring beyond the default summary: only for modules read alone /`nbdev-export` then lists module in package docs, llms.txt
40
+ - `#| exportd`: code cell → fenced docstring source, not module code; compact runnable demo after its definitions; preceding exported summary ends with a colon; unexported cells between: omitted
41
+ - run code before documenting a claim; `nbdev.export.nb_mdoc` previews assembly
42
+ - `index.ipynb`: export opening paragraphs /package intro
43
+
44
+ ## State and execution
45
+
46
+ - define near first use; introduce shared setup; reuse objects; never reassign names later cells use; exploration: end with an expression recording its finding, when useful
47
+ - imports: own cells, lessons too /docs builds run import cells in fresh namespaces: mixed state-dependent code fails or runs at docs time
48
+ - outputs: generated, may be stale mid-edit; source/output mismatch: keep working, never clear by hand or edit notebook JSON; `nbdev-test --save` at PR time if outputs are kept; `--file-re`: several named notebooks in parallel
49
+
50
+ ## nbdev v3 (Jan 2026)
51
+
52
+ - `#| hide`: distracting necessities; exported `_helpers` needn't be public; `#| eval: false`: must not run, not broken; later cells can't use its state
53
+ - directives also as metadata `{"nbdev": {"export": "true"}}`; `fastcore.nbio.mk_cell(...).directives` merges both, content wins
54
+ - `pyproject.toml`, not `settings.ini`: `[project]` metadata, `[tool.nbdev]` settings (defaults `nbs_path='nbs'`, `doc_path='_docs'`), version via `dynamic = ["version"]` in `__init__.py`, `_modidx` in `[project.entry-points.nbdev]`
55
+ - CLIs hyphenated (`nbdev-export`, `nbdev-test`), Python names underscored; actions `fastai/workflows/nbdev3-ci`, `quarto-ghp3`
56
+
57
+ ## Finish
58
+
59
+ - update affected prose, displays, assertions together; reread touched narrative for readability; `nbdev-test <nb>`
60
+ """
@@ -156,7 +156,7 @@ def nbdev_test(
156
156
  cell_timing_min:float=None, # Print cells slower than this many seconds (default `cell_timing_min` config, else none)
157
157
  **kwargs
158
158
  ):
159
- "Test in parallel notebooks matching `path`, passing along `flags`"
159
+ "Test in parallel notebooks matching `path`, each in a process of its own, passing along `flags`"
160
160
  if path and not Path(path).exists(): kwargs['path_glob'],path = os.path.abspath(path),Path.cwd()
161
161
  cfg = get_config(Path(path).resolve() if path else None)
162
162
  cell_timeout,cell_timing_min = ifnone(cell_timeout, cfg.cell_timeout),ifnone(cell_timing_min, cfg.get('cell_timing_min'))
@@ -168,8 +168,6 @@ def nbdev_test(
168
168
  if len(files)==0: return print('No files were eligible for testing')
169
169
 
170
170
  if n_workers is None: n_workers = 0 if len(files)==1 else min(num_cpus(), 8)
171
- if in_notebook(): kw = {'method':'spawn'} if os.name=='nt' or sys.platform=='darwin' else {'method':'forkserver'}
172
- else: kw = {'method':'spawn'} if sys.platform=='darwin' else {}
173
171
  wd_pth = cfg.nbs_path
174
172
  with working_directory(wd_pth if (wd_pth and wd_pth.exists()) else os.getcwd()):
175
173
  setup = cfg.get('test_setup')
@@ -177,7 +175,7 @@ def nbdev_test(
177
175
  try:
178
176
  results = parallel(test_nb, files, skip_flags=skip_flags, force_flags=force_flags, n_workers=n_workers,
179
177
  basepath=cfg.config_path, pause=pause, do_print=do_print, verbose=verbose, save=save,
180
- cell_timeout=cell_timeout or None, cell_timing_min=cell_timing_min, **kw)
178
+ cell_timeout=cell_timeout or None, cell_timing_min=cell_timing_min, max_tasks_per_child=1)
181
179
  except KeyboardInterrupt:
182
180
  sys.stderr.write('\nnbdev-test interrupted; in-flight notebook stacks shown above\n')
183
181
  sys.exit(130)
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: nbdev
3
- Version: 3.3.27
3
+ Version: 3.3.29
4
4
  Summary: Create delightful software with Jupyter Notebooks
5
5
  Author-email: "Jeremy Howard and the fast.ai community" <j@fast.ai>
6
6
  License-Expression: Apache-2.0
@@ -12,10 +12,10 @@ Classifier: Intended Audience :: Developers
12
12
  Classifier: Natural Language :: English
13
13
  Classifier: Programming Language :: Python :: 3
14
14
  Classifier: Programming Language :: Python :: 3 :: Only
15
- Requires-Python: >=3.10
15
+ Requires-Python: >=3.11
16
16
  Description-Content-Type: text/markdown
17
17
  License-File: LICENSE
18
- Requires-Dist: fastcore>=2.2.28
18
+ Requires-Dist: fastcore>=2.2.39
19
19
  Requires-Dist: execnb>=0.3.3
20
20
  Requires-Dist: astunparse
21
21
  Requires-Dist: ghapi>=2.0.2
@@ -25,7 +25,6 @@ Requires-Dist: setuptools
25
25
  Requires-Dist: build
26
26
  Requires-Dist: fastgit>=0.0.7
27
27
  Requires-Dist: pyyaml
28
- Requires-Dist: tomli; python_version < "3.11"
29
28
  Provides-Extra: dev
30
29
  Requires-Dist: ipywidgets; extra == "dev"
31
30
  Requires-Dist: nbdev-numpy; extra == "dev"
@@ -40,6 +39,7 @@ Requires-Dist: sphinx; extra == "dev"
40
39
  Requires-Dist: plum-dispatch; extra == "dev"
41
40
  Requires-Dist: nbclient; extra == "dev"
42
41
  Requires-Dist: ipykernel; extra == "dev"
42
+ Requires-Dist: pytest; extra == "dev"
43
43
  Dynamic: license-file
44
44
 
45
45
  # Getting Started
@@ -8,6 +8,7 @@ nbdev/clean.py
8
8
  nbdev/cli.py
9
9
  nbdev/config.py
10
10
  nbdev/diff.py
11
+ nbdev/diff_core.py
11
12
  nbdev/doclinks.py
12
13
  nbdev/export.py
13
14
  nbdev/extract_attachments.py
@@ -37,4 +38,5 @@ nbdev.egg-info/dependency_links.txt
37
38
  nbdev.egg-info/entry_points.txt
38
39
  nbdev.egg-info/requires.txt
39
40
  nbdev.egg-info/top_level.txt
41
+ tests/test_diff.py
40
42
  tests/test_quarto_render.py
@@ -1,4 +1,4 @@
1
- fastcore>=2.2.28
1
+ fastcore>=2.2.39
2
2
  execnb>=0.3.3
3
3
  astunparse
4
4
  ghapi>=2.0.2
@@ -9,9 +9,6 @@ build
9
9
  fastgit>=0.0.7
10
10
  pyyaml
11
11
 
12
- [:python_version < "3.11"]
13
- tomli
14
-
15
12
  [dev]
16
13
  ipywidgets
17
14
  nbdev-numpy
@@ -26,3 +23,4 @@ sphinx
26
23
  plum-dispatch
27
24
  nbclient
28
25
  ipykernel
26
+ pytest
@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
6
6
  name = "nbdev"
7
7
  description = "Create delightful software with Jupyter Notebooks"
8
8
  readme = "README.md"
9
- requires-python = ">=3.10"
9
+ requires-python = ">=3.11"
10
10
  license = "Apache-2.0"
11
11
  authors = [{name = "Jeremy Howard and the fast.ai community", email = "j@fast.ai"}]
12
12
  keywords = ["nbdev", "fastai", "jupyter", "notebook", "export"]
@@ -18,11 +18,11 @@ classifiers = [
18
18
  "Programming Language :: Python :: 3 :: Only",
19
19
  ]
20
20
  dynamic = ["version"]
21
- dependencies = [ "fastcore>=2.2.28", "execnb>=0.3.3", "astunparse", "ghapi>=2.0.2", "watchdog", "asttokens",
22
- "setuptools", "build", "fastgit>=0.0.7", "pyyaml", "tomli; python_version < '3.11'", ]
21
+ dependencies = [ "fastcore>=2.2.39", "execnb>=0.3.3", "astunparse", "ghapi>=2.0.2", "watchdog", "asttokens",
22
+ "setuptools", "build", "fastgit>=0.0.7", "pyyaml", ]
23
23
 
24
24
  [project.optional-dependencies]
25
- dev = [ "ipywidgets", "nbdev-numpy", "nbdev-stdlib", "pandas", "matplotlib", "black", "svg.py", "pysym2md>=0.0.6", "llms-txt", "sphinx", "plum-dispatch", "nbclient", "ipykernel", ]
25
+ dev = [ "ipywidgets", "nbdev-numpy", "nbdev-stdlib", "pandas", "matplotlib", "black", "svg.py", "pysym2md>=0.0.6", "llms-txt", "sphinx", "plum-dispatch", "nbclient", "ipykernel", "pytest", ]
26
26
 
27
27
  [project.urls]
28
28
  Repository = "https://github.com/AnswerDotAI/nbdev"
@@ -0,0 +1,93 @@
1
+ import os, re, shlex, subprocess, sys
2
+ from fastcore.nbio import new_nb, write_nb
3
+ from nbdev.diff_core import render_source_diff, source_diff
4
+
5
+
6
+ def test_source_changes():
7
+ old = 'x=1\ny=2\nz=3\nk=4\n'
8
+ new = 'x = 100\ny=2\nw=5\nq=6\n'
9
+ result = render_source_diff(old, new)
10
+ assert '!x{+ +}=[-1-]{+ 100+}' in result
11
+ assert '-z=3\n-k=4\n+w=5\n+q=6' in result
12
+ assert '+hello' in render_source_diff('', 'hello\n')
13
+ assert '-hello' in render_source_diff('hello\n', '')
14
+ assert not render_source_diff(old, old)
15
+
16
+
17
+ def test_phrase_changes():
18
+ old = 'selects the fields, shown by the vectors'
19
+ new = 'reverses the coordinates, arranged in two columns'
20
+ suffix = '. Dragging either point redraws both points.\n'
21
+ for color in (False, True):
22
+ result = render_source_diff('The model '+old+suffix, 'The model '+new+suffix, color=color)
23
+ phrase = f'\x1b[31;9m{old}\x1b[0m\x1b[32m{new}\x1b[0m' if color else f'[-{old}-]{{+{new}+}}'
24
+ assert phrase in result and suffix.rstrip() in result
25
+ shared = ' while both points retain their positions '
26
+ result = render_source_diff('We select fields'+shared+'using the old formula.\n',
27
+ 'We reverse coordinates'+shared+'using the new expression.\n')
28
+ assert '[-select fields-]{+reverse coordinates+}' in result and shared in result
29
+ assert '[-old formula-]{+new expression+}' in result
30
+
31
+
32
+ def test_long_line_changes():
33
+ prefix = ' '.join(f'word{i}' for i in range(80))+' '
34
+ tail = ' '.join(f'tail{i}' for i in range(80))
35
+ old, new = prefix+'green '+tail+' old\n', prefix+'blue '+tail+' new\n'
36
+ for color in (False, True):
37
+ result = render_source_diff(old, new, maxlen=80, color=color)
38
+ visible = re.sub(r'\x1b\[[\d;]*m', '', result)
39
+ assert all(len(line)<=80 for line in visible.splitlines())
40
+ assert 'green' in result and 'blue' in result and 'word0 ' not in result
41
+ assert '…[' in result and ' new' not in result
42
+ if color: assert '\x1b[31;9mgreen\x1b[0m' in result and '\x1b[32mblue\x1b[0m' in result
43
+ else: assert '[-green-]{+blue+}' in result
44
+ full = render_source_diff(old, new, maxlen=0)
45
+ assert 'word0 ' in full and '[-old-]{+new+}' in full
46
+ before,after = 'a'*22,'b'*22
47
+ result = render_source_diff(prefix+before+' '+tail, prefix+after+' '+tail, maxlen=80)
48
+ assert f'[-{before}-]{{+{after}+}}' in result
49
+
50
+
51
+ def test_final_newline():
52
+ assert r'' in source_diff('hello\n', 'hello')
53
+ assert 'new file' in render_source_diff('hello\n', 'hello')
54
+ assert 'old file' in render_source_diff('hello', 'hello\n')
55
+ assert not render_source_diff('hello\n', 'hello', final_newline=False)
56
+
57
+
58
+ def test_git_driver(tmp_path):
59
+ env = {**os.environ, 'GIT_CONFIG_GLOBAL': os.devnull, 'GIT_CONFIG_NOSYSTEM': '1'}
60
+ def git(*args):
61
+ return subprocess.run(['git', *args], cwd=tmp_path, env=env, capture_output=True, text=True, check=True).stdout
62
+ driver = tmp_path/'driver.py'
63
+ driver.write_text('from nbdev.diff import nbdev_diff_driver\nnbdev_diff_driver()')
64
+ git('init', '-q')
65
+ git('config', 'user.name', 'nbdev')
66
+ git('config', 'user.email', 'nbdev@fast.ai')
67
+ git('config', 'diff.nbdev.command', shlex.join([sys.executable, str(driver), '--maxlen', '80']))
68
+ (tmp_path/'.gitattributes').write_text('*.md diff=nbdev\n*.py diff=nbdev\n*.ipynb diff=nbdev\n')
69
+ note, source = tmp_path/'note.md', tmp_path/'source.py'
70
+ prefix = 'A notebook has words. '*20
71
+ note.write_text(prefix+'This is a green notebook.\n')
72
+ source.write_text('#| export\nx=1\n')
73
+ nb_path = tmp_path/'demo.ipynb'
74
+ nb = new_nb(['#| export\nx=1'])
75
+ write_nb(nb, nb_path)
76
+ git('add', '.')
77
+ git('commit', '-qm', 'initial')
78
+ note.write_text(prefix+'This is a blue notebook.\n')
79
+ source.write_text('#| exports\nx=100\n')
80
+ nb.cells[0].source = '#| export\nx=100'
81
+ write_nb(nb, nb_path)
82
+ result = git('diff')
83
+ assert '[-green-]{+blue+}' in result
84
+ assert all(len(line)<=80 for line in result.splitlines() if line.startswith('!'))
85
+ assert '-#| export\n+#| exports' in result and '## modified' in result and '[export]' in result
86
+ assert '[-1-]{+100+}' in result
87
+ git('add', '.')
88
+ git('commit', '-qm', 'edited')
89
+ git('mv', 'note.md', 'renamed.md')
90
+ git('update-index', '--chmod=+x', 'source.py')
91
+ result = git('diff', '--cached', '--find-renames')
92
+ assert 'note.md' in result and 'renamed.md' in result
93
+ assert '100644' in result and '100755' in result
@@ -1,63 +0,0 @@
1
- """Write, read and review nbdev notebooks as source, documentation, examples and tests. You MUST read this before any work on a project with notebooks.
2
-
3
- ## Source and reading
4
-
5
- Published-library notebooks: interleave implementation, explanation, examples, outputs and failure demonstrations for top-to-bottom reading. Internal projects: match surrounding notebooks.
6
-
7
- Check each file, not the repo: `# AUTOGENERATED! DO NOT EDIT!` names the source notebook; `# %% ../nbs/04_usage.ipynb #a45f753a` identifies its source cell. Edit that notebook, never generated modules, `__all__`, `_modidx.py`, or module docstrings; `nbdev-export` regenerates them. Unmarked modules are hand-written. Understand and review work through notebooks and their `git diff`, never generated modules: those lack prose, examples and tests.
8
-
9
- Use notebooks to teach APIs through narrative/examples, plain `.py` for dense mechanism iterated under pytest (rarely; never add pytests to a project without them unless asked). Lessons test notebook code; `tests/*.py` test plain modules. For project-level questions, read `nbs/index.ipynb` (README.md's source); regenerate with `nbdev-readme` after changes.
10
-
11
- ## Authoring
12
-
13
- Before notebook edits, read the whole notebook or a summary of every cell. For a new notebook, read a role model such as claudette's `00_core.ipynb` to learn the literate form. Choose tutorial, how-to, explanation, or reference. Open with an H1 title cell, `>` subtitle and brief introduction. Use H2s for major concepts without fragmenting short narratives; each heading gets its own markdown cell for Jupyter collapse.
14
-
15
- Develop one idea at a time: small implementation, explanation, lesson for every substantive API. Basic functionality outside the API, with little logic, generally needs no lesson. Introduce lessons in markdown; split when the introduction becomes complicated. Extend examples rather than adding near-duplicates. Build classes with `@patch` beside each method's explanation unless that obscures the API.
16
-
17
- When new behavior needs teaching (often unnecessary for trivial logic), choose its narrative position, write the lesson first (better, slightly modify an existing lesson), run preceding cells as needed, see it fail. Add its markdown introduction and implementation above it; run implementation and lesson to see it pass. For bugs, revise an existing example to exercise the failing path; if no revision fits, add an assertion to it. No documented behavior change: change no lesson cells. Never add a cell merely to witness a fix.
18
-
19
- Read existing examples before experimenting. Explore in the notebook, not a separate kernel check; keep useful explorations as lessons, delete the rest. Always try to leave the notebook better. Reserve underscore names for machinery without a coherent independent contract, not everything used internally. A helper useful only internally often signals awkward design.
20
-
21
- ## Lessons
22
-
23
- Readability comes first: guide readers through what code does using code and displayed output. If a displayed value is worth testing, assign, test, then show it. Test the point of the code, not incidentals such as formatting. Show errors only when readers are likely to hit them.
24
-
25
- Use realistic values, minimal setup, reused objects and direct supporting assertions. Show evidence with compact `_repr_markdown_`, structured summaries, plots, tables, images or HTML; explain what matters in large outputs and keep stored results focused. Use `fastcore.test` assertions and `expect_fail` in ordinary code cells.
26
-
27
- Test-serving cells don't belong on the page: signs include safety-only setup and glue checks. Usually remove them; move substantive logic checks to `#| hide` or pytest. Discuss such changes carefully. Keep plumbing, mocks, dense comprehensions and long setup out of reader-facing examples: extract a small helper or hide the check.
28
-
29
- Consider these rewrites:
30
- - Commented examples -> prose/code; numbered steps -> markdown/code pairs. Keep comments stating constraints code cannot show.
31
- - Sentence-wrapped `print` -> prose/displayed value; final `print(x)` -> bare `x`.
32
- - Reassignment stages and blank-line groups -> separate cells, each introduced.
33
- - Near-duplicates: explain the differing argument; don't repeat setup.
34
- - Errors: `expect_fail`, not `try`/`except`. Long docstrings -> markdown.
35
- - Resolve TODO/FIXME example comments or move them to issues.
36
-
37
- ## Prose and exports
38
-
39
- In markdown, explain rationale, distinctions and guarantees, not code line-by-line or filler transitions. Place it for correct generated-page reading, normally after definitions. Use short symbol docstrings, parameter/return docments beside signatures and backticked symbols for nbdev links. Extended explanation, examples and warnings belong in markdown.
40
-
41
- `nbdev-export` assembles module docstrings from the title's `>` description and exported markdown after H1; use one exported markdown cell per major section for long docstrings. `doc(module)` lists functions; in module prose, explain only shared constraints/workflows.
42
-
43
- `#| exportd` includes a code cell as fenced docstring source, not module code. Use compact runnable demos after required definitions; end the preceding exported summary with a colon. Intervening unexported definitions don't appear in the docstring. Check claims by running code before documenting; preview assembly with `nbdev.export.nb_mdoc`. A docstring beyond the default summary makes `nbdev-export` list the module in package docs and `llms.txt`. Only extend docstrings of modules read independently. Export `index.ipynb`'s opening paragraphs for the package intro.
44
-
45
- ## State and execution
46
-
47
- Define values near first use, introduce shared setup, reuse established objects; don't reassign names later cells depend on. End explorations with the expression recording what was learned when useful. Keep imports in dedicated cells, lessons included: docs builds execute import-containing cells in fresh namespaces; mixed state-dependent code can fail or run at documentation time.
48
-
49
- Outputs are generated and may be stale during edits. Don't stop work, clear them manually, or edit notebook JSON because of source/output mismatch. At PR time, regenerate with `nbdev-test --save` if the project keeps saved outputs. `nbdev-test --file-re` runs several named notebooks in parallel.
50
-
51
- ## Directives and nbdev v3
52
-
53
- - `#| default_exp` names the generated module; `#| export` includes a cell there. Exported underscore helpers need not be public.
54
- - `#| hide` hides distracting necessities from the page.
55
- - `#| eval: false`: code that must not run, not broken code. Later cells can't use its state.
56
- - Metadata directives work too: `{"nbdev": {"export": "true"}}`; `fastcore.nbio.mk_cell(...).directives` merges both forms, content winning conflicts.
57
-
58
- nbdev v3 (Jan 2026): `pyproject.toml`, not `settings.ini`; metadata in `[project]`, settings in `[tool.nbdev]` (defaults `nbs_path='nbs'`, `doc_path='_docs'`), version in `__init__.py` via `dynamic = ["version"]`, `_modidx` in `[project.entry-points.nbdev]`. CLIs use hyphens (`nbdev-export`, `nbdev-test`); Python names retain underscores. GitHub actions: `fastai/workflows/nbdev3-ci`, `quarto-ghp3`.
59
-
60
- ## Finish
61
-
62
- Revise affected prose, displays and assertions together; changed assertions must fail on old code and pass on new. Re-read touched narrative for readability. Run `nbdev-test <nb>`.
63
- """
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes
File without changes