nnsight 0.5.0.dev0__tar.gz → 0.5.0.dev1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/PKG-INFO +1 -1
- nnsight-0.5.0.dev1/src/nnsight/__init__.py +150 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/envoy.py +83 -4
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/globals.py +14 -1
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight.egg-info/PKG-INFO +1 -1
- nnsight-0.5.0.dev0/src/nnsight/__init__.py +0 -68
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/.github/workflows/publish.yml +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/.github/workflows/python-app.yml +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/.gitignore +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/CHANGELOG.md +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/CODE_OF_CONDUCT.md +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/LICENSE +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/NNsight_Walkthrough.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/NNsight_v0_3_guide.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/NNsight_v0_4_guide.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/README.md +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/conftest.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/Makefile +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/contributing.md +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/make.bat +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/requirements.txt +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/css/custom.css +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/css/home.css +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/css/pygments.css +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/css/status.css +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/execution.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/icon.ico +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/interleaved.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/ioi.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/nnsight_logo.svg +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/activation_patching.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/attribution_patching.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/dictionary_learning.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/function_vectors.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/future_lens.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/logit_lens.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/walkthrough.webp +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/js/.DS_Store +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/js/code.js +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/js/custom.js +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_templates/layout.html +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_templates/ndif_status.html +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/about.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/conf.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation/intervention.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation/modeling.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation/schema.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation/tracing.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation/util.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/documentation.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/features.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/index.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/conditionals.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/cross_prompt.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/custom_functions.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/early_stopping.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/editing_tutorial.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/getting.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/gradients.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/iterator.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/lora_training.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/model_editing.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/modules.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/multiple_token.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/operations.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/remote_execution.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/scan_validate.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/sessions.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/setting.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/streaming.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/vllm_support.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/LoRA_tutorial.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/activation_patching.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/attribution_patching.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/boundless_DAS.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/dict_learning.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/diffusion_lens.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/images/Activation_patching-figure1.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/images/LoRA_tutorial_figure_1.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/images/LoRA_tutorial_figure_2.png +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/logit_lens.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/start_remote_access.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/tutorial_price_tagging_utils.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/walkthrough.ipynb +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/pages/home.html +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/start.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/status.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/tutorials.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/conf.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/contexts.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/intervention.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/models.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/module.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/patching.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/tracing.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/documentation/util.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/sourcelatex/index.rst +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/pyproject.toml +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/pytest.ini +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/setup.cfg +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/setup.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/_c/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/_c/py_mount.c +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/config.yaml +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/backends/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/backends/base.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/backends/editing.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/backends/execution.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/backends/remote.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/batching.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/inject.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/interleaver.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/backwards.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/base.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/editing.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/invoker.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/iterator.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/tracer.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/intervention/tracing/util.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/log.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/base.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/diffusion.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/language.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/mixins/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/mixins/loadable.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/mixins/meta.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/mixins/remoteable.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/model_runners/GPUModelRunner.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/model_runners/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/sampling.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/vllm.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/workers/GPUWorker.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/workers/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/schema/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/schema/config.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/schema/request.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/schema/response.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/util.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight.egg-info/SOURCES.txt +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight.egg-info/dependency_links.txt +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight.egg-info/requires.txt +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight.egg-info/top_level.txt +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/tests/__init__.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/tests/test_diffusion.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/tests/test_lm.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/tests/test_tiny.py +0 -0
- {nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/tests/test_vllm.py +0 -0
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
from functools import wraps
|
|
2
|
+
import os, yaml
|
|
3
|
+
import warnings
|
|
4
|
+
from typing import Optional
|
|
5
|
+
from .schema.config import ConfigModel
|
|
6
|
+
|
|
7
|
+
PATH = os.path.dirname(os.path.abspath(__file__))
|
|
8
|
+
with open(os.path.join(PATH, "config.yaml"), "r") as file:
|
|
9
|
+
CONFIG = ConfigModel(**yaml.safe_load(file))
|
|
10
|
+
|
|
11
|
+
from importlib.metadata import PackageNotFoundError, version
|
|
12
|
+
|
|
13
|
+
try:
|
|
14
|
+
__version__ = version("nnsight")
|
|
15
|
+
except PackageNotFoundError:
|
|
16
|
+
__version__ = "unknown version"
|
|
17
|
+
|
|
18
|
+
from IPython import get_ipython
|
|
19
|
+
|
|
20
|
+
try:
|
|
21
|
+
__IPYTHON__ = get_ipython() is not None
|
|
22
|
+
except NameError:
|
|
23
|
+
__IPYTHON__ = False
|
|
24
|
+
|
|
25
|
+
base_deprecation_message = "is deprecated as of v0.5.0 and will be removed in a future version."
|
|
26
|
+
|
|
27
|
+
def deprecated(message:Optional[str]=None, error:bool=False):
|
|
28
|
+
|
|
29
|
+
def decorator(func):
|
|
30
|
+
|
|
31
|
+
@wraps(func)
|
|
32
|
+
def inner(*args, **kwargs):
|
|
33
|
+
|
|
34
|
+
deprecation_message = (
|
|
35
|
+
f"{func.__module__}.{func.__name__} {base_deprecation_message}"
|
|
36
|
+
+ (f"\n{message}" if message is not None else "")
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
if error:
|
|
40
|
+
raise DeprecationWarning(deprecation_message)
|
|
41
|
+
else:
|
|
42
|
+
warnings.warn(deprecation_message)
|
|
43
|
+
|
|
44
|
+
return func(*args, **kwargs)
|
|
45
|
+
|
|
46
|
+
return inner
|
|
47
|
+
|
|
48
|
+
return decorator
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
|
|
52
|
+
|
|
53
|
+
from .intervention.envoy import Envoy
|
|
54
|
+
from .modeling.base import NNsight
|
|
55
|
+
from .modeling.language import LanguageModel
|
|
56
|
+
from .intervention.tracing.base import Tracer
|
|
57
|
+
|
|
58
|
+
|
|
59
|
+
|
|
60
|
+
def session(*args, **kwargs):
|
|
61
|
+
return Tracer(*args, **kwargs)
|
|
62
|
+
|
|
63
|
+
from .util import Patcher, Patch
|
|
64
|
+
|
|
65
|
+
DEFAULT_PATCHER = Patcher()
|
|
66
|
+
|
|
67
|
+
# Tensor creation operations
|
|
68
|
+
from torch._subclasses.fake_tensor import FakeTensor
|
|
69
|
+
|
|
70
|
+
|
|
71
|
+
def fake_bool(self):
|
|
72
|
+
return True
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
DEFAULT_PATCHER.add(Patch(FakeTensor, fake_bool, "__bool__"))
|
|
76
|
+
|
|
77
|
+
from torch.amp.autocast_mode import autocast
|
|
78
|
+
|
|
79
|
+
def wrap_autocast(func):
|
|
80
|
+
|
|
81
|
+
@wraps(func)
|
|
82
|
+
def inner(self, device_type:str, *args, **kwargs):
|
|
83
|
+
|
|
84
|
+
if device_type == "meta":
|
|
85
|
+
device_type = "cpu"
|
|
86
|
+
|
|
87
|
+
return func(self, device_type, *args, **kwargs)
|
|
88
|
+
|
|
89
|
+
return inner
|
|
90
|
+
|
|
91
|
+
|
|
92
|
+
DEFAULT_PATCHER.add(Patch(autocast, wrap_autocast(autocast.__init__), "__init__"))
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
DEFAULT_PATCHER.__enter__()
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
## TODO: Legacy
|
|
99
|
+
|
|
100
|
+
@deprecated()
|
|
101
|
+
def apply(fn, *args, **kwargs):
|
|
102
|
+
|
|
103
|
+
return fn(*args, **kwargs)
|
|
104
|
+
|
|
105
|
+
@deprecated()
|
|
106
|
+
def log(message:str):
|
|
107
|
+
|
|
108
|
+
print(message)
|
|
109
|
+
|
|
110
|
+
@deprecated(error=True)
|
|
111
|
+
def local(*args, **kwargs):
|
|
112
|
+
pass
|
|
113
|
+
|
|
114
|
+
@deprecated(error=True)
|
|
115
|
+
def cond(*args, **kwargs):
|
|
116
|
+
pass
|
|
117
|
+
|
|
118
|
+
@deprecated(error=True)
|
|
119
|
+
def iter(*args, **kwargs):
|
|
120
|
+
pass
|
|
121
|
+
|
|
122
|
+
from .intervention.interleaver import EarlyStopException
|
|
123
|
+
|
|
124
|
+
@deprecated()
|
|
125
|
+
def stop():
|
|
126
|
+
raise EarlyStopException()
|
|
127
|
+
|
|
128
|
+
@deprecated()
|
|
129
|
+
def trace(fn):
|
|
130
|
+
|
|
131
|
+
return fn
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
bool = deprecated(message="Use the standard `bool()` instead.")(bool)
|
|
135
|
+
bytes = deprecated(message="Use the standard `bytes()` instead.")(bytes)
|
|
136
|
+
complex = deprecated(message="Use the standard `complex()` instead.")(complex)
|
|
137
|
+
dict = deprecated(message="Use the standard `dict()` instead.")(dict)
|
|
138
|
+
float = deprecated(message="Use the standard `float()` instead.")(float)
|
|
139
|
+
int = deprecated(message="Use the standard `int()` instead.")(int)
|
|
140
|
+
list = deprecated(message="Use the standard `list()` instead.")(list)
|
|
141
|
+
str = deprecated(message="Use the standard `str()` instead.")(str)
|
|
142
|
+
tuple = deprecated(message="Use the standard `tuple()` instead.")(tuple)
|
|
143
|
+
bytearray = deprecated(message="Use the standard `bytearray()` instead.")(bytearray)
|
|
144
|
+
|
|
145
|
+
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+
|
|
150
|
+
|
|
@@ -2,6 +2,7 @@ from __future__ import annotations
|
|
|
2
2
|
|
|
3
3
|
import inspect
|
|
4
4
|
import os
|
|
5
|
+
import warnings
|
|
5
6
|
from types import (BuiltinFunctionType, BuiltinMethodType, FunctionType,
|
|
6
7
|
MethodType)
|
|
7
8
|
from typing import (TYPE_CHECKING, Any, Callable, Dict, List, Optional, Tuple,
|
|
@@ -9,8 +10,9 @@ from typing import (TYPE_CHECKING, Any, Callable, Dict, List, Optional, Tuple,
|
|
|
9
10
|
|
|
10
11
|
import dill
|
|
11
12
|
import torch
|
|
13
|
+
from torch.nn.modules.module import _addindent
|
|
12
14
|
|
|
13
|
-
from .. import CONFIG, util
|
|
15
|
+
from .. import CONFIG, base_deprecation_message, deprecated, util
|
|
14
16
|
from ..util import apply, fetch_attr
|
|
15
17
|
from .batching import Batchable
|
|
16
18
|
from .inject import convert as inject
|
|
@@ -390,6 +392,9 @@ class Envoy(Batchable):
|
|
|
390
392
|
"""
|
|
391
393
|
|
|
392
394
|
#TODO trace= is Legacy
|
|
395
|
+
if trace is not None:
|
|
396
|
+
deprecation_message = f"The `trace` argument {base_deprecation_message}\nJust call the method without a with context instead."
|
|
397
|
+
warnings.warn(deprecation_message)
|
|
393
398
|
|
|
394
399
|
if fn is None:
|
|
395
400
|
fn = self.__call__
|
|
@@ -525,10 +530,12 @@ class Envoy(Batchable):
|
|
|
525
530
|
|
|
526
531
|
# TODO legacy
|
|
527
532
|
@property
|
|
533
|
+
@deprecated(message="Use `tracer.iter` instead.")
|
|
528
534
|
def iter(self):
|
|
529
535
|
return IteratorProxy(self._interleaver)
|
|
530
536
|
|
|
531
537
|
# TODO legacy
|
|
538
|
+
@deprecated(message="Use `tracer.all()` instead.")
|
|
532
539
|
def all(self):
|
|
533
540
|
return self.iter[:]
|
|
534
541
|
|
|
@@ -794,6 +801,20 @@ class Envoy(Batchable):
|
|
|
794
801
|
return copy
|
|
795
802
|
|
|
796
803
|
#### Dunder methods ####
|
|
804
|
+
|
|
805
|
+
|
|
806
|
+
def __len__(self):
|
|
807
|
+
"""
|
|
808
|
+
Get the length of the Envoy.
|
|
809
|
+
"""
|
|
810
|
+
return len(self._module)
|
|
811
|
+
|
|
812
|
+
def __iter__(self):
|
|
813
|
+
"""
|
|
814
|
+
Iterate over the Envoy.
|
|
815
|
+
"""
|
|
816
|
+
return iter(self._children)
|
|
817
|
+
|
|
797
818
|
|
|
798
819
|
def __str__(self):
|
|
799
820
|
"""
|
|
@@ -802,8 +823,39 @@ class Envoy(Batchable):
|
|
|
802
823
|
Returns:
|
|
803
824
|
A string representation of the Envoy showing its path
|
|
804
825
|
"""
|
|
805
|
-
|
|
806
|
-
|
|
826
|
+
return self.__repr__()
|
|
827
|
+
|
|
828
|
+
def __reprlist__(self):
|
|
829
|
+
|
|
830
|
+
list_of_reprs = [repr(item) for item in self]
|
|
831
|
+
if len(list_of_reprs) == 0:
|
|
832
|
+
return self._module._get_name() + "()"
|
|
833
|
+
|
|
834
|
+
start_end_indices = [[0, 0]]
|
|
835
|
+
repeated_blocks = [list_of_reprs[0]]
|
|
836
|
+
for i, r in enumerate(list_of_reprs[1:], 1):
|
|
837
|
+
if r == repeated_blocks[-1]:
|
|
838
|
+
start_end_indices[-1][1] += 1
|
|
839
|
+
continue
|
|
840
|
+
|
|
841
|
+
start_end_indices.append([i, i])
|
|
842
|
+
repeated_blocks.append(r)
|
|
843
|
+
|
|
844
|
+
lines = []
|
|
845
|
+
main_str = self._module._get_name() + "("
|
|
846
|
+
for (start_id, end_id), b in zip(start_end_indices, repeated_blocks):
|
|
847
|
+
local_repr = f"({start_id}): {b}" # default repr
|
|
848
|
+
|
|
849
|
+
if start_id != end_id:
|
|
850
|
+
n = end_id - start_id + 1
|
|
851
|
+
local_repr = f"({start_id}-{end_id}): {n} x {b}"
|
|
852
|
+
|
|
853
|
+
local_repr = _addindent(local_repr, 2)
|
|
854
|
+
lines.append(local_repr)
|
|
855
|
+
|
|
856
|
+
main_str += "\n " + "\n ".join(lines) + "\n"
|
|
857
|
+
main_str += ")"
|
|
858
|
+
return main_str
|
|
807
859
|
|
|
808
860
|
def __repr__(self):
|
|
809
861
|
"""
|
|
@@ -812,7 +864,34 @@ class Envoy(Batchable):
|
|
|
812
864
|
Returns:
|
|
813
865
|
The string representation of the Envoy
|
|
814
866
|
"""
|
|
815
|
-
|
|
867
|
+
|
|
868
|
+
if isinstance(self._module, torch.nn.ModuleList):
|
|
869
|
+
return self.__reprlist__()
|
|
870
|
+
|
|
871
|
+
# We treat the extra repr like the sub-module, one item per line
|
|
872
|
+
extra_lines = []
|
|
873
|
+
extra_repr = self._module.extra_repr()
|
|
874
|
+
# empty string will be split into list ['']
|
|
875
|
+
if extra_repr:
|
|
876
|
+
extra_lines = extra_repr.split("\n")
|
|
877
|
+
child_lines = []
|
|
878
|
+
for envoy in self._children:
|
|
879
|
+
key = envoy.path.split(".")[-1]
|
|
880
|
+
mod_str = repr(envoy)
|
|
881
|
+
mod_str = _addindent(mod_str, 2)
|
|
882
|
+
child_lines.append("(" + key + "): " + mod_str)
|
|
883
|
+
lines = extra_lines + child_lines
|
|
884
|
+
|
|
885
|
+
main_str = self._module._get_name() + "("
|
|
886
|
+
if lines:
|
|
887
|
+
# simple one-liner info, which most builtin Modules will use
|
|
888
|
+
if len(extra_lines) == 1 and not child_lines:
|
|
889
|
+
main_str += extra_lines[0]
|
|
890
|
+
else:
|
|
891
|
+
main_str += "\n " + "\n ".join(lines) + "\n"
|
|
892
|
+
|
|
893
|
+
main_str += ")"
|
|
894
|
+
return main_str
|
|
816
895
|
|
|
817
896
|
def __getattr__(self, name: str) -> Union[torch.nn.Module, Envoy, Any]:
|
|
818
897
|
"""
|
|
@@ -2,7 +2,7 @@ from typing import Any, Callable, Union
|
|
|
2
2
|
|
|
3
3
|
import torch
|
|
4
4
|
from typing_extensions import Self
|
|
5
|
-
|
|
5
|
+
from ... import deprecated
|
|
6
6
|
from ..._c.py_mount import mount, unmount
|
|
7
7
|
|
|
8
8
|
|
|
@@ -24,6 +24,17 @@ class Object(torch.Tensor):
|
|
|
24
24
|
|
|
25
25
|
return self
|
|
26
26
|
|
|
27
|
+
@deprecated(message="Use `tracer.stop()` instead.")
|
|
28
|
+
def stop(self, _=0):
|
|
29
|
+
"""
|
|
30
|
+
Stop the trace context.
|
|
31
|
+
"""
|
|
32
|
+
|
|
33
|
+
from ..interleaver import EarlyStopException
|
|
34
|
+
|
|
35
|
+
raise EarlyStopException()
|
|
36
|
+
|
|
37
|
+
|
|
27
38
|
def __getattr__(self, name: str) -> Self:
|
|
28
39
|
|
|
29
40
|
return super().__getattr__(name)
|
|
@@ -47,6 +58,7 @@ class Globals:
|
|
|
47
58
|
def enter():
|
|
48
59
|
if Globals.stack == 0:
|
|
49
60
|
mount(Object.save, "save")
|
|
61
|
+
mount(Object.stop, "stop")
|
|
50
62
|
Globals.stack += 1
|
|
51
63
|
|
|
52
64
|
@staticmethod
|
|
@@ -54,3 +66,4 @@ class Globals:
|
|
|
54
66
|
Globals.stack -= 1
|
|
55
67
|
if Globals.stack == 0:
|
|
56
68
|
unmount("save")
|
|
69
|
+
unmount("stop")
|
|
@@ -1,68 +0,0 @@
|
|
|
1
|
-
from functools import wraps
|
|
2
|
-
import os, yaml
|
|
3
|
-
from .schema.config import ConfigModel
|
|
4
|
-
|
|
5
|
-
PATH = os.path.dirname(os.path.abspath(__file__))
|
|
6
|
-
with open(os.path.join(PATH, "config.yaml"), "r") as file:
|
|
7
|
-
CONFIG = ConfigModel(**yaml.safe_load(file))
|
|
8
|
-
|
|
9
|
-
from importlib.metadata import PackageNotFoundError, version
|
|
10
|
-
|
|
11
|
-
try:
|
|
12
|
-
__version__ = version("nnsight")
|
|
13
|
-
except PackageNotFoundError:
|
|
14
|
-
__version__ = "unknown version"
|
|
15
|
-
|
|
16
|
-
from IPython import get_ipython
|
|
17
|
-
|
|
18
|
-
try:
|
|
19
|
-
__IPYTHON__ = get_ipython() is not None
|
|
20
|
-
except NameError:
|
|
21
|
-
__IPYTHON__ = False
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
from .intervention.envoy import Envoy
|
|
25
|
-
from .modeling.base import NNsight
|
|
26
|
-
from .modeling.language import LanguageModel
|
|
27
|
-
from .intervention.tracing.base import Tracer
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
def session(*args, **kwargs):
|
|
32
|
-
return Tracer(*args, **kwargs)
|
|
33
|
-
|
|
34
|
-
from .util import Patcher, Patch
|
|
35
|
-
|
|
36
|
-
#TODO legacy stuff like nnsight.list
|
|
37
|
-
|
|
38
|
-
DEFAULT_PATCHER = Patcher()
|
|
39
|
-
|
|
40
|
-
# Tensor creation operations
|
|
41
|
-
from torch._subclasses.fake_tensor import FakeTensor
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
def fake_bool(self):
|
|
45
|
-
return True
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
DEFAULT_PATCHER.add(Patch(FakeTensor, fake_bool, "__bool__"))
|
|
49
|
-
|
|
50
|
-
from torch.amp.autocast_mode import autocast
|
|
51
|
-
|
|
52
|
-
def wrap_autocast(func):
|
|
53
|
-
|
|
54
|
-
@wraps(func)
|
|
55
|
-
def inner(self, device_type:str, *args, **kwargs):
|
|
56
|
-
|
|
57
|
-
if device_type == "meta":
|
|
58
|
-
device_type = "cpu"
|
|
59
|
-
|
|
60
|
-
return func(self, device_type, *args, **kwargs)
|
|
61
|
-
|
|
62
|
-
return inner
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
DEFAULT_PATCHER.add(Patch(autocast, wrap_autocast(autocast.__init__), "__init__"))
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
DEFAULT_PATCHER.__enter__()
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/function_vectors.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/future_lens.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/logit_lens.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/_static/images/tutorials/walkthrough.webp
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/custom_functions.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/early_stopping.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/editing_tutorial.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/lora_training.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/model_editing.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/multiple_token.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/remote_execution.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/features/scan_validate.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/LoRA_tutorial.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/activation_patching.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/attribution_patching.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/boundless_DAS.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/dict_learning.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/diffusion_lens.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/docs/source/notebooks/tutorials/start_remote_access.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/model_runners/GPUModelRunner.py
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev0 → nnsight-0.5.0.dev1}/src/nnsight/modeling/vllm/model_runners/__init__.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|