nnsight 0.5.0.dev4__tar.gz → 0.5.0.dev6__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/PKG-INFO +1 -1
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/__init__.py +29 -6
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/envoy.py +68 -5
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/interleaver.py +3 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/base.py +47 -30
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight.egg-info/PKG-INFO +1 -1
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/.github/workflows/publish.yml +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/.github/workflows/python-app.yml +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/.gitignore +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/CHANGELOG.md +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/CODE_OF_CONDUCT.md +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/LICENSE +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/NNsight_Walkthrough.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/NNsight_v0_3_guide.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/NNsight_v0_4_guide.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/README.md +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/conftest.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/Makefile +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/contributing.md +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/make.bat +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/requirements.txt +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/css/custom.css +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/css/home.css +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/css/pygments.css +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/css/status.css +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/execution.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/icon.ico +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/interleaved.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/ioi.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/nnsight_logo.svg +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/activation_patching.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/attribution_patching.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/dictionary_learning.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/function_vectors.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/future_lens.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/logit_lens.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/walkthrough.webp +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/js/.DS_Store +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/js/code.js +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/js/custom.js +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_templates/layout.html +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_templates/ndif_status.html +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/about.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/conf.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation/intervention.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation/modeling.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation/schema.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation/tracing.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation/util.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/documentation.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/features.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/index.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/conditionals.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/cross_prompt.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/custom_functions.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/early_stopping.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/editing_tutorial.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/getting.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/gradients.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/iterator.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/lora_training.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/model_editing.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/modules.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/multiple_token.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/operations.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/remote_execution.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/scan_validate.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/sessions.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/setting.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/streaming.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/vllm_support.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/LoRA_tutorial.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/activation_patching.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/attribution_patching.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/boundless_DAS.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/dict_learning.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/diffusion_lens.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/images/Activation_patching-figure1.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/images/LoRA_tutorial_figure_1.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/images/LoRA_tutorial_figure_2.png +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/logit_lens.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/start_remote_access.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/tutorial_price_tagging_utils.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/walkthrough.ipynb +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/pages/home.html +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/start.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/status.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/tutorials.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/conf.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/contexts.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/intervention.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/models.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/module.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/patching.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/tracing.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/documentation/util.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/sourcelatex/index.rst +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/pyproject.toml +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/pytest.ini +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/setup.cfg +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/setup.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/_c/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/_c/py_mount.c +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/config.yaml +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/backends/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/backends/base.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/backends/editing.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/backends/execution.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/backends/remote.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/batching.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/inject.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/serialization.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/backwards.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/editing.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/globals.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/invoker.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/iterator.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/tracer.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/intervention/tracing/util.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/log.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/base.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/diffusion.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/language.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/mixins/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/mixins/loadable.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/mixins/meta.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/mixins/remoteable.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/model_runners/GPUModelRunner.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/model_runners/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/sampling.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/vllm.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/workers/GPUWorker.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/workers/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/schema/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/schema/config.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/schema/request.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/schema/response.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/util.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight.egg-info/SOURCES.txt +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight.egg-info/dependency_links.txt +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight.egg-info/requires.txt +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight.egg-info/top_level.txt +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/tests/__init__.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/tests/test_diffusion.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/tests/test_lm.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/tests/test_tiny.py +0 -0
- {nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/tests/test_vllm.py +0 -0
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
from functools import wraps
|
|
2
|
-
import
|
|
2
|
+
import inspect
|
|
3
|
+
import os, yaml, sys
|
|
3
4
|
import warnings
|
|
4
5
|
from typing import Optional
|
|
5
6
|
from .schema.config import ConfigModel
|
|
@@ -22,6 +23,8 @@ try:
|
|
|
22
23
|
except NameError:
|
|
23
24
|
__IPYTHON__ = False
|
|
24
25
|
|
|
26
|
+
__INTERACTIVE__ = (sys.flags.interactive or not sys.argv[0]) and not __IPYTHON__
|
|
27
|
+
|
|
25
28
|
base_deprecation_message = "is deprecated as of v0.5.0 and will be removed in a future version."
|
|
26
29
|
|
|
27
30
|
def deprecated(message:Optional[str]=None, error:bool=False):
|
|
@@ -142,9 +145,29 @@ str = deprecated(message="Use the standard `str()` instead.")(str)
|
|
|
142
145
|
tuple = deprecated(message="Use the standard `tuple()` instead.")(tuple)
|
|
143
146
|
bytearray = deprecated(message="Use the standard `bytearray()` instead.")(bytearray)
|
|
144
147
|
|
|
148
|
+
if __INTERACTIVE__:
|
|
149
|
+
from code import InteractiveConsole
|
|
150
|
+
import readline
|
|
151
|
+
|
|
152
|
+
# We need to use our own console so when we trace, we can get the source code from the buffer of our custom console.
|
|
153
|
+
class NNsightConsole(InteractiveConsole):
|
|
154
|
+
pass
|
|
155
|
+
|
|
156
|
+
# We want the new console to have the locals of the interactive frame the user is already in.
|
|
157
|
+
frame = inspect.currentframe()
|
|
158
|
+
|
|
159
|
+
while frame.f_back:
|
|
160
|
+
frame = frame.f_back
|
|
161
|
+
|
|
162
|
+
# This was kicked off by importing nnsight, but upon entering the new console, we wont actually have the import in the new locals.
|
|
163
|
+
# So we need to get the last command from the history and execute it in the new locals.
|
|
164
|
+
length = readline.get_current_history_length()
|
|
165
|
+
l1 = readline.get_history_item(length)
|
|
166
|
+
ilocals = {**frame.f_locals}
|
|
167
|
+
exec(l1, frame.f_globals, ilocals)
|
|
168
|
+
|
|
169
|
+
__INTERACTIVE_CONSOLE__ = NNsightConsole(filename="<nnsight-console>", locals=ilocals)
|
|
170
|
+
__INTERACTIVE_CONSOLE__.interact(banner="")
|
|
145
171
|
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
172
|
+
else:
|
|
173
|
+
__INTERACTIVE_CONSOLE__ = None
|
|
@@ -188,6 +188,7 @@ class Envoy(Batchable):
|
|
|
188
188
|
The module's input values as a tuple of positional and keyword arguments. i.e (args, kwargs)
|
|
189
189
|
|
|
190
190
|
"""
|
|
191
|
+
|
|
191
192
|
if self.interleaving:
|
|
192
193
|
return self._interleaver.current.request(
|
|
193
194
|
self._interleaver.current.iterate(f"{self.path}.input")
|
|
@@ -734,11 +735,60 @@ class Envoy(Batchable):
|
|
|
734
735
|
|
|
735
736
|
# If the module already has a sub-module named 'input' or 'output',
|
|
736
737
|
# mount the proxy access to 'nns_input' or 'nns_output instead.
|
|
738
|
+
|
|
737
739
|
if hasattr(Envoy, name):
|
|
738
740
|
self._handle_overloaded_mount(envoy, name)
|
|
739
741
|
else:
|
|
740
742
|
super().__setattr__(name, envoy)
|
|
741
743
|
|
|
744
|
+
def _handle_overloaded_mount(self, envoy: Envoy, mount_point: str) -> None:
|
|
745
|
+
"""If a given module already has an attribute of the same name as something nnsight wants to add, we need to rename it.
|
|
746
|
+
|
|
747
|
+
Directly edits the underlying class to accomplish this.
|
|
748
|
+
|
|
749
|
+
Args:
|
|
750
|
+
envoy (Envoy): Envoy to handle.
|
|
751
|
+
mount_point (str): Overloaded attribute name.
|
|
752
|
+
"""
|
|
753
|
+
|
|
754
|
+
warnings.warn(
|
|
755
|
+
f"Module `{self.path}` of type `{type(self._module)}` has pre-defined a `{mount_point}` attribute. nnsight access for `{mount_point}` will be mounted at `.nns_{mount_point}` instead of `.{mount_point}` for this module only."
|
|
756
|
+
)
|
|
757
|
+
|
|
758
|
+
# If we already shifted a mount point dont create another new class.
|
|
759
|
+
if "Preserved" in self.__class__.__name__:
|
|
760
|
+
|
|
761
|
+
new_cls = self.__class__
|
|
762
|
+
|
|
763
|
+
else:
|
|
764
|
+
|
|
765
|
+
new_cls = type(
|
|
766
|
+
f"{self.__class__.__name__}.Preserved",
|
|
767
|
+
(self.__class__,),
|
|
768
|
+
{},
|
|
769
|
+
)
|
|
770
|
+
|
|
771
|
+
self.__class__ = new_cls
|
|
772
|
+
|
|
773
|
+
# Get the normal proxy mount point
|
|
774
|
+
mount = getattr(Envoy, mount_point)
|
|
775
|
+
|
|
776
|
+
setattr(new_cls, f"nns_{mount_point}", mount)
|
|
777
|
+
|
|
778
|
+
if isinstance(mount, property):
|
|
779
|
+
|
|
780
|
+
mount = property(
|
|
781
|
+
lambda slf: slf.__dict__[mount_point],
|
|
782
|
+
mount.fset,
|
|
783
|
+
mount.fdel,
|
|
784
|
+
mount.__doc__,
|
|
785
|
+
)
|
|
786
|
+
|
|
787
|
+
setattr(new_cls, mount_point, mount)
|
|
788
|
+
|
|
789
|
+
# Move it to nns_<mount point>
|
|
790
|
+
self.__dict__[mount_point] = envoy
|
|
791
|
+
|
|
742
792
|
def _update(self, module: torch.nn.Module) -> None:
|
|
743
793
|
"""Updates the ._model attribute using a new model of the same architecture.
|
|
744
794
|
Used when loading the real weights (dispatching) and need to replace the underlying modules.
|
|
@@ -759,12 +809,19 @@ class Envoy(Batchable):
|
|
|
759
809
|
self._module.__path__ = self.path
|
|
760
810
|
|
|
761
811
|
if self._source is not None:
|
|
812
|
+
|
|
762
813
|
def wrap(fn: Callable, **kwargs):
|
|
763
814
|
|
|
764
|
-
bound_obj =
|
|
815
|
+
bound_obj = (
|
|
816
|
+
fn.__self__
|
|
817
|
+
if inspect.ismethod(fn) and fn.__name__ != "forward"
|
|
818
|
+
else None
|
|
819
|
+
)
|
|
765
820
|
|
|
766
821
|
if self.interleaving:
|
|
767
|
-
return self._interleaver.wrap_operation(
|
|
822
|
+
return self._interleaver.wrap_operation(
|
|
823
|
+
fn, **kwargs, bound_obj=bound_obj
|
|
824
|
+
)
|
|
768
825
|
else:
|
|
769
826
|
return fn
|
|
770
827
|
|
|
@@ -1168,15 +1225,21 @@ class OperationEnvoy:
|
|
|
1168
1225
|
|
|
1169
1226
|
if self._source is None:
|
|
1170
1227
|
fn = self._interleaver.current.request(f"{self.name}.fn")
|
|
1228
|
+
|
|
1229
|
+
#TODO maybe do something else here
|
|
1230
|
+
if isinstance(fn, torch.nn.Module):
|
|
1231
|
+
|
|
1232
|
+
msg = f"Don't call .source on a module ({getattr(fn, '__path__', '')}) from within another .source. Call it directly with: {getattr(fn, '__path__', '')}.source"
|
|
1233
|
+
raise ValueError(msg)
|
|
1171
1234
|
|
|
1172
1235
|
def wrap(fn: Callable, **kwargs):
|
|
1173
|
-
|
|
1236
|
+
|
|
1174
1237
|
bound_obj = (
|
|
1175
1238
|
fn.__self__
|
|
1176
|
-
if fn
|
|
1239
|
+
if getattr(fn, "__name__", None) != "forward" and inspect.ismethod(fn)
|
|
1177
1240
|
else None
|
|
1178
1241
|
)
|
|
1179
|
-
|
|
1242
|
+
|
|
1180
1243
|
return self._interleaver.wrap_operation(
|
|
1181
1244
|
fn, **kwargs, bound_obj=bound_obj
|
|
1182
1245
|
)
|
|
@@ -110,6 +110,9 @@ class Interleaver:
|
|
|
110
110
|
|
|
111
111
|
@wraps(fn)
|
|
112
112
|
def inner(module: torch.nn.Module, *args, **kwargs):
|
|
113
|
+
|
|
114
|
+
if torch.compiler.is_compiling():
|
|
115
|
+
return fn(module, *args, **kwargs)
|
|
113
116
|
|
|
114
117
|
if not hasattr(module, "__path__"):
|
|
115
118
|
return fn(module, *args, **kwargs)
|
|
@@ -58,7 +58,12 @@ class Tracer:
|
|
|
58
58
|
"""
|
|
59
59
|
|
|
60
60
|
def __init__(
|
|
61
|
-
self,
|
|
61
|
+
self,
|
|
62
|
+
source: List[str],
|
|
63
|
+
frame: FrameType,
|
|
64
|
+
start_line: int,
|
|
65
|
+
node: ast.With,
|
|
66
|
+
filename: str = None,
|
|
62
67
|
):
|
|
63
68
|
"""
|
|
64
69
|
Initialize Info with source code and frame information.
|
|
@@ -73,11 +78,15 @@ class Tracer:
|
|
|
73
78
|
self.frame = frame
|
|
74
79
|
self.start_line = start_line
|
|
75
80
|
self.node = node
|
|
76
|
-
self.filename =
|
|
77
|
-
|
|
81
|
+
self.filename = (
|
|
82
|
+
filename if filename is not None else f"<nnsight {id(self)}>"
|
|
83
|
+
)
|
|
84
|
+
|
|
78
85
|
def copy(self):
|
|
79
|
-
return Tracer.Info(
|
|
80
|
-
|
|
86
|
+
return Tracer.Info(
|
|
87
|
+
self.source, self.frame, self.start_line, self.node, self.filename
|
|
88
|
+
)
|
|
89
|
+
|
|
81
90
|
def __getstate__(self):
|
|
82
91
|
"""Get the state of the info for serialization."""
|
|
83
92
|
return {
|
|
@@ -86,17 +95,17 @@ class Tracer:
|
|
|
86
95
|
"filename": self.filename,
|
|
87
96
|
"frame": self.frame,
|
|
88
97
|
}
|
|
89
|
-
|
|
98
|
+
|
|
90
99
|
def __setstate__(self, state):
|
|
91
100
|
"""Set the state of the info for deserialization."""
|
|
92
101
|
self.source = state["source"]
|
|
93
102
|
self.start_line = state["start_line"]
|
|
94
103
|
self.filename = state["filename"]
|
|
95
104
|
self.frame = state["frame"]
|
|
96
|
-
|
|
105
|
+
|
|
97
106
|
self.node = None
|
|
98
107
|
|
|
99
|
-
def __init__(self, *args, backend: Backend = None, _info:Info = None, **kwargs):
|
|
108
|
+
def __init__(self, *args, backend: Backend = None, _info: Info = None, **kwargs):
|
|
100
109
|
"""
|
|
101
110
|
Initialize a Tracer instance.
|
|
102
111
|
|
|
@@ -113,7 +122,7 @@ class Tracer:
|
|
|
113
122
|
|
|
114
123
|
self.info = _info if _info is not None else None
|
|
115
124
|
|
|
116
|
-
if self.info is None:
|
|
125
|
+
if self.info is None:
|
|
117
126
|
self.capture()
|
|
118
127
|
|
|
119
128
|
def capture(self):
|
|
@@ -127,7 +136,7 @@ class Tracer:
|
|
|
127
136
|
"""
|
|
128
137
|
# Find the frame outside of nnsight by walking up the call stack
|
|
129
138
|
frame = inspect.currentframe()
|
|
130
|
-
|
|
139
|
+
|
|
131
140
|
while frame:
|
|
132
141
|
frame = frame.f_back
|
|
133
142
|
if frame and (
|
|
@@ -139,10 +148,12 @@ class Tracer:
|
|
|
139
148
|
# Get source code lines from the appropriate location
|
|
140
149
|
start_line = frame.f_lineno
|
|
141
150
|
|
|
151
|
+
# CASE 1: Were already inside of another nnsight trace.
|
|
142
152
|
if "__nnsight_tracing_info__" in frame.f_locals:
|
|
143
153
|
# For dynamically generated code, get source from tracing info
|
|
144
154
|
source_lines = frame.f_locals["__nnsight_tracing_info__"].source
|
|
145
155
|
|
|
156
|
+
# CASE 2: We're in an IPython console.
|
|
146
157
|
elif "_ih" in frame.f_locals:
|
|
147
158
|
import IPython
|
|
148
159
|
|
|
@@ -152,12 +163,24 @@ class Tracer:
|
|
|
152
163
|
if not source_lines[-1].endswith("\n"):
|
|
153
164
|
source_lines[-1] += "\n"
|
|
154
165
|
|
|
166
|
+
# CASE 3: We're in a regular Python file.
|
|
155
167
|
elif not frame.f_code.co_filename.startswith("<nnsight"):
|
|
156
168
|
# For regular files, get source lines using inspect
|
|
157
169
|
source_lines, offset = inspect.getsourcelines(frame)
|
|
158
170
|
|
|
159
171
|
start_line = start_line if offset == 0 else start_line - offset + 1
|
|
160
172
|
|
|
173
|
+
# CASE 4: We're in a regular Python interactive console.
|
|
174
|
+
elif frame.f_code.co_filename == "<nnsight-console>":
|
|
175
|
+
from ... import __INTERACTIVE_CONSOLE__
|
|
176
|
+
|
|
177
|
+
source_lines = __INTERACTIVE_CONSOLE__.buffer
|
|
178
|
+
# Add newline to each source line if it doesn't end with one
|
|
179
|
+
source_lines = [
|
|
180
|
+
line if line.endswith("\n") else line + "\n" for line in source_lines
|
|
181
|
+
]
|
|
182
|
+
|
|
183
|
+
|
|
161
184
|
else:
|
|
162
185
|
raise ValueError("No source code found")
|
|
163
186
|
|
|
@@ -166,10 +189,10 @@ class Tracer:
|
|
|
166
189
|
"\t "
|
|
167
190
|
) # indent for removing leading tabs/spaces
|
|
168
191
|
indent = len(source_lines[0]) - len(stripped)
|
|
169
|
-
|
|
192
|
+
|
|
170
193
|
# If theres an indent, we need to remove it. This handles the case of say a trace in an indented function. E.x. a trace inside a method on a class.
|
|
171
194
|
if indent > 0:
|
|
172
|
-
|
|
195
|
+
|
|
173
196
|
source_lines = [
|
|
174
197
|
line[indent:] if line.strip() else line for line in source_lines
|
|
175
198
|
]
|
|
@@ -178,22 +201,18 @@ class Tracer:
|
|
|
178
201
|
start_line, source_lines, node = self.parse(source_lines, start_line)
|
|
179
202
|
|
|
180
203
|
# Calculate indentation level of the Tracer creation line.
|
|
181
|
-
stripped = source_lines[0].lstrip(
|
|
182
|
-
"\t "
|
|
183
|
-
) # removes leading tabs/spaces
|
|
204
|
+
stripped = source_lines[0].lstrip("\t ") # removes leading tabs/spaces
|
|
184
205
|
indent = len(source_lines[0]) - len(stripped) - 4
|
|
185
|
-
|
|
186
|
-
|
|
206
|
+
|
|
207
|
+
# If theres an indent (more than just the indentation of the with block), we need to remove it. This handles the case of say a trace in an indented block. E.x. a trace inside a for loop or another with block.
|
|
187
208
|
if indent > 0:
|
|
188
|
-
|
|
209
|
+
|
|
189
210
|
source_lines = [
|
|
190
211
|
line[indent:] if line.strip() else line for line in source_lines
|
|
191
212
|
]
|
|
192
213
|
|
|
193
214
|
# Store the captured information for later use
|
|
194
|
-
self.info = Tracer.Info(
|
|
195
|
-
source_lines, frame, start_line, node
|
|
196
|
-
)
|
|
215
|
+
self.info = Tracer.Info(source_lines, frame, start_line, node)
|
|
197
216
|
|
|
198
217
|
def parse(self, source_lines, start_line):
|
|
199
218
|
"""
|
|
@@ -233,7 +252,7 @@ class Tracer:
|
|
|
233
252
|
raise WithBlockNotFoundError(f"With block not found at line {start_line}")
|
|
234
253
|
|
|
235
254
|
end_line = visitor.target.end_lineno
|
|
236
|
-
|
|
255
|
+
|
|
237
256
|
start_line = visitor.target.body[0].lineno - 1
|
|
238
257
|
|
|
239
258
|
return start_line, source_lines[start_line:end_line], visitor.target
|
|
@@ -318,9 +337,8 @@ class Tracer:
|
|
|
318
337
|
ctypes.pythonapi.PyFrame_LocalsToFast(
|
|
319
338
|
ctypes.py_object(frame), ctypes.c_int(0)
|
|
320
339
|
)
|
|
321
|
-
|
|
340
|
+
|
|
322
341
|
state.clear()
|
|
323
|
-
|
|
324
342
|
|
|
325
343
|
def __enter__(self):
|
|
326
344
|
"""
|
|
@@ -335,7 +353,7 @@ class Tracer:
|
|
|
335
353
|
|
|
336
354
|
if self.info is None:
|
|
337
355
|
self.capture()
|
|
338
|
-
|
|
356
|
+
|
|
339
357
|
if isinstance(self.info.node.body[0], ast.Pass):
|
|
340
358
|
return self
|
|
341
359
|
|
|
@@ -390,11 +408,11 @@ class Tracer:
|
|
|
390
408
|
self.backend(self)
|
|
391
409
|
|
|
392
410
|
return True
|
|
393
|
-
|
|
411
|
+
|
|
394
412
|
self.backend(self)
|
|
395
|
-
|
|
413
|
+
|
|
396
414
|
### Serialization ###
|
|
397
|
-
|
|
415
|
+
|
|
398
416
|
def __getstate__(self):
|
|
399
417
|
"""Get the state of the tracer for serialization."""
|
|
400
418
|
return {
|
|
@@ -402,7 +420,7 @@ class Tracer:
|
|
|
402
420
|
"kwargs": self.kwargs,
|
|
403
421
|
"info": self.info,
|
|
404
422
|
}
|
|
405
|
-
|
|
423
|
+
|
|
406
424
|
def __setstate__(self, state):
|
|
407
425
|
"""Set the state of the tracer for deserialization."""
|
|
408
426
|
self.args = state["args"]
|
|
@@ -410,4 +428,3 @@ class Tracer:
|
|
|
410
428
|
self.info = state["info"]
|
|
411
429
|
self.info.start_line = 0
|
|
412
430
|
self.backend = ExecutionBackend()
|
|
413
|
-
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/function_vectors.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/future_lens.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/logit_lens.webp
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/_static/images/tutorials/walkthrough.webp
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/custom_functions.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/early_stopping.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/editing_tutorial.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/lora_training.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/model_editing.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/multiple_token.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/remote_execution.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/features/scan_validate.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/LoRA_tutorial.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/activation_patching.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/attribution_patching.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/boundless_DAS.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/dict_learning.ipynb
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/diffusion_lens.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/docs/source/notebooks/tutorials/start_remote_access.ipynb
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/model_runners/GPUModelRunner.py
RENAMED
|
File without changes
|
{nnsight-0.5.0.dev4 → nnsight-0.5.0.dev6}/src/nnsight/modeling/vllm/model_runners/__init__.py
RENAMED
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|
|
File without changes
|