kerf-multikernel 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- kerf_multikernel-0.1.0/LICENSE +201 -0
- kerf_multikernel-0.1.0/PKG-INFO +382 -0
- kerf_multikernel-0.1.0/README.md +362 -0
- kerf_multikernel-0.1.0/pyproject.toml +76 -0
- kerf_multikernel-0.1.0/src/kerf/__init__.py +74 -0
- kerf_multikernel-0.1.0/src/kerf/baseline.py +187 -0
- kerf_multikernel-0.1.0/src/kerf/cli.py +56 -0
- kerf_multikernel-0.1.0/src/kerf/console/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/console/main.py +252 -0
- kerf_multikernel-0.1.0/src/kerf/create/__init__.py +17 -0
- kerf_multikernel-0.1.0/src/kerf/create/main.py +780 -0
- kerf_multikernel-0.1.0/src/kerf/data/__init__.py +24 -0
- kerf_multikernel-0.1.0/src/kerf/data/kerf-init +0 -0
- kerf_multikernel-0.1.0/src/kerf/daxfs/__init__.py +31 -0
- kerf_multikernel-0.1.0/src/kerf/daxfs/mkdaxfs.py +597 -0
- kerf_multikernel-0.1.0/src/kerf/delete/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/delete/main.py +238 -0
- kerf_multikernel-0.1.0/src/kerf/docker/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/docker/image.py +234 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/__init__.py +27 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/cells.py +51 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/extractor.py +362 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/overlay.py +378 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/parser.py +1548 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/reporter.py +308 -0
- kerf_multikernel-0.1.0/src/kerf/dtc/validator.py +792 -0
- kerf_multikernel-0.1.0/src/kerf/exceptions.py +49 -0
- kerf_multikernel-0.1.0/src/kerf/exec/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/exec/main.py +260 -0
- kerf_multikernel-0.1.0/src/kerf/init/__init__.py +13 -0
- kerf_multikernel-0.1.0/src/kerf/init/main.py +696 -0
- kerf_multikernel-0.1.0/src/kerf/kill/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/kill/main.py +275 -0
- kerf_multikernel-0.1.0/src/kerf/lazy_cma.py +132 -0
- kerf_multikernel-0.1.0/src/kerf/load/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/load/main.py +613 -0
- kerf_multikernel-0.1.0/src/kerf/metadata.py +134 -0
- kerf_multikernel-0.1.0/src/kerf/models.py +241 -0
- kerf_multikernel-0.1.0/src/kerf/resources.py +369 -0
- kerf_multikernel-0.1.0/src/kerf/runtime.py +527 -0
- kerf_multikernel-0.1.0/src/kerf/show/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/show/main.py +616 -0
- kerf_multikernel-0.1.0/src/kerf/unload/__init__.py +21 -0
- kerf_multikernel-0.1.0/src/kerf/unload/main.py +310 -0
- kerf_multikernel-0.1.0/src/kerf/update/__init__.py +15 -0
- kerf_multikernel-0.1.0/src/kerf/update/main.py +450 -0
- kerf_multikernel-0.1.0/src/kerf/utils.py +95 -0
- kerf_multikernel-0.1.0/src/kerf/vmlinuz.py +213 -0
|
@@ -0,0 +1,201 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for reasonable and customary use in describing the
|
|
141
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
166
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
167
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
168
|
+
or other liability obligations and/or rights consistent with this
|
|
169
|
+
License. However, in accepting such obligations, You may act only
|
|
170
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
171
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
172
|
+
defend, and hold each Contributor harmless for any liability
|
|
173
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
174
|
+
of your accepting any such warranty or additional liability.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
|
177
|
+
|
|
178
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
179
|
+
|
|
180
|
+
To apply the Apache License to your work, attach the following
|
|
181
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
182
|
+
replaced with your own identifying information. (Don't include
|
|
183
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
184
|
+
comment syntax for the file format. We also recommend that a
|
|
185
|
+
file or class name and description of purpose be included on the
|
|
186
|
+
same "printed page" as the copyright notice for easier
|
|
187
|
+
identification within third-party archives.
|
|
188
|
+
|
|
189
|
+
Copyright 2025 Multikernel Technologies, Inc.
|
|
190
|
+
|
|
191
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
192
|
+
you may not use this file except in compliance with the License.
|
|
193
|
+
You may obtain a copy of the License at
|
|
194
|
+
|
|
195
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
196
|
+
|
|
197
|
+
Unless required by applicable law or agreed to in writing, software
|
|
198
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
199
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
200
|
+
See the License for the specific language governing permissions and
|
|
201
|
+
limitations under the License.
|
|
@@ -0,0 +1,382 @@
|
|
|
1
|
+
Metadata-Version: 2.1
|
|
2
|
+
Name: kerf-multikernel
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Multikernel Management CLI tool
|
|
5
|
+
Author: Cong Wang
|
|
6
|
+
Requires-Python: >=3.9,<4.0
|
|
7
|
+
Classifier: Programming Language :: Python :: 3
|
|
8
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
9
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
13
|
+
Requires-Dist: click (>=8.0.0,<9.0.0)
|
|
14
|
+
Requires-Dist: pylibfdt (>=1.7.0,<2.0.0)
|
|
15
|
+
Requires-Dist: pyudev (>=0.24.0,<0.25.0)
|
|
16
|
+
Requires-Dist: pyyaml (>=6.0,<7.0)
|
|
17
|
+
Requires-Dist: rdtsc (>=0.2.1,<0.3.0)
|
|
18
|
+
Description-Content-Type: text/markdown
|
|
19
|
+
|
|
20
|
+

|
|
21
|
+
|
|
22
|
+
# Kerf: Multikernel Management System
|
|
23
|
+
|
|
24
|
+
## Overview
|
|
25
|
+
|
|
26
|
+
`kerf` is a comprehensive multikernel management system designed to orchestrate and manage multiple kernel instances on a single host. Starting with advanced device tree compilation and validation, `kerf` provides the foundation for complete multikernel lifecycle management.
|
|
27
|
+
|
|
28
|
+
Unlike standard tools that only perform basic format conversion, `kerf` understands multikernel semantics and **always validates** resource allocations and detects conflicts. The system is architected to evolve into a complete multikernel runtime environment.
|
|
29
|
+
|
|
30
|
+
## Features
|
|
31
|
+
|
|
32
|
+
`kerf` is a comprehensive multikernel management platform with the following capabilities:
|
|
33
|
+
|
|
34
|
+
- **Resource Pool Initialization**: Initialize hardware resource pools available for multikernel allocation
|
|
35
|
+
- **Resource Conflict Detection**: Detect and prevent allocation conflicts for CPUs, memory regions, and devices
|
|
36
|
+
- **Resource Validation**: Ensure allocations don't exceed available resources and references are valid
|
|
37
|
+
- **Atomicity**: Validate all allocations together before deployment
|
|
38
|
+
- **Kernel Loading & Execution**: Load and execute multiple kernel instances with proper isolation
|
|
39
|
+
- **Instance Lifecycle**: Create, delete, and manage kernel instances
|
|
40
|
+
- **Dynamic Resource Management**: Allocation and deallocation of system resources
|
|
41
|
+
- **Monitoring & Debugging**: Real-time monitoring of kernel instances and system health
|
|
42
|
+
- **Security & Isolation**: Advanced security policies and isolation mechanisms
|
|
43
|
+
- **Orchestration**: High-level orchestration of complex multikernel workloads
|
|
44
|
+
|
|
45
|
+
## Architecture
|
|
46
|
+
|
|
47
|
+
### Design Philosophy
|
|
48
|
+
|
|
49
|
+
The `kerf` system is built on foundational principles that support both current resource pool management and future multikernel runtime features:
|
|
50
|
+
|
|
51
|
+
1. **Single Source of Truth**: Baseline DTS describes hardware resources available for allocation
|
|
52
|
+
2. **Mandatory Validation**: Every operation validates the configuration - validation is not optional
|
|
53
|
+
3. **Fail-Fast**: Catch resource conflicts immediately, never produce invalid output
|
|
54
|
+
4. **Overlay-based Management**: Dynamic instance changes are managed via device tree overlays
|
|
55
|
+
5. **Extensible Architecture**: Designed to support future kernel loading, execution, and management capabilities
|
|
56
|
+
6. **Developer-Friendly**: Clear error messages with suggestions for fixing problems
|
|
57
|
+
7. **Runtime-Ready**: Current design anticipates future kernel execution and lifecycle management needs
|
|
58
|
+
|
|
59
|
+
### Compilation Model
|
|
60
|
+
|
|
61
|
+
**Resource pool initialization:**
|
|
62
|
+
```
|
|
63
|
+
Input: Baseline DTS (resources only)
|
|
64
|
+
│
|
|
65
|
+
▼
|
|
66
|
+
┌─────────┐
|
|
67
|
+
│ kerf │ ← Initializes resource pool
|
|
68
|
+
│ init │ and validates
|
|
69
|
+
└─────────┘
|
|
70
|
+
│
|
|
71
|
+
▼
|
|
72
|
+
Baseline DTB
|
|
73
|
+
(resources only)
|
|
74
|
+
→ /sys/fs/multikernel/device_tree
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
**Overlay-based dynamic changes:**
|
|
78
|
+
```
|
|
79
|
+
Current State Modified State
|
|
80
|
+
(Baseline + Overlays) (After change)
|
|
81
|
+
│ │
|
|
82
|
+
├───────────────────────┤
|
|
83
|
+
│ │
|
|
84
|
+
▼ ▼
|
|
85
|
+
┌─────────┐ ┌─────────┐
|
|
86
|
+
│ Compute │ │ Compute │
|
|
87
|
+
│ Delta │ │ Delta │
|
|
88
|
+
└─────────┘ └─────────┘
|
|
89
|
+
│ │
|
|
90
|
+
└───────────┬───────────┘
|
|
91
|
+
│
|
|
92
|
+
▼
|
|
93
|
+
┌─────────────┐
|
|
94
|
+
│ kerf │ ← Validates full state
|
|
95
|
+
│ (create/ │ before generating overlay
|
|
96
|
+
│ update/ │
|
|
97
|
+
│ delete) │
|
|
98
|
+
└─────────────┘
|
|
99
|
+
│
|
|
100
|
+
▼
|
|
101
|
+
DTBO Overlay
|
|
102
|
+
│
|
|
103
|
+
▼
|
|
104
|
+
→ /sys/fs/multikernel/overlays/new
|
|
105
|
+
│
|
|
106
|
+
▼
|
|
107
|
+
Applied Overlay
|
|
108
|
+
→ /sys/fs/multikernel/overlays/tx_XXX/
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
**Complete system state:**
|
|
112
|
+
```
|
|
113
|
+
Baseline DTB (static)
|
|
114
|
+
│
|
|
115
|
+
├─── Overlay tx_101 (instance: web-server)
|
|
116
|
+
├─── Overlay tx_102 (instance: database)
|
|
117
|
+
└─── Overlay tx_103 (update: web-server resources)
|
|
118
|
+
│
|
|
119
|
+
▼
|
|
120
|
+
Effective Device Tree
|
|
121
|
+
(Baseline + All Applied Overlays)
|
|
122
|
+
│
|
|
123
|
+
▼
|
|
124
|
+
Kernel Instance Views
|
|
125
|
+
/sys/fs/multikernel/instances/*
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
**Key Points:**
|
|
129
|
+
- **Baseline contains only resources**: Hardware resources available for allocation, loaded once via `kerf init`
|
|
130
|
+
- **Instances created via overlays**: Dynamic instance lifecycle managed through device tree overlays (DTBO)
|
|
131
|
+
- **Overlay generation**: Computes delta between current and modified state, generates minimal DTBO
|
|
132
|
+
- **Transactional overlays**: Each overlay is a transaction with rollback support via `rmdir`
|
|
133
|
+
- **Validation is mandatory**: Always validates full state (baseline + all overlays) before applying
|
|
134
|
+
- **Single source of truth**: Baseline DTB is the authoritative resource configuration, overlays add instances dynamically
|
|
135
|
+
|
|
136
|
+
## Current Capabilities
|
|
137
|
+
|
|
138
|
+
### Resource Pool Management & Validation
|
|
139
|
+
- **Resource Pool Initialization**: Initialize hardware resource pools for multikernel allocation
|
|
140
|
+
- **Advanced Validation**: Comprehensive resource conflict detection and validation
|
|
141
|
+
- **Baseline Management**: Initialize and manage baseline device tree containing hardware resources
|
|
142
|
+
- **Format Support**: DTS to DTB compilation for baseline configuration
|
|
143
|
+
- **Error Reporting**: Detailed error messages with actionable suggestions
|
|
144
|
+
- **Resource Analysis**: Complete resource utilization reporting
|
|
145
|
+
- **CPU & NUMA Topology**: Full support for CPU topology and NUMA-aware resource allocation
|
|
146
|
+
|
|
147
|
+
### Command Line Interface
|
|
148
|
+
```bash
|
|
149
|
+
# Initialize resource pool with CPUs (memory parsed from /proc/iomem)
|
|
150
|
+
kerf init --cpus=4-7
|
|
151
|
+
|
|
152
|
+
# Initialize with CPUs and devices
|
|
153
|
+
kerf init --cpus=4-31 --devices=enp9s0_dev,nvme0
|
|
154
|
+
|
|
155
|
+
# Create kernel instance with resource allocation
|
|
156
|
+
kerf create web-server --cpus=4-7 --memory=2GB
|
|
157
|
+
kerf create database --cpu-count=8 --memory=16GB
|
|
158
|
+
|
|
159
|
+
# Load kernel image with initrd and boot parameters
|
|
160
|
+
kerf load --kernel=/boot/vmlinuz --initrd=/boot/initrd.img \
|
|
161
|
+
--cmdline="root=/dev/sda1 ro" --id=1
|
|
162
|
+
|
|
163
|
+
# Boot a kernel instance
|
|
164
|
+
kerf exec web-server
|
|
165
|
+
|
|
166
|
+
# Show kernel instance information
|
|
167
|
+
kerf show
|
|
168
|
+
kerf show web-server
|
|
169
|
+
|
|
170
|
+
# Shutdown a running kernel instance
|
|
171
|
+
kerf kill web-server
|
|
172
|
+
|
|
173
|
+
# Unload kernel image from an instance
|
|
174
|
+
kerf unload web-server
|
|
175
|
+
|
|
176
|
+
# Delete a kernel instance
|
|
177
|
+
kerf delete web-server
|
|
178
|
+
|
|
179
|
+
# Use --help for detailed options and usage
|
|
180
|
+
kerf --help
|
|
181
|
+
kerf <command> --help
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
### Technical Foundation
|
|
185
|
+
The current resource pool management provides essential building blocks for future multikernel capabilities:
|
|
186
|
+
|
|
187
|
+
- **Resource Pool Initialization**: Initializes hardware resource pools for safe multikernel allocation
|
|
188
|
+
- **Resource Validation**: Ensures safe resource allocation before kernel execution
|
|
189
|
+
- **Instance Isolation**: Provides the foundation for secure kernel isolation
|
|
190
|
+
- **Configuration Management**: Enables consistent and validated system configurations
|
|
191
|
+
- **Error Handling**: Establishes patterns for robust error reporting and recovery
|
|
192
|
+
- **Extensible Architecture**: Designed to support future kernel management APIs
|
|
193
|
+
|
|
194
|
+
These foundational capabilities are essential for safe and reliable multikernel execution, making `kerf` the ideal platform for building comprehensive multikernel management systems.
|
|
195
|
+
|
|
196
|
+
|
|
197
|
+
## Global Device Tree Format
|
|
198
|
+
|
|
199
|
+
### Structure Overview
|
|
200
|
+
|
|
201
|
+
The baseline device tree contains only the **Resources** section, which describes all physical hardware available for allocation. Instances and device references are added dynamically via overlays when using `kerf create`.
|
|
202
|
+
|
|
203
|
+
1. **Resources** (`/resources`): Complete description of all physical resources (baseline only)
|
|
204
|
+
2. **Instances** (`/instances`): Resource assignments for each spawn kernel (added via overlays)
|
|
205
|
+
3. **Device References**: Linkage between instances and hardware devices (added via overlays)
|
|
206
|
+
|
|
207
|
+
### Baseline Example
|
|
208
|
+
|
|
209
|
+
The baseline contains hardware resources used for allocation. Resources are typically passed via command line arguments during the `kerf init` command. Instances are created dynamically via overlays using `kerf create`.
|
|
210
|
+
|
|
211
|
+
### Mapping to Kernel Filesystem Interface
|
|
212
|
+
|
|
213
|
+
**Device Tree Structure → Kernel Filesystem Interface:**
|
|
214
|
+
|
|
215
|
+
```
|
|
216
|
+
DTS: /resources → /sys/kernel/multikernel/device_tree (writable, single source of truth)
|
|
217
|
+
DTS: /instances/web-server → /sys/kernel/multikernelinstances/web-server/ (read-only)
|
|
218
|
+
DTS: /instances/database → /sys/kernel/multikernel/instances/database/ (read-only)
|
|
219
|
+
DTS: /instances/compute → /sys/kernel/multikernel/instances/compute/ (read-only)
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
**Name-based addressing:**
|
|
223
|
+
- Instance node name in DTS (`web-server`) = directory name in kernel filesystem (`instances/web-server/`)
|
|
224
|
+
- Kernel assigns numeric IDs, but users reference by name
|
|
225
|
+
- No manual ID coordination needed
|
|
226
|
+
- Instance directories are auto-generated by the kernel from the global device tree
|
|
227
|
+
|
|
228
|
+
## Validation Rules
|
|
229
|
+
|
|
230
|
+
### Validation is Mandatory
|
|
231
|
+
|
|
232
|
+
**All `kerf` operations perform validation automatically:**
|
|
233
|
+
- Compiling DTS to DTB → validates
|
|
234
|
+
- Converting formats → validates
|
|
235
|
+
- Generating reports → validates first
|
|
236
|
+
|
|
237
|
+
**Validation cannot be disabled or skipped.**
|
|
238
|
+
|
|
239
|
+
### CPU Allocation Validation
|
|
240
|
+
|
|
241
|
+
**Rules:**
|
|
242
|
+
1. CPUs must be defined in the baseline resource pool
|
|
243
|
+
2. No CPU can be allocated to multiple instances
|
|
244
|
+
3. CPU lists should be explicitly enumerated
|
|
245
|
+
|
|
246
|
+
### Memory Allocation Validation
|
|
247
|
+
|
|
248
|
+
**Rules:**
|
|
249
|
+
1. Memory regions must be within the baseline memory pool
|
|
250
|
+
2. Memory regions cannot overlap between instances
|
|
251
|
+
3. Sum of all allocations must not exceed baseline memory pool size
|
|
252
|
+
4. Memory base addresses must be page-aligned (4KB = 0x1000)
|
|
253
|
+
|
|
254
|
+
|
|
255
|
+
### Device Allocation Validation
|
|
256
|
+
|
|
257
|
+
**Rules:**
|
|
258
|
+
1. Referenced devices must be defined in the baseline
|
|
259
|
+
2. Devices can only be allocated to one instance (exclusive access)
|
|
260
|
+
3. Device references must be valid (no dangling phandles)
|
|
261
|
+
4. SR-IOV VF numbers must be within device limits
|
|
262
|
+
5. Namespace IDs must be within device limits
|
|
263
|
+
|
|
264
|
+
### Global Resource Validation
|
|
265
|
+
|
|
266
|
+
**Rules:**
|
|
267
|
+
1. Instance names must be unique
|
|
268
|
+
2. All phandle references must resolve
|
|
269
|
+
3. Baseline resource configuration must be complete and consistent
|
|
270
|
+
|
|
271
|
+
|
|
272
|
+
## Integration with Kernel
|
|
273
|
+
|
|
274
|
+
### Kernel Interface
|
|
275
|
+
|
|
276
|
+
The kernel exposes a filesystem interface (mounted at `/sys/fs/multikernel/`) that manages baseline resources and overlay-based instance changes:
|
|
277
|
+
|
|
278
|
+
**Kernel Interface Structure:**
|
|
279
|
+
```
|
|
280
|
+
/sys/fs/multikernel/
|
|
281
|
+
├── device_tree # Baseline DTB (resources only, writable via kerf init)
|
|
282
|
+
├── overlays/ # Overlay subsystem
|
|
283
|
+
│ ├── new # Write DTBO here to apply overlay
|
|
284
|
+
│ ├── tx_101/ # Applied overlay transaction
|
|
285
|
+
│ │ ├── id # Transaction ID: "101"
|
|
286
|
+
│ │ ├── status # "applied" | "failed" | "removed"
|
|
287
|
+
│ │ ├── dtbo # Original overlay blob (binary)
|
|
288
|
+
│ │ └── ...
|
|
289
|
+
│ └── tx_102/
|
|
290
|
+
│ └── ...
|
|
291
|
+
└── instances/ # Runtime kernel instances (read-only)
|
|
292
|
+
├── web-server/
|
|
293
|
+
│ ├── id # Instance ID
|
|
294
|
+
│ ├── status # Instance status
|
|
295
|
+
│ └── ...
|
|
296
|
+
└── ...
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
**Key Design Principles:**
|
|
300
|
+
- **Baseline Separation**: Baseline (`device_tree`) contains only resources - no instances
|
|
301
|
+
- **Overlay-based Changes**: All dynamic changes (create, update, delete instances) via overlays
|
|
302
|
+
- **Rollback Support**: Remove overlay transaction directory (`rmdir /sys/fs/multikernel/overlays/tx_XXX/`) to rollback changes
|
|
303
|
+
- **Kernel-Generated**: Instance directories auto-generated from baseline + applied overlays
|
|
304
|
+
|
|
305
|
+
|
|
306
|
+
## Dependencies
|
|
307
|
+
|
|
308
|
+
### Required Dependencies
|
|
309
|
+
|
|
310
|
+
```toml
|
|
311
|
+
[tool.poetry.dependencies]
|
|
312
|
+
python = "^3.8"
|
|
313
|
+
pylibfdt = "^1.7.0" # Device tree parsing (from dtc project)
|
|
314
|
+
```
|
|
315
|
+
|
|
316
|
+
### Installation
|
|
317
|
+
|
|
318
|
+
```bash
|
|
319
|
+
# From PyPI (the distribution is named kerf-multikernel;
|
|
320
|
+
# the installed command is still 'kerf')
|
|
321
|
+
pip install kerf-multikernel
|
|
322
|
+
|
|
323
|
+
# Or isolated with pipx, avoiding conflicts with other packages
|
|
324
|
+
pipx install kerf-multikernel
|
|
325
|
+
|
|
326
|
+
# From source (recommended for development)
|
|
327
|
+
git clone https://github.com/multikernel/kerf.git
|
|
328
|
+
cd kerf
|
|
329
|
+
# Installs 'kerf' command to ~/.local/bin/kerf
|
|
330
|
+
pip install -e .
|
|
331
|
+
|
|
332
|
+
# Installs 'kerf' command to the system Python's scripts directory
|
|
333
|
+
# (typically /usr/local/bin/kerf, or /usr/bin/kerf if using system Python)
|
|
334
|
+
sudo pip install .
|
|
335
|
+
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
### Getting Started
|
|
339
|
+
|
|
340
|
+
```bash
|
|
341
|
+
# Install in development mode
|
|
342
|
+
pip install -e .
|
|
343
|
+
|
|
344
|
+
# Test the installation
|
|
345
|
+
kerf --help
|
|
346
|
+
kerf init --help
|
|
347
|
+
|
|
348
|
+
# Try with example baseline configuration
|
|
349
|
+
kerf init --input=examples/baseline.dts --report
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
## Examples
|
|
353
|
+
|
|
354
|
+
The `examples/` directory contains sample baseline Device Tree Source (DTS) files demonstrating various hardware resource configurations:
|
|
355
|
+
|
|
356
|
+
- **`baseline.dts`** - Complete baseline with CPU, memory, and device resources (32 CPUs, 16GB memory)
|
|
357
|
+
- **`minimal.dts`** - Simple baseline for testing and development (8 CPUs, 8GB memory)
|
|
358
|
+
- **`edge_computing.dts`** - Edge computing baseline with GPU support for AI inference (16 CPUs, 32GB memory)
|
|
359
|
+
- **`numa_topology.dts`** - Advanced NUMA topology baseline with 4 NUMA nodes and topology-aware allocation
|
|
360
|
+
- **`system.dts`** - Example baseline with various device configurations
|
|
361
|
+
- **`conflict_example.dts`** - Intentionally invalid baseline demonstrating common validation errors
|
|
362
|
+
|
|
363
|
+
**Note**: All baseline files contain **only** hardware resources - no instances. Instances are created dynamically via overlays using `kerf create` command.
|
|
364
|
+
|
|
365
|
+
## CPU and NUMA Topology Support
|
|
366
|
+
|
|
367
|
+
Kerf provides comprehensive support for CPU and NUMA topology management:
|
|
368
|
+
|
|
369
|
+
### Key Features
|
|
370
|
+
- **CPU Topology**: Socket, core, and thread mapping with SMT/hyperthreading support
|
|
371
|
+
- **NUMA Awareness**: NUMA node definition with memory regions and CPU assignments
|
|
372
|
+
- **Topology Policies**: CPU affinity (`compact`, `spread`, `local`) and memory policies (`local`, `interleave`, `bind`)
|
|
373
|
+
- **Performance Validation**: Automatic validation of topology constraints and performance warnings
|
|
374
|
+
|
|
375
|
+
For detailed information about CPU and NUMA topology support, see [CPU_NUMA_TOPOLOGY.md](docs/CPU_NUMA_TOPOLOGY.md).
|
|
376
|
+
|
|
377
|
+
## References
|
|
378
|
+
|
|
379
|
+
- **Device Tree Specification**: https://devicetree-specification.readthedocs.io/
|
|
380
|
+
- **libfdt Documentation**: https://git.kernel.org/pub/scm/utils/dtc/dtc.git/tree/Documentation
|
|
381
|
+
|
|
382
|
+
|