damo 3.3.2__tar.gz → 3.3.4__tar.gz

{damo-3.3.2 → damo-3.3.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: damo
-Version: 3.3.2
+Version: 3.3.4
 Summary: DAMON user-space tool
 Home-page: https://github.com/damonitor/damo
 Author: SeongJae Park
@@ -71,7 +71,7 @@ The second and last commands will show the access pattern of your workload,
 like below:
 
 ![masim_stairs_snapshot](images/masim_stairs_snapshot.png)
-![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.2/images/masim_stairs_heatmap_ascii.png)
+![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.4/images/masim_stairs_heatmap_ascii.png)
 
 
 FAQs
@@ -90,7 +90,7 @@ Where can I get more detailed usage?
 ------------------------------------
 
 The below sections provide quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 What does the version numbers mean?
@@ -115,7 +115,7 @@ We try our best to make `damo` stable and doesn't introduce regressions to
 users.  However, nothing goes forever.  Sometimes, some features will be
 deprecated.  Some features will have longer support more than others.
 
-In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) and not explicitly
+In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) and not explicitly
 marked as experimental will be better supported, and provides at least three
 months of deprecation grace period.  Within the grace period, users can ask
 extension of the support.  Even after the grace period, please reach out to
@@ -138,7 +138,7 @@ Quick Intro for Major Features
 ==============================
 
 Below are quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 Snapshot Data Access Pattern

{damo-3.3.2 → damo-3.3.4}/README.md

@@ -47,7 +47,7 @@ The second and last commands will show the access pattern of your workload,
 like below:
 
 ![masim_stairs_snapshot](images/masim_stairs_snapshot.png)
-![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.2/images/masim_stairs_heatmap_ascii.png)
+![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.4/images/masim_stairs_heatmap_ascii.png)
 
 
 FAQs
@@ -66,7 +66,7 @@ Where can I get more detailed usage?
 ------------------------------------
 
 The below sections provide quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 What does the version numbers mean?
@@ -91,7 +91,7 @@ We try our best to make `damo` stable and doesn't introduce regressions to
 users.  However, nothing goes forever.  Sometimes, some features will be
 deprecated.  Some features will have longer support more than others.
 
-In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) and not explicitly
+In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) and not explicitly
 marked as experimental will be better supported, and provides at least three
 months of deprecation grace period.  Within the grace period, users can ask
 extension of the support.  Even after the grace period, please reach out to
@@ -114,7 +114,7 @@ Quick Intro for Major Features
 ==============================
 
 Below are quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 Snapshot Data Access Pattern

{damo-3.3.2 → damo-3.3.4}/src/damo/_damo_records.py

@@ -246,6 +246,28 @@ class DamonRecord:
         self.snapshots += other.snapshots
         self.snapshots.sort(key=lambda s: s.start_time)
 
+class DamoRecords:
+    kdamonds = None # list of _damon.Kdamond
+    records = None  # list of DamonRecord
+
+    def __init__(self, kdamonds, records):
+        if kdamonds is None:
+            kdamonds = []
+        if records is None:
+            records = []
+        self.kdamonds = kdamonds
+        self.records = records
+
+    @classmethod
+    def from_kvpairs(self, kv):
+        return DamoRecords(kdamonds=kv['kdamonds'], records=kv['records'])
+
+    def to_kvpairs(self, raw=False):
+        return collections.OrderedDict([
+            ('kdamonds', [k.to_kvapairs(raw=raw) for k in self.kdamonds]),
+            ('records', [r.to_kvpairs(raw=raw) for r in self.records]),
+            ])
+
 # for monitoring results manipulation
 
 def merge_records(records):
@@ -539,6 +561,11 @@ def parse_perf_damon_record(
 
 def parse_json(json_str):
     kvpairs = json.loads(json_str)
+    damo_record_ver = 0
+    if 'damo_reocrd_ver' in kvpairs:
+        damo_record_ver = kvpairs['damo_record_ver']
+    if damo_record_ver == 1:
+        return DamoRecords.from_kvpairs(kvpairs['damo_records'])
     return [DamonRecord.from_kvpairs(kvp) for kvp in kvpairs]
 
 def parse_compressed_json(record_file):
@@ -1913,6 +1940,19 @@ def get_records(tried_regions_of=None, record_file=None,
     filter_copy.filter_records(records)
     return records, None
 
+def get_damo_records(
+        tried_regions_of=None, record_file=None, snapshot_damos_filters=None,
+        record_filter=None, total_sz_only=False, dont_merge_regions=True):
+    '''
+    Returns DamoRecords and err.
+    '''
+    damon_records, err = get_records(
+            tried_regions_of, record_file, snapshot_damos_filters,
+            record_filter, total_sz_only, dont_merge_regions)
+    if err is not None:
+        return None, err
+    return DamoRecords(kdamonds=[], records=[damon_records]), None
+
 def parse_sort_bytes_ranges_input(bytes_ranges_input):
     try:
         ranges = [[_damo_fmt_str.text_to_bytes(start),

{damo-3.3.2 → damo-3.3.4}/src/damo/_damon.py

@@ -15,6 +15,50 @@ import _damo_fmt_str
 
 # Core data structures
 
+prep_action_set_pgidle = 'set_pgidle'
+
+prep_actions = [
+        prep_action_set_pgidle,
+        ]
+
+class DamonPrep:
+    prep_action = None
+
+    def __init__(self, prep_action=prep_action_set_pgidle):
+        if not prep_action in prep_actions:
+            raise Exception('wrong prep action (%s)' % prep_action)
+
+        self.prep_action = prep_action
+
+    def to_str(self, raw):
+        return self.prep_action
+
+    def __str__(self):
+        return self.to_str(False)
+
+    def __eq__(self, other):
+        return type(self) == type(other) and \
+                self.prep_action == other.prep_action
+
+    @classmethod
+    def from_kvpairs(cls, kv):
+        return DamonPrep(prep_action=kv['prep_action'])
+
+    def to_kvpairs(self, raw=False):
+        return collections.OrderedDict([
+            ('prep_action', self.prep_action),
+            ])
+
+damon_filter_type_anon = 'anon'
+damon_filter_type_memcg = 'memcg'
+damon_filter_type_pgidle_unset = 'pgidle_unset'
+
+damon_filter_types = [
+        damon_filter_type_anon,
+        damon_filter_type_memcg,
+        damon_filter_type_pgidle_unset,
+        ]
+
 class DamonFilter:
     filter_type = None  # anon, memcg
     matching = None
@@ -22,6 +66,8 @@ class DamonFilter:
     path = None
 
     def __init__(self, filter_type, matching, allow=False, path=None):
+        if not filter_type in damon_filter_types:
+            raise Exception('wrong damon filter type (%s)' % filter_type)
         self.filter_type = filter_type
         self.matching = _damo_fmt_str.text_to_bool(matching)
         self.allow = _damo_fmt_str.text_to_bool(allow)
@@ -36,10 +82,9 @@ class DamonFilter:
         if self.matching is False:
             words.append('non')
         words.append(self.filter_type)
-        if self.filter_type in ['anon']:
-            return ' '.join(words)
         if self.filter_type == 'memcg':
             return ' '.join(words + [self.path])
+        return ' '.join(words)
 
     def __str__(self):
         return self.to_str(False)
@@ -67,8 +112,9 @@ class DamonFilter:
 class DamonProbe:
     filters = None
     weight = None
+    preps = None
 
-    def __init__(self, filters, weight=None):
+    def __init__(self, filters, weight=None, preps=None):
         if type(filters) is not list:
             raise Exception('filters for DamonProbe() is not a list')
         self.filters = filters
@@ -77,9 +123,19 @@ class DamonProbe:
         if type(weight) is not int:
             raise Exception('weight is not an int')
         self.weight = weight
+        if preps is None:
+            preps = []
+        if type(preps) is not list:
+            raise Exception('preps is not a list')
+        self.preps = preps
 
     def to_str(self, raw):
-        words = [', '.join([f.to_str(raw) for f in self.filters])]
+        words = []
+        if len(self.preps) > 0:
+            words.append('prep: %s' % ', '.join([
+                prep.to_str(raw) for prep in self.preps]))
+            words.append(', filter: ')
+        words.append(', '.join([f.to_str(raw) for f in self.filters]))
         words.append(' (weight: %s)' % self.weight)
         return ''.join(words)
 
@@ -89,19 +145,26 @@ class DamonProbe:
     def __eq__(self, other):
         return type(self) == type(other) and \
                 self.filters == other.filters and \
-                self.weight == other.weigt
+                self.weight == other.weigt and \
+                self.preps == other.preps
 
     @classmethod
     def from_kvpairs(cls, kv):
+        if 'preps' in kv:
+            preps = [DamonPrep.from_kvpairs(pkv) for pkv in kv['preps']]
+        else:
+            preps = []
         return DamonProbe(
                 filters=[
                     DamonFilter.from_kvpairs(filter_kv)
                     for filter_kv in kv['filters']],
                 weight=kv.get('weight', None),
+                preps=preps,
                 )
 
     def to_kvpairs(self, raw=False):
         return collections.OrderedDict([
+            ('preps', [p.to_kvpairs(raw) for p in self.preps]),
             ('filters', [f.to_kvpairs(raw) for f in self.filters]),
             ('weight', self.weight),
             ])

{damo-3.3.2 → damo-3.3.4}/src/damo/_damon_args.py

@@ -752,6 +752,16 @@ def gen_assign_schemes(ctxs, args):
         scheme_idx += nr
     return None
 
+def probe_preps_for(preps_args):
+    preps = []
+    for prep_action in preps_args:
+        try:
+            prep = _damon.DamonPrep(prep_action=prep_action)
+        except Exception as e:
+            return None, 'prep creation fail (%s)' % e
+        preps.append(prep)
+    return preps, None
+
 def probe_filter_for(filter_arg_fields):
     fields = filter_arg_fields
     if len(fields) < 2:
@@ -790,13 +800,25 @@ def probe_filters_for(filters_args):
     return filters, None
 
 def probes_for(args):
+    preps, err = probe_preps_for(args.probe_prep)
+    if err is not None:
+        return None, err
+    if args.nr_probe_preps is None:
+        args.nr_probe_preps = [len(preps)]
+    if sum(args.nr_probe_preps) != len(preps):
+        return None, '--nr_probe_preps mismatches --probe_prep (%d != %d)' % (
+                sum(args.nr_probe_preps), len(preps))
+
     filters, err = probe_filters_for(args.probe_filter)
     if err is not None:
         return None, err
-    if len(filters) == 0:
+
+    if len(preps) == 0 and len(filters) == 0:
         return [], None
+
     probes = []
     filter_idx = 0
+
     if args.nr_probe_filters is None:
         args.nr_probe_filters = [len(args.probe_filter)]
     if sum(args.nr_probe_filters) != len(filters):
@@ -810,11 +832,21 @@ def probes_for(args):
                     % (len(args.probe_weight), len(args.nr_probe_filters))
     else:
         args.probe_weight = [0 for _ in args.nr_probe_filters]
-    for nr in args.nr_probe_filters:
+
+    nr_probes = len(args.nr_probe_filters)
+    for i in range(nr_probes - len(preps)):
+        args.nr_probe_preps.append(0)
+    prep_idx = 0
+
+    for probe_idx, nr in enumerate(args.nr_probe_filters):
+        preps_for_probe = preps[
+                prep_idx:prep_idx + args.nr_probe_preps[probe_idx]]
         filters_for_probe = filters[filter_idx:filter_idx + nr]
         weight = args.probe_weight[len(probes)]
         try:
-            probe = _damon.DamonProbe(filters=filters_for_probe, weight=weight)
+            probe = _damon.DamonProbe(
+                    preps=preps_for_probe, filters=filters_for_probe,
+                    weight=weight)
         except Exception as e:
             return None, 'probe creation fail (%s)' % e
         probes.append(probe)
@@ -947,6 +979,21 @@ def deduce_target_update_args(args):
             print('warning: override --ops by <deducible target> and --regions')
         args.ops = ['fvaddr']
 
+def update_args_defaults(args):
+    # having default=[] makes complicated global shared states.  Instantiate
+    # default values here.
+    for arg_name in [
+            'probe_prep', 'probe_filter', 'probe_weight', 'damos_action',
+            'damos_sz_region', 'damos_access_rate', 'damos_age',
+            'damos_apply_interval', 'damos_quotas', 'damos_quota_interval',
+            'damos_quota_time', 'damos_quota_space', 'damos_quota_weights',
+            'damos_quota_fail_charge_ratio', 'damos_quota_goal',
+            'damos_quota_goal_tuner', 'damos_nr_quota_goals', 'damos_filter',
+            'damos_nr_filters', 'damos_wmarks', 'damos_max_nr_snapshots',
+            ]:
+        if getattr(args, arg_name) is None:
+            setattr(args, arg_name, [])
+
 def warn_for(arg, feature):
     if _damo_sysinfo.damon_feature_available(feature):
         return
@@ -966,6 +1013,17 @@ def warn_unsupported_damon_features_for(args):
     if args.sample_primitives is not None:
         warn_for('--sample_primitives', 'sysfs/damon_sample_control')
 
+    if len(args.probe_prep) > 0:
+        warn_for('--probe_prep', 'sysfs/probe_preps')
+    for probe_filter_args in args.probe_filter:
+        if _damon.damon_filter_type_pgidle_unset in args.probe_filter:
+            warn_for(
+                    '--probe_filter of %s' %
+                    _damon.damon_filter_type_pgidle_unset,
+                    'sysfs/probe_type_pgidle_unset')
+
+    # mm.git
+
     if args.probe_weight != []:
         warn_for('--probe_weight', 'sysfs/probe_weights')
 
@@ -1158,6 +1216,7 @@ def warn_unsupported_damon_features_for(args):
                       ['interface/damon_debugfs', 'interface/damon_sysfs'])
 
 def evaluate_args(args):
+    update_args_defaults(args)
     warn_unsupported_damon_features_for(args)
 
     '''
@@ -1385,8 +1444,16 @@ def set_monitoring_damos_common_args(parser, hide_help=False):
                         help='monitoring operations set'
                         if not hide_help else argparse.SUPPRESS)
 
+    parser.add_argument('--probe_prep', action='append',
+                        metavar='<prep action>',
+                        help='data attribute monitoring probe prep'
+                        if not hide_help else argparse.SUPPRESS)
+    parser.add_argument('--nr_probe_preps', type=int, nargs='+',
+                        metavar='<integer>',
+                        help='number of preps for each probe (in order)'
+                        if not hide_help else argparse.SUPPRESS)
+
     parser.add_argument('--probe_filter', nargs='+', action='append',
-                        default=[],
                         metavar='<<allow|reject> [non] <type> [option]...>',
                         help='data attribute monitoring probe filter'
                         if not hide_help else argparse.SUPPRESS)
@@ -1395,7 +1462,7 @@ def set_monitoring_damos_common_args(parser, hide_help=False):
             help='number of filters for each probe (in order)'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument('--probe_weight', type=int, action='append',
-                        default=[], metavar='<int>',
+                        metavar='<int>',
                         help='data attribute probe weight'
                         if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
@@ -1456,7 +1523,7 @@ def set_monitoring_argparser(parser, hide_help=False):
 
 def set_damos_argparser(parser, hide_help):
     parser.add_argument('--damos_action', metavar='<action>', nargs='+',
-                        action='append', default=[],
+                        action='append',
                         help=' '.join([
                             'damos action to apply to the target regions.',
                             '<action> should be {%s}.' %
@@ -1467,21 +1534,21 @@ def set_damos_argparser(parser, hide_help):
                             'should also be given.'
                             ]))
     parser.add_argument('--damos_sz_region', metavar=('<min>', '<max>'),
-                        nargs=2, default=[], action='append',
+                        nargs=2, action='append',
                         help='min/max size of damos target regions (bytes)'
                         if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
-            '--damos_access_rate', metavar=('<min>', '<max>'),
-            nargs=2, default=[], action='append',
+            '--damos_access_rate', metavar=('<min>', '<max>'), nargs=2,
+            action='append',
             help='min/max access rate of damos target regions (percent)'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
             '--damos_age', metavar=('<min>', '<max>'), nargs=2,
-            default=[], action='append',
+            action='append',
             help='min/max age of damos target regions (microseconds)'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument('--damos_apply_interval', metavar='<microseconds>',
-                        action='append', default=[],
+                        action='append',
                         help='the apply interval for the scheme'
                         if not hide_help else argparse.SUPPRESS)
     damos_quotas_help = ' '.join([
@@ -1490,32 +1557,31 @@ def set_damos_argparser(parser, hide_help):
         '[<access rate priority weight> (permil)',
         '[<age priority weight> (permil)]]]]])'])
     parser.add_argument(
-            '--damos_quotas', default=[],
-            metavar='<quota parameter>', nargs='+', action='append',
+            '--damos_quotas', metavar='<quota parameter>', nargs='+',
+            action='append',
             help=damos_quotas_help if not hide_help else argparse.SUPPRESS)
-    parser.add_argument('--damos_quota_interval', default=[],
-                        metavar='<milliseconds>', action='append',
+    parser.add_argument('--damos_quota_interval', metavar='<milliseconds>',
+                        action='append',
                         help='quota reset interval'
                         if not hide_help else argparse.SUPPRESS)
-    parser.add_argument('--damos_quota_time', default=[],
-                        metavar='<milliseconds>', action='append',
+    parser.add_argument('--damos_quota_time', metavar='<milliseconds>',
+                        action='append',
                         help='time quota'
                         if not hide_help else argparse.SUPPRESS)
-    parser.add_argument('--damos_quota_space', default=[], metavar='<bytes>',
+    parser.add_argument('--damos_quota_space', metavar='<bytes>',
                         action='append', help='space quota'
                         if not hide_help else argparse.SUPPRESS)
-    parser.add_argument('--damos_quota_weights', default=[],
-                        metavar='<permil>', nargs=3, action='append',
+    parser.add_argument('--damos_quota_weights', metavar='<permil>', nargs=3,
+                        action='append',
                         help='quota\'s prioritization weights'
                         if not hide_help else argparse.SUPPRESS)
-    parser.add_argument('--damos_quota_fail_charge_ratio', default=[],
+    parser.add_argument('--damos_quota_fail_charge_ratio',
                         metavar=('<numerator>', '<denominator>'),
                         action='append', nargs=2,
                         help='quota failed regions charge ratio'
                         if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
             '--damos_quota_goal', nargs='+', action='append',
-            default=[],
             metavar='<metric or value>',
             help=' '.join([
                 'damos quota goal (<metric> <target value> [optional value]).',
@@ -1523,32 +1589,29 @@ def set_damos_argparser(parser, hide_help):
                 ','.join(_damon.qgoal_metrics)])
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
-            '--damos_quota_goal_tuner', action='append', default=[],
+            '--damos_quota_goal_tuner', action='append',
             help='quota\'s goal-based tuner')
     parser.add_argument(
-            '--damos_nr_quota_goals', type=int, nargs='+',
-            default=[], metavar='<integer>',
+            '--damos_nr_quota_goals', type=int, nargs='+', metavar='<integer>',
             help='number of quota goals for each scheme (in order)'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
             '--damos_filter', nargs='+', action='append',
-            default=[],
             metavar='<<allow|reject> [none] <type> [option]...>',
             help='damos filter' if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
-            '--damos_nr_filters', type=int, nargs='+', default=[],
-            metavar='<integer>',
+            '--damos_nr_filters', type=int, nargs='+', metavar='<integer>',
             help='number of filters for each scheme (in order)'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
-            '--damos_wmarks', nargs=5, action='append', default=[],
+            '--damos_wmarks', nargs=5, action='append',
             metavar=('<metric (none|free_mem_rate)>', '<interval (us)>',
                 '<high mark (permil)>', '<mid mark (permil)>',
                 '<low mark (permil)>'),
             help='damos watermarks'
             if not hide_help else argparse.SUPPRESS)
     parser.add_argument(
-            '--damos_max_nr_snapshots', action='append', default=[],
+            '--damos_max_nr_snapshots', action='append',
             help='damos max_nr_snapshots')
     parser.add_argument('--nr_schemes', metavar='<number>', nargs='+', type=int,
                         help='number of schemes for each context (in order)'

{damo-3.3.2 → damo-3.3.4}/src/damo/_damon_features.py

@@ -348,11 +348,27 @@ features_list = [
         DamonFeature(name='sysfs/attrs_monitoring',
                      upstream_status='merged in v7.2-rc1 (45c49d9fd6089)',
                      upstreamed_version='7.2'),
+        DamonFeature(name='sysfs/probe_type_anon',
+                     upstream_status='merged in v7.2-rc1 (8caba1448278)',
+                     upstreamed_version='7.2'),
+        DamonFeature(name='sysfs/probe_type_memcg',
+                     upstream_status='merged in v7.2-rc1 (c71f8e13462d)',
+                     upstreamed_version='7.2'),
         DamonFeature(name='trace/damon_region_aggregated',
                      upstream_status='merged in v7.2-rc1 (45c49d9fd6089)',
                      upstreamed_version='7.2'),
 
         DamonFeature(name='sysfs/probe_weights',
+                     upstream_status='merged into mm.git',
+                     upstreamed_version='none'),
+
+        DamonFeature(name='sysfs/probe_preps',
+                     upstream_status='hacking on damon/next',
+                     upstreamed_version='none'),
+        DamonFeature(name='sysfs/probe_prep_set_pgidle',
+                     upstream_status='hacking on damon/next',
+                     upstreamed_version='none'),
+        DamonFeature(name='sysfs/probe_type_pgidle_unset',
                      upstream_status='hacking on damon/next',
                      upstreamed_version='none'),
 

{damo-3.3.2 → damo-3.3.4}/src/damo/_damon_sysfs.py

@@ -570,6 +570,24 @@ def write_ops_attrs_dir(dir_path, ops_attrs):
     if err is not None:
         return err
 
+def write_probe_prep_dir(dir_path, prep):
+    err = _damo_fs.write_file(
+            os.path.join(dir_path, 'prep_action'), prep.prep_action)
+    if err is not None:
+        return err
+    return None
+
+def write_probe_preps_dir(dir_path, preps):
+    err = ensure_nr_file_for(os.path.join(dir_path, 'nr_preps'), preps)
+    if err is not None:
+        return err
+
+    for idx, prep in enumerate(preps):
+        err = write_probe_prep_dir(os.path.join(dir_path, '%d' % idx), prep)
+        if err is not None:
+            return err
+    return None
+
 def write_probe_filter_dir(dir_path, filter):
     err = _damo_fs.write_file(
             os.path.join(dir_path, 'type'), filter.filter_type)
@@ -590,6 +608,12 @@ def write_probe_filter_dir(dir_path, filter):
     return None
 
 def write_probe_dir(dir_path, probe):
+    preps_dir = os.path.join(dir_path, 'preps')
+    if os.path.isdir(preps_dir):
+        err = write_probe_preps_dir(preps_dir, probe.preps)
+        if err is not None:
+            return err
+
     filters_dir = os.path.join(dir_path, 'filters')
     err = ensure_nr_file_for(os.path.join(filters_dir, 'nr_filters'),
                              probe.filters)
@@ -1056,16 +1080,27 @@ def files_content_to_target(files_content):
     regions = files_content_to_regions(files_content['regions'])
     return _damon.DamonTarget(pid, regions, obsolete=obsolete)
 
+def files_content_to_probe_preps(files_content):
+    preps = []
+    for kv in number_sorted_dirs(files_content):
+        preps.append(_damon.DamonPrep(prep_action=kv['prep_action'].strip()))
+    return preps
+
 def files_content_to_damon_filter(files_content):
     path = None
     if 'path' in files_content:
-        files_content.get('path').strip()
+        path = files_content['path'].strip()
     return _damon.DamonFilter(
         filter_type=files_content['type'].strip(),
         matching=files_content['matching'].strip(),
         allow=files_content['allow'].strip(), path=path)
 
 def files_content_to_probe(files_content):
+    if 'preps' in files_content:
+        preps = files_content_to_probe_preps(files_content['preps'])
+    else:
+        preps = []
+
     filters_content = files_content['filters']
     filters = []
     for kv in number_sorted_dirs(filters_content):
@@ -1075,7 +1110,7 @@ def files_content_to_probe(files_content):
         weight = int(files_content['weight'])
     else:
         weight = None
-    return _damon.DamonProbe(filters=filters, weight=weight)
+    return _damon.DamonProbe(preps=preps, filters=filters, weight=weight)
 
 def files_content_to_probes(files_content):
     probes = []
@@ -1450,6 +1485,8 @@ def mk_feature_supports_map():
     if os.path.isdir(
             os.path.join(ctx_dir_of(0, 0), 'monitoring_attrs', 'probes')):
         supports_map['sysfs/attrs_monitoring'] = True
+        supports_map['sysfs/probe_type_anon'] = True
+        supports_map['sysfs/probe_type_memcg'] = True
 
 
     if supports_map['sysfs/attrs_monitoring'] is True:
@@ -1473,6 +1510,11 @@ def mk_feature_supports_map():
         if os.path.isfile(os.path.join(probe_dir, 'weight')):
             supports_map['sysfs/probe_weights'] = True
 
+        if os.path.isdir(os.path.join(probe_dir, 'preps')):
+            supports_map['sysfs/probe_preps'] = True
+            supports_map['sysfs/probe_prep_set_pgidle'] = True
+            supports_map['sysfs/probe_type_pgidle_unset'] = True
+
     if os.path.isdir(os.path.join(ctx_dir_of(0, 0), 'operations_attrs')):
         supports_map['sysfs/ops_attrs'] = True
 

{damo-3.3.2 → damo-3.3.4}/src/damo/damo_report_attrs.py

@@ -339,15 +339,14 @@ def max_access_hz(intervals):
 
 def snapshot_intervals_str(snapshot, record, fmt):
     intervals = record.intervals
-    aggr_sample_ratio = intervals.aggr / intervals.sample
+    aggr_sample_ratio = intervals.aggr // intervals.sample
     snapshot_aggr_us = infer_aggr_time_us(snapshot, record)
     snapshot_sample_us = snapshot_aggr_us / aggr_sample_ratio
-    max_access_hz = aggr_sample_ratio / (snapshot_aggr_us / 1000000)
 
-    return 'sample %s aggr %s (max access hz %s)' % (
+    return 'sample %s aggr %s (max probe hits %s)' % (
         _damo_fmt_str.format_time_us(snapshot_sample_us, fmt.raw_number),
         _damo_fmt_str.format_time_us(snapshot_aggr_us, fmt.raw_number),
-        _damo_fmt_str.format_hz(max_access_hz, fmt.raw_number, no_suffix=True))
+        _damo_fmt_str.format_nr(aggr_sample_ratio, fmt.raw_number))
 
 def estimated_mem_bw(snapshot, record, fmt, filter_passed_only=False):
     access_bytes = 0
@@ -768,95 +767,6 @@ def df_passed_heatmap_str(snapshot, record, fmt):
                 total_bytes=None),
             record, fmt)
 
-def rescale(val, orig_scale_minmax, new_scale_minmax, logscale=True):
-    '''Return a value in new scale
-
-    Parameters
-    ----------
-    val : int, float
-        The value to rescale
-    orig_scale_minmax : list
-        min/max values of original scale
-    new_scale_minmax : list
-        min/max values of new scale
-    logscale : bool
-        whether to use logscale (True) or linearscale (False)
-
-    Returns
-    -------
-    float
-        The rescaled value
-    '''
-
-    if logscale:
-        log_val = math.log(val, 2) if val > 0 else 0
-        log_minmax = [math.log(v, 2) if v > 0 else 0
-                for v in orig_scale_minmax]
-        val = log_val
-        orig_scale_minmax = log_minmax
-    orig_length = orig_scale_minmax[1] - orig_scale_minmax[0]
-    new_length = new_scale_minmax[1] - new_scale_minmax[0]
-    ratio = new_length / orig_length if orig_length > 0 else 1
-    return (val - orig_scale_minmax[0]) * ratio + new_scale_minmax[0]
-
-class BoxValue:
-    val = None
-    val_min_max = None      # list of min/max
-    display_min_max = None  # list of min/max
-    display_logscale = None # bool
-
-    def __init__(self, val, val_min_max, display_min_max, display_logscale):
-        self.val = val
-        self.val_min_max = val_min_max
-        self.display_min_max = display_min_max
-        self.display_logscale = display_logscale
-
-    def display_value(self):
-        return rescale(self.val, self.val_min_max, self.display_min_max,
-                self.display_logscale)
-
-class ColoredBox:
-    # BoxValue
-    length = None
-    color = None
-    height = None
-
-    colorset = None
-    horizontal_align = None
-
-    def __init__(self, length, horizontal_align, color, colorset, height):
-        self.length = length
-        self.color = color
-        self.horizontal_align = horizontal_align
-        self.colorset = colorset
-        self.height = height
-
-    def __str__(self):
-        length = int(self.length.display_value())
-
-        if self.height.val != None:
-            height = int(self.height.display_value())
-        else:
-            self.height.display_min, self.height.display_max = [1, 1]
-            height = 1
-
-        if type(self.color.val) == str:
-            row = '%s' % (self.color.val * length)
-        else:
-            color_level = int(self.color.display_value())
-            row = '%s' % _damo_ascii_color.colored(
-                    ('%d' % color_level) * length, self.colorset, color_level)
-        row = '|%s|' % row
-
-        if self.horizontal_align == 'right':
-            indent = ' ' * (self.length.display_min_max[1] - length)
-            row = indent + row
-
-        box = '\n'.join([row] * height)
-        if self.height.display_min_max[1] > 1:
-            box += '\n'
-        return box
-
 def apply_min_chars(min_chars, field_name, txt):
     # min_chars: [[<field name>, <number of min chars>]...]
     for name, nr in min_chars:

{damo-3.3.2 → damo-3.3.4}/src/damo/damo_version.py

@@ -3,7 +3,7 @@
 import os
 import subprocess
 
-__version__ = '3.3.2'
+__version__ = '3.3.4'
 
 def get_release_version():
     return __version__

{damo-3.3.2 → damo-3.3.4}/src/damo.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: damo
-Version: 3.3.2
+Version: 3.3.4
 Summary: DAMON user-space tool
 Home-page: https://github.com/damonitor/damo
 Author: SeongJae Park
@@ -71,7 +71,7 @@ The second and last commands will show the access pattern of your workload,
 like below:
 
 ![masim_stairs_snapshot](images/masim_stairs_snapshot.png)
-![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.2/images/masim_stairs_heatmap_ascii.png)
+![masim stairs heatmap in ascii](https://raw.githubusercontent.com/damonitor/damo/v3.3.4/images/masim_stairs_heatmap_ascii.png)
 
 
 FAQs
@@ -90,7 +90,7 @@ Where can I get more detailed usage?
 ------------------------------------
 
 The below sections provide quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 What does the version numbers mean?
@@ -115,7 +115,7 @@ We try our best to make `damo` stable and doesn't introduce regressions to
 users.  However, nothing goes forever.  Sometimes, some features will be
 deprecated.  Some features will have longer support more than others.
 
-In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) and not explicitly
+In short, features that documented on [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) and not explicitly
 marked as experimental will be better supported, and provides at least three
 months of deprecation grace period.  Within the grace period, users can ask
 extension of the support.  Even after the grace period, please reach out to
@@ -138,7 +138,7 @@ Quick Intro for Major Features
 ==============================
 
 Below are quick introductions for `damo`'s major features.
-For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.2/USAGE.md) file.
+For more detailed usage, please refer to [USAGE.md](https://github.com/damonitor/damo/blob/v3.3.4/USAGE.md) file.
 
 
 Snapshot Data Access Pattern