Docs dense alert aggregation

docs/alert_aggregator.md

@@ -71,6 +71,7 @@ The alert aggregator are numbered from simplest to most complex, and the sidebar
 |---|---|---|
 | 00 | [Basic Concat Aggregator](alert_aggregators/basic_concatenation.md) | Concat a set of alerts. |
 | 01 | [LogID Count Aggregator](alert_aggregators/logid_count.md) | Count the amount of times a logID is in alerts. |
+| 02 | [Density Cluster](alert_aggregators/density_cluster.md) | Add description... ADD_DENSITY_CLUSTER|
 
 
 ### Common parameters (all alert aggregators)

docs/alert_aggregators/density_cluster.md

@@ -0,0 +1,48 @@
+# Density Cluster Alert Aggregation
+
+add stuff, dont remove start and end arguments.
+
+## In/out
+
+Input and output schemas in the pipeline
+
+## In/out
+
+|            | Schema                     | Description        |
+|------------|----------------------------|--------------------|
+| **Input**  | [DetectorSchema](../schemas.md)    | Alerts from detectors  |
+| **Output** | [AggregateSchema](../schemas.md) | Aggregated alerts    |
+
+## Description
+
+
+## At a glance
+
+| Learns from training data | Auto-configuration | Needs `events` | Federation |
+|---|---|---|---|
+| ❌ | ❌ | ❌ | ❌ |
+
+
+## Example
+
+```python
+--8<-- "docs/examples/alert_aggregators/density_cluster.py:example_1"
+```
+
+## Configuration file
+
+The configuration used by the example above. It sets only what this use case needs; every other parameter keeps its default (see [Configuration arguments](#configuration-arguments)).
+
+```yaml
+--8<-- "docs/examples/alert_aggregators/density_cluster.yaml"
+```
+
+Load it with `yaml.safe_load` and pass the dict as `config=`, as in the example. The key under `alert_aggregators:` must match the aggregator's `name`.
+
+## Configuration arguments
+
+All parameters this alert aggregator accepts, grouped by the YAML block they go in. **Scope** tells whether a parameter is `specific` to this aggregator or `shared` with other alert aggregators.
+
+<!-- Start arguments -->
+
+<!-- End arguments -->

docs/alert_aggregators/logid_count.md

@@ -41,20 +41,21 @@ All parameters this detector accepts, grouped by the YAML block they go in. **Sc
 <!-- Start arguments -->
 ??? note "Top level"
 
-| Field | Type | Default | Scope | Description |
-|---|---|---|---|---|
-| `method_type` | string | basic_concat_aggregator | shared | Indicates what type of method it is. |
-| `auto_config` | boolean | False | shared | Runs the configuration step before the training process. |
+    | Field | Type | Default | Scope | Description |
+    |---|---|---|---|---|
+    | `method_type` | string | logid_count_aggregator | shared | Indicates what type of method it is. |
+    | `auto_config` | boolean | False | shared | Runs the configuration step before the training process. |
 
 ???+ note "params"
 
-| Field | Type | Default | Scope | Description |
-|---|---|---|---|---|
-| `buffer_size` | integer | 3 | specific | Buffer size use in the aggregation. |
-| `start_id` | integer | 10 | shared | Number used to start the unique ID generator. |
-| `data_use_training` | integer, null | None | shared | Data used for training, if None, training is not done. |
-| `data_use_configure` | integer, null | None | shared | Data used for configuration, if None, configuration is not done. |
-| `use_config_data_as_training` | boolean | True | shared | Combine the configured data in the training process if True. |
-| `train_buffer_max_records` | integer | 100000 | shared | Configure records kept in memory for training (use_config_data_as_training) before the buffer spills to Parquet files on disk, in parts of this many records. |
-| `train_buffer_dir` | string, null | None | shared | Local directory for the spilled training buffer. None uses the system temp directory (TMPDIR). Each spill goes to a private detectmate-train-* directory, removed after training reads it; a killed process leaves it behind. |
+    | Field | Type | Default | Scope | Description |
+    |---|---|---|---|---|
+    | `max_count` | integer | 4 | specific | Max count of alerts in a specific logID. |
+    | `max_memory` | integer | 5000 | specific | Max number of logIDs store in the count memory. |
+    | `start_id` | integer | 10 | shared | Number used to start the unique ID generator. |
+    | `data_use_training` | integer, null | None | shared | Number of logs used for training. None skips training. |
+    | `data_use_configure` | integer, null | None | shared | Number of logs used for the configuration step. None skips it. |
+    | `use_config_data_as_training` | boolean | True | shared | Combine the configured data in the training process if True. |
+    | `train_buffer_max_records` | integer | 100000 | shared | Configure records kept in memory for training (use_config_data_as_training) before the buffer spills to Parquet files on disk, in parts of this many records. |
+    | `train_buffer_dir` | string, null | None | shared | Local directory for the spilled training buffer. None uses the system temp directory (TMPDIR). Each spill goes to a private detectmate-train-* directory, removed after training reads it; a killed process leaves it behind. |
 <!-- End arguments -->

docs/examples/alert_aggregators/density_cluster.py

@@ -0,0 +1,9 @@
+# --8<-- [start:example_1]
+
+print("Hello World")
+
+#  Check BasicConcat of LogID count documentation to add your example here
+#  Use the example config -> density_cluster.yaml from same folder
+#  ADD_DENSITY_CLUSTER
+
+# --8<-- [end:example_1]

docs/examples/alert_aggregators/density_cluster.yaml

@@ -0,0 +1,3 @@
+alert_aggregators:
+    NAME:
+        method_type: "ADD_DENSITY_CLUSTER"

docs/examples/config/update.py

@@ -28,6 +28,7 @@
 from detectmatelibrary.common.parser import CoreParserConfig
 from detectmatelibrary.common.alert_aggregator import CoreAlertAggregatorConfig
 from detectmatelibrary.alert_aggregation.basic_concat import BasicConcatAggregationConfig
+from detectmatelibrary.alert_aggregation.logid_count import LogIDCountAggegationConfig
 from detectmatelibrary.base_detectors import TrackerDetectorConfig
 from detectmatelibrary.base_detectors import VariableDetectorConfig
 from detectmatelibrary.base_detectors import DeepLearningDetectorConfig
@@ -271,7 +272,18 @@ def update_shared_args_aggregation(doc_path: str) -> None:
         BasicConcatAggregationConfig(),
         CoreAlertAggregatorConfig,
         "docs/alert_aggregators/basic_concatenation.md"
-    )
+    ),
+    (
+        LogIDCountAggegationConfig(),
+        CoreAlertAggregatorConfig,
+        "docs/alert_aggregators/logid_count.md"
+    ),
+    # ADD_DENSITY_CLUSTER
+    # (
+    #   DensityClusteringAggregationConfig,
+    #   CoreAlertAggregatorConfig,
+    #   "docs/alert_aggregators/logid_count.md"
+    # ),
 ]
 
 update_shared_args_detectors("docs/detectors.md")

mkdocs.yml

@@ -68,6 +68,7 @@ nav:
     - Alert Aggregation methods:
       - 00_Basic Concat: alert_aggregators/basic_concatenation.md
       - 01_LogID Count: alert_aggregators/logid_count.md
+      - 02_Density Cluster: alert_aggregators/density_cluster.md
     - Helpers:
       - FromTo: helper/from_to.md
       - Select Component: helper/select_component.md