Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 34 additions & 2 deletions documentation/docs/tirith-reference/evaluators.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ site_name: Tirith
slug: evaluators/
---

Every evaluator in a Tirith policy applies a **condition** to one or more values extracted by a provider. This page is the complete reference for all 13 condition types, including exactly how each one treats strings, numbers, lists, dictionaries, and `null`.
Every evaluator in a Tirith policy applies a **condition** to one or more values extracted by a provider. This page is the complete reference for all 15 condition types, including exactly how each one treats strings, numbers, lists, dictionaries, and `null`.

## Anatomy of a condition

Expand All @@ -33,7 +33,7 @@ The `condition` object accepts three keys:

| Key | Required | Meaning |
| --- | --- | --- |
| `type` | yes | One of the 13 evaluator names listed below. The name is case-sensitive. |
| `type` | yes | One of the 15 evaluator names listed below. The name is case-sensitive. |
| `value` | yes, except for `IsEmpty` and `IsNotEmpty`, which ignore it | The value the extracted input is compared against. Any JSON type is accepted; each evaluator defines which types it supports. |
| `error_tolerance` | no (default `0`) | The maximum provider error severity that is *skipped* instead of failing the evaluator. See [Error Tolerance](../tirith-policies/tirith-policy-error-tolerance.md). |

Expand Down Expand Up @@ -74,6 +74,8 @@ Without `--fail-on-error`, the process exits **0** regardless of the verdict; th
| [`IsNotEmpty`](#isnotempty) | input is a **non-empty string, list, or dictionary** | ignored | returns false for numbers, booleans, and `null` |
| [`RegexMatch`](#regexmatch) | the pattern is found in the input | string (regular expression) | returns false for non-string/list/dict input; invalid pattern returns false with the regex error message |
| [`NotRegexMatch`](#notregexmatch) | the pattern is **not** found in the input | string (regular expression) | returns false for non-string/list/dict input; invalid pattern returns false with the regex error message |
| [`StartsWith`](#startswith) | the input starts with the condition string | string | returns false for non-string input and names its type |
| [`EndsWith`](#endswith) | the input ends with the condition string | string | returns false for non-string input and names its type |
| [`ContainedIn`](#containedin) | the input value occurs inside the condition value | string, list, or dictionary | returns false with an "unsupported data type" message |
| [`NotContainedIn`](#notcontainedin) | the input value does **not** occur inside the condition value | string, list, or dictionary | returns false (not true) with an "unsupported data type" message |
| [`Contains`](#contains) | the condition value occurs inside the input value | any JSON (input must be string, list, or dictionary) | returns false with an "unsupported data type" message |
Expand Down Expand Up @@ -228,6 +230,36 @@ If the input is an unsupported data type (like a number or boolean), the check *
| `"us-east-1"` | fails |
| `42` (against pattern `"4"`) | fails — numbers are not coerced |

## StartsWith

Passes when the input string starts with `condition.value`. Matching is case-sensitive; an empty condition string matches every string. Non-string input returns false with a message naming its type.

```json
"condition": { "type": "StartsWith", "value": "acme-" }
```

| Input value | Result |
| --- | --- |
| `"acme-bucket"` | passes |
| `"other-bucket"` | fails |
| `"anything"` with condition value `""` | passes |
| `42` | fails — message names `int` |

## EndsWith

Passes when the input string ends with `condition.value`. Matching is case-sensitive; an empty condition string matches every string. Non-string input returns false with a message naming its type.

```json
"condition": { "type": "EndsWith", "value": "-hardened" }
```

| Input value | Result |
| --- | --- |
| `"ami-hardened"` | passes |
| `"ami-default"` | fails |
| `"anything"` with condition value `""` | passes |
| `42` | fails — message names `int` |

## ContainedIn

Asks: **is the input value inside `condition.value`?** The condition value is the container. Which check runs depends on the types of both sides:
Expand Down
4 changes: 4 additions & 0 deletions src/tirith/core/evaluators/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
from .not_equals import NotEquals
from .not_contained_in import NotContainedIn
from .not_contains import NotContains
from .starts_with import StartsWith
from .ends_with import EndsWith

EVALUATORS_DICT: Dict[str, Type[BaseEvaluator]] = {
"ContainedIn": ContainedIn,
Expand All @@ -31,4 +33,6 @@
"NotEquals": NotEquals,
"NotContainedIn": NotContainedIn,
"NotContains": NotContains,
"StartsWith": StartsWith,
"EndsWith": EndsWith,
}
14 changes: 14 additions & 0 deletions src/tirith/core/evaluators/ends_with.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
from .base_evaluator import BaseEvaluator


class EndsWith(BaseEvaluator):
def evaluate(self, evaluator_input, evaluator_data):
if not isinstance(evaluator_input, str):
return {
"passed": False,
"message": "Expected input to be a string, got {}".format(type(evaluator_input).__name__),
}

passed = evaluator_input.endswith(evaluator_data)
message = ("{} ends with {}" if passed else "{} does not end with {}").format(evaluator_input, evaluator_data)
return {"passed": passed, "message": message}
16 changes: 16 additions & 0 deletions src/tirith/core/evaluators/starts_with.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
from .base_evaluator import BaseEvaluator


class StartsWith(BaseEvaluator):
def evaluate(self, evaluator_input, evaluator_data):
if not isinstance(evaluator_input, str):
return {
"passed": False,
"message": "Expected input to be a string, got {}".format(type(evaluator_input).__name__),
}

passed = evaluator_input.startswith(evaluator_data)
message = ("{} starts with {}" if passed else "{} does not start with {}").format(
evaluator_input, evaluator_data
)
return {"passed": passed, "message": message}
2 changes: 2 additions & 0 deletions src/tirith/tui/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -296,6 +296,8 @@ class EvaluatorInfo(NamedTuple):
"IsNotEmpty": EvaluatorInfo("IsNotEmpty", "Value is not empty.", "none"),
"RegexMatch": EvaluatorInfo("RegexMatch", "Value matches this regular expression.", "regex"),
"NotRegexMatch": EvaluatorInfo("NotRegexMatch", "Value does not match this regular expression.", "regex"),
"StartsWith": EvaluatorInfo("StartsWith", "Value starts with this string.", "scalar"),
"EndsWith": EvaluatorInfo("EndsWith", "Value ends with this string.", "scalar"),
}


Expand Down
37 changes: 37 additions & 0 deletions tests/core/evaluators/test_ends_with.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
from tirith.core.evaluators import EndsWith

evaluator = EndsWith()


def test_ends_with_passes():
assert evaluator.evaluate("ami-hardened", "-hardened") == {
"passed": True,
"message": "ami-hardened ends with -hardened",
}


def test_ends_with_fails():
assert evaluator.evaluate("ami-default", "-hardened") == {
"passed": False,
"message": "ami-default does not end with -hardened",
}


def test_ends_with_empty_suffix():
assert evaluator.evaluate("anything", "") == {
"passed": True,
"message": "anything ends with ",
}


def test_ends_with_empty_input():
assert evaluator.evaluate("", "-hardened") == {
"passed": False,
"message": " does not end with -hardened",
}


def test_ends_with_non_string_input_names_type():
result = evaluator.evaluate(42, "-hardened")
assert result["passed"] is False
assert "int" in result["message"]
37 changes: 37 additions & 0 deletions tests/core/evaluators/test_starts_with.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
from tirith.core.evaluators import StartsWith

evaluator = StartsWith()


def test_starts_with_passes():
assert evaluator.evaluate("acme-bucket", "acme-") == {
"passed": True,
"message": "acme-bucket starts with acme-",
}


def test_starts_with_fails():
assert evaluator.evaluate("other-bucket", "acme-") == {
"passed": False,
"message": "other-bucket does not start with acme-",
}


def test_starts_with_empty_prefix():
assert evaluator.evaluate("anything", "") == {
"passed": True,
"message": "anything starts with ",
}


def test_starts_with_empty_input():
assert evaluator.evaluate("", "acme-") == {
"passed": False,
"message": " does not start with acme-",
}


def test_starts_with_non_string_input_names_type():
result = evaluator.evaluate(42, "acme-")
assert result["passed"] is False
assert "int" in result["message"]
Loading