Skip to content

Task Definition REST API

A task is the single unit of work performed on event data. Each task has the following built-in functions that can perform processing on an event:

  • APPLY_RULES: Apply predefined rules (per event and/or correlated/aggregated) to streaming events. The input must be JSON.
  • EXTRACT: Extract any event input with provided Regular Expression defition (named groups). The output is JSON.
  • FILTER: Filter an event (keep or drop) based on PDL or regex definition. For PDL, the input must be JSON.
  • OUTPUT_FIELD: Outputs the value of a given field. The input must be JSON and the output is String representation of the selected field value.
  • PARSE_CSV: Parse input CSV event into JSON.
  • PARSE_KV: Parse input key-value pairs event into JSON.
  • PDL_EXPRESSION: Allows event data transformation and enrichment via PDL expressions. The input must be JSON.
  • TIMESTAMP: Define a field from within the event data (JSON formatted) to use as the timestamp.

Important Notes

  • This section describes the function definition of a task, for generic API call, please refer to Tasks REST API.
  • Padas Engine verifies requests against the JSON schema defined as Padas Task Schema.
  • For description of each definition field and default values please refer to Stream Configuration

Task Definition Examples

APPLY_RULES Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "APPLY_RULES",
  "definition": {
    "rules": [
      "rule1",
      "rule2"
      ],
    "matchAll": false
  }
}

EXTRACT Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "EXTRACT",
  "definition": {
    "field": "somefield",
    "regex": "someregex text here",
    "keepRaw": false
  }
}

FILTER Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "FILTER",
  "definition": {
    "type": "regex",
    "action": "drop",
    "value": "regex goes here"
  }
}

OUTPUT_FIELD Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "OUTPUT_FIELD",
  "definition": {
    "field": "someFieldName"
  }
}

PARSE_CSV Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "PARSE_CSV",
  "definition": {
    "fieldNames": "field1,field2,field3",
    "delimeter":"|"
  }
}

PARSE_KV Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "PARSE_KV",
  "definition": {
    "delimeter":":"
  }
}

PDL_EXPRESSION Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "PDL_EXPRESSION",
  "definition": {
    "pdl" : "field1=\"value1\" AND field2 > 100 | eval field3=if(field2 < 400, 0, 1)"
  }
}

TIMESTAMP Definition Example

{
  "id": 1,
  "name": "MyTask Name",
  "description": "MyTask description goes here.",
  "function": "TIMESTAMP",
  "definition": {
    "field": "somefield",
    "format": "%H%M%S"
  }
}