Skip to main content

tokenize (Stream Processor)

Tokenize the map and return each key, value as new attributes in events


map:tokenize(<OBJECT> map)
map:tokenize(<OBJECT> map, <OBJECT> ...)

Query Parameters

NameDescriptionDefault ValuePossible Data TypesOptionalDynamic
mapHash map containing key value pairsOBJECTNoYes

Extra Return Attributes

NameDescriptionPossible Types
keyKey of an entry consisted in the mapOBJECT
valueValue of an entry consisted in the map. If more than one map is given, then an Array List of values from each map is returned for the value attribute.OBJECT

Example 1

CREATE STREAM StockStream(symbol STRING, price FLOAT);

@info(name = 'collectSymbolPrice')
SELECT map:collect(symbol, price) AS symbolPriceMap

@info(name = 'tokenizeSymbolPrice')
INSERT INTO SymbolStream
SELECT key, value
FROM TempStream#map:tokenize(customMap);

In the collectSymbolPrice query, the map:collect(symbol, price) function is used to create a symbolPriceMap that collects symbol and price pairs from two events in the StockStream based on a tumbling length window of 2.

In the tokenizeSymbolPrice query, the map:tokenize(customMap) function is applied on the TempStream to tokenize the symbolPriceMap and generate two separate events in the SymbolStream with the key and value attributes set to the symbol names and their corresponding prices.