| “Token” refers to any sequence of letters found in posts between text delimiters. While most tokens are words in real languages or variants, there are also many special purpose tokens such as emojis, XML tags, single letter symbols, etc.
Currently Social Standards back-end database contains over 125 million (!) unique tokens. |
