Natural language processing

Tokenizer for StackOverflow Posts

We developed two tokenizers on Stack Overflow posts. One is based on regular expression, and the other is based on Conditional Random Field (CRF).