Python NLTK | tokenize.regexp()
Last Updated :
07 Jun, 2019
With the help of NLTK tokenize.regexp()
module, we are able to extract the tokens from string by using regular expression with RegexpTokenizer()
method.
Syntax : tokenize.RegexpTokenizer()
Return : Return array of tokens using regular expression
Example #1 :
In this example we are using RegexpTokenizer()
method to extract the stream of tokens with the help of regular expressions.
from nltk.tokenize import RegexpTokenizer
tk = RegexpTokenizer( '\s+' , gaps = True )
gfg = "I love Python"
geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘I’, ‘love’, ‘Python’]
Example #2 :
from nltk.tokenize import RegexpTokenizer
tk = RegexpTokenizer( '\s+' , gaps = True )
gfg = "Geeks for Geeks"
geek = tk.tokenize(gfg)
print (geek)
|
Output :
[‘Geeks’, ‘for’, ‘Geeks’]
Share your thoughts in the comments
Please Login to comment...