Tokenizing and POS tagging in Python from CSV file

295
September 02, 2017, at 05:57 AM

I am a newbie in Python and would like to do POS tagging after importing csv file from my local machine. I looked up some resources from online and found that the following code works.

text = 'Senator Elizabeth Warren from Massachusetts announced her support of 
Social Security in Washington, D.C. on Tuesday. Warren joined other 
Democrats in support.'  
import nltk
from nltk import tokenize
sentences = tokenize.sent_tokenize(text)
sentences
from nltk.tokenize import TreebankWordTokenizer
texttokens = []
for sent in sentences:
 texttokens.append(TreebankWordTokenizer().tokenize(sent))
texttokens
from nltk.tag import pos_tag
taggedsentences = []
for sentencetokens in texttokens:
 taggedsentences.append(pos_tag(sentencetokens))
taggedsentences
print(taggedsentences)

Since I printed it, the result from the code above looks like this.

[[('Senator', 'NNP'), ('Elizabeth', 'NNP'), ('Warren', 'NNP'), ('from', 
'IN'), ('Massachusetts', 'NNP'), ('announced', 'VBD'), ('her', 'PRP$'), 
('support', 'NN'), ('of', 'IN'), ('Social', 'NNP'), ('Security', 'NNP'), 
('in', 'IN'), ('Washington', 'NNP'), (',', ','), ('D.C.', 'NNP'), ('on', 
'IN'), ('Tuesday', 'NNP'), ('.', '.')], [('Warren', 'NNP'), ('joined', 
'VBD'), ('other', 'JJ'), ('Democrats', 'NNPS'), ('in', 'IN'), ('support', 
'NN'), ('.', '.')]]

This is a desirable result that I would like to get, but I would like to get the result after importing csv file which contains several rows (in each row, there are several sentences.). For example, the csv file looks like this:

---------------------------------------------------------------
I like this product. This product is beautiful. I love it. 
---------------------------------------------------------------
This product is awesome. It have many convenient features.
---------------------------------------------------------------
I went this restaurant three days ago. The food is too bad.
---------------------------------------------------------------

In the end, I would like to save the desirable pos tagging results that I displayed above after importing the csv file. I would like to save (write) the (pos tagged) each sentence in each row as a csv format.

Two formats might be possible. First one might be as follows (no header, each (pos tagged) sentence in one row).

----------------------------------------------------------------------------
[[('I', 'PRON'), ('like', 'VBD'), ('this', 'PRON'), ('product', 'NN')]]
----------------------------------------------------------------------------
[[('This', 'PRON'), ('product', 'NN'), ('is', 'VERB'), ('beautiful', 'ADJ')]]
---------------------------------------------------------------------------
[[('I', 'PRON'), ('love', 'VERB'), ('it', 'PRON')]]
----------------------------------------------------------------------------
...

The second format might look like this (no header, each set of token and pos tagger saved in one cell):

----------------------------------------------------------------------------
('I', 'PRON')    | ('like', 'VBD')   | ('this', 'PRON') | ('product', 'NN')
----------------------------------------------------------------------------
('This', 'PRON') | ('product', 'NN') | ('is', 'VERB')   | ('beautiful', 'ADJ')
---------------------------------------------------------------------------
('I', 'PRON')    | ('love', 'VERB')  | ('it', 'PRON')   |
----------------------------------------------------------------------------
...

I prefer the second format to the first one.

The python code that I wrote here perfectly works but I would like to do the same thing for csv file and in the end save it in my local machine.

Final purpose of doing this is that I would like to extract only noun types of words (e.g., NN, NNP) from the sentences.

Can somebody help me how to fix the python code?

Answer 1

Please refer to the question already answered here. You can just do some tagging to filter out just the Nouns as described in the post.SO Link

READ ALSO
Using a compiled Python shell in Linux

Using a compiled Python shell in Linux

Is it possible to use a "more complex" shell than just a single command shell? We have written a python shell that is a command loop, and it works fine in /etc/passwd like this:

429
Saving scraped documents in two sheets in an excel file

Saving scraped documents in two sheets in an excel file

I've created a scraper which is supposed to parse some documents from a webpage and save it to an excel file creating two sheetsHowever, when I run it, I can see that It only saves the documents of last link in a single sheet whereas there should be two sheets...

271
Extract all kind of objects from a pdf file

Extract all kind of objects from a pdf file

I am looking for a Linux tool that can extract all the files (images, streams embedded file, filespec) from a PDFI have looked at pdf-parser (a Python tool) but it cannot do it automatically

203