Using Xsv to Process Csv

July 12, 2020

Sometimes, you have a csv file that you need to do something with. xsv is a great tool for doing this. For example, I have a csv file that looks like this:

# Last update: 2016-12-28 (v1.0.0-csv.1)
# PLEASE DON'T REMOVE THIS IMPORTANT INFORMATION!
#---------------------------",,,,
id,sura,aya,translation,footnotes
1,1,1,"Ní orúkọ Allāhu, Àjọkẹ́-ayé, Àṣàkẹ́-ọ̀run.",""
2,1,2,"Gbogbo ọpẹ́ ń jẹ́ ti Allāhu, Olúwa gbogbo ẹ̀dá,",""
3,1,3,"Àjọkẹ́-ayé, Àṣàkẹ́-ọ̀run,",""
4,1,4,"Olùkápá-ọjọ́-ẹ̀san.",""
5,1,5,"Ìwọ nìkan ni à ń jọ́sìn fún. Ọ̀dọ̀ Rẹ nìkan sì ni à ń wá oore sí.",""
5,1,6,...

Let’s say I want to know if any rows have footnotes or not - I can use xsv to find this out pretty easily:

tail -n +3 input.csv | xsv select footnotes | grep -v \"\"

Essentially, we’re saying, “give me the rest of the file without that top comment header” (by passing in -n +3, meaning tail the entire file skipping the first 3 lines), and asking xsv to select footnotes from it, and finally removing any entries that are just quotes.

xsv has many more advanced uses as well, so it’s worth taking a look at the project for more details.