In this recipe we’ll look into the topic of loading text files in Pandas dataframes. for further data wrangling for visualization purposes or as a preparatory step for Machine Learning.
Use pd.read_csv() to load text file with tab delimiters
Let’s outline this using a simple example. Suppose we have a text file that has several rows. The values are separated by a tab delimiter. If you want to follow along, you can copy and paste the text into a .txt file:
Let’s now go ahead and load it to Pandas. In this example i suumed the file is saved in a folder named data (will work in Windows, Linux or macOS).
import pandas as pd sales= pd.read_csv('../data/sales_tab.txt') sales.head()
As you can see below, Pandas didn’t read correctly the tabstops in your file.
The solution is to swith the delimiter=’\t’ parameter of the pd.read_csv() function to define the tabspace as the delimiting character.
sales= pd.read_csv('../data/sales_tab.txt', delimiter='\t') sales.head()
This is pretty cool. with a couple of code rows you got the text file line by line (including the headers) into your Pandas sales dataframe and now you can start to work with the data.
Note that you can obviously customize the column headers as needed, by using the names parameter, as shown below.
sales= pd.read_csv('../data/sales_tab.txt', delimiter='\t', names=['singer', 'net worth'])
Read text with no spaces into Pandas
What of the content of the text file is just separated by commas, like a .csv file and the file is encoded as utf-16, as it was probably saved from Excel?
This won’t work:
You’ll get an encoding error: ‘utf-8’ codec can’t decode byte 0xff in position 0: invalid start byte
Solution is to use the encoding parameter and set it to utf-16 or other relevant encoding as needed:
sales2= pd.read_csv('../data/sales_txt.txt', \ encoding='utf-16')