Home How do I read a file line-by-line into a list?
Reply: 30

How do I read a file line-by-line into a list?

Julie Raswick
1#
Julie Raswick Published in 2010-07-18 22:25:01Z

How do I read every line of a file in Python and store each line as an element in a list?

I want to read the file line by line and append each line to the end of the list.

Noctis Skytower
2#
Noctis Skytower Reply to 2010-07-18 22:27:24Z

This will yield an "array" of lines from the file.

lines = tuple(open(filename, 'r'))
Martin Thoma
3#
Martin Thoma Reply to 2015-01-08 08:50:43Z

This is more explicit than necessary, but does what you want.

with open("file.txt", "r") as ins:
    array = []
    for line in ins:
        array.append(line)
mklement0
4#
mklement0 Reply to 2015-05-22 16:44:36Z

See Input and Ouput:

with open('filename') as f:
    lines = f.readlines()

or with stripping the newline character:

lines = [line.rstrip('\n') for line in open('filename')]

Editor's note: This answer's original whitespace-stripping command, line.strip(), as implied by Janus Troelsen's comment, would remove all leading and trailing whitespace, not just the trailing \n.

holzkohlengrill
5#
holzkohlengrill Reply to 2017-01-11 14:24:50Z
with open(fname) as f:
    content = f.readlines()
# you may also want to remove whitespace characters like `\n` at the end of each line
content = [x.strip() for x in content] 

I'm guessing that you meant list and not array.

Peter Mortensen
6#
Peter Mortensen Reply to 2017-03-04 21:52:54Z

Another option is numpy.genfromtxt, for example:

import numpy as np
data = np.genfromtxt("yourfile.dat",delimiter="\n")

This will make data a NumPy array with as many rows as are in your file.

cevaris
7#
cevaris Reply to 2013-10-28 15:40:52Z

This should encapsulate the open command.

array = []
with open("file.txt", "r") as f:
  for line in f:
    array.append(line)
oliland
8#
oliland Reply to 2013-11-22 14:57:48Z

If you'd like to read a file from the command line or from stdin, you can also use the fileinput module:

# reader.py
import fileinput

content = []
for line in fileinput.input():
    content.append(line.strip())

fileinput.close()

Pass files to it like so:

$ python reader.py textfile.txt 

Read more here: http://docs.python.org/2/library/fileinput.html

moldovean
9#
moldovean Reply to 2014-01-12 10:58:04Z
f = open("your_file.txt",'r')
out = f.readlines() # will append in the list out

Now variable out is a list (array) of what you want. You could either do:

for line in out:
    print line

or

for line in f:
    print line

you'll get the same results.

Eneko Alonso
10#
Eneko Alonso Reply to 2014-03-02 04:22:39Z

If you want the \n included:

with open(fname) as f:
    content = f.readlines()

If you do not want \n included:

with open(fname) as f:
    content = f.read().splitlines()
user1833244
11#
user1833244 Reply to 2014-05-27 12:21:01Z

Here's one more option by using list comprehensions on files;

lines = [line.rstrip() for line in open('file.txt')]

This should be more efficient way as the most of the work is done inside the Python interpreter.

Nikko Dot Jpg
12#
Nikko Dot Jpg Reply to 2014-07-10 11:19:24Z
lines = list(open("dict.lst", "r"))
linesSanitized = map(lambda each:each.strip("\n"), lines)
print linesSanitized
Johnny
13#
Johnny Reply to 2014-12-20 18:31:22Z

Clean and Pythonic Way of Reading the Lines of a File Into a List


First and foremost, you should focus on opening your file and reading its contents in an efficient and pythonic way. Here is an example of the way I personally DO NOT prefer:

infile = open('my_file.txt', 'r')  # Open the file for reading.

data = infile.read()  # Read the contents of the file.

infile.close()  # Close the file since we're done using it.

Instead, I prefer the below method of opening files for both reading and writing as it is very clean, and does not require an extra step of closing the file once you are done using it. In the statement below, we're opening the file for reading, and assigning it to the variable 'infile.' Once the code within this statement has finished running, the file will be automatically closed.

# Open the file for reading.
with open('my_file.txt', 'r') as infile:

    data = infile.read()  # Read the contents of the file into memory.

Now we need to focus on bringing this data into a Python List because they are iterable, efficient, and flexible. In your case, the desired goal is to bring each line of the text file into a separate element. To accomplish this, we will use the splitlines() method as follows:

# Return a list of the lines, breaking at line boundaries.
my_list = data.splitlines()

The Final Product:

# Open the file for reading.
with open('my_file.txt', 'r') as infile:

    data = infile.read()  # Read the contents of the file into memory.

# Return a list of the lines, breaking at line boundaries.
my_list = data.splitlines()

Testing Our Code:

  • Contents of the text file:
     A fost odatã ca-n povesti,
     A fost ca niciodatã,
     Din rude mãri împãrãtesti,
     O prea frumoasã fatã.
  • Print statements for testing purposes:
    print my_list  # Print the list.

    # Print each line in the list.
    for line in my_list:
        print line

    # Print the fourth element in this list.
    print my_list[3]
  • Output (different-looking because of unicode characters):
     ['A fost odat\xc3\xa3 ca-n povesti,', 'A fost ca niciodat\xc3\xa3,',
     'Din rude m\xc3\xa3ri \xc3\xaemp\xc3\xa3r\xc3\xa3testi,', 'O prea
     frumoas\xc3\xa3 fat\xc3\xa3.']

     A fost odatã ca-n povesti, A fost ca niciodatã, Din rude mãri
     împãrãtesti, O prea frumoasã fatã.

     O prea frumoasã fatã.
Jean-Francois T.
14#
Jean-Francois T. Reply to 2015-02-10 05:47:57Z

The simplest way to do it

A simple way is to:

  1. Read the whole file as a string
  2. Split the string line by line

In one line, that would give:

lines = open('C:/path/file.txt').read().splitlines()
Ajoy
15#
Ajoy Reply to 2015-03-26 21:23:07Z
with open(fname) as fo:
        data=fo.read().replace('\n', ' ').replace (',', ' ')

This should answer your question. The replace function will act as delimiter to strip the file.

Pedro Lobito
16#
Pedro Lobito Reply to 2017-06-04 01:44:19Z

if you don't care about closing the file, this one-liner works:

lines = open('file.txt').read().split("\n")

The traditional way:

fp = open('file.txt') # open file on read mode
lines = fp.read().split("\n") # create a list containing all lines
fp.close() # close file

Using with (recommended):

with open('file.txt') as fp:
    lines = fp.read().split("\n")
asampat3090
17#
asampat3090 Reply to 2015-07-20 17:33:03Z

Could also use the loadtxt command in numpy. This checks for fewer conditions than genfromtxt so it may be faster.

import numpy 
data = numpy.loadtxt(filename,delimiter="\n")
Chinthaka Suren Fernando
18#
Chinthaka Suren Fernando Reply to 2016-02-01 04:49:58Z
textFile = open("E:\Values.txt","r")
textFileLines = textFile.readlines()

"textFileLines" is the array you wanted

DevShark
19#
DevShark Reply to 2016-08-24 18:04:48Z

You could simply do the following, as has been suggested:

with open('/your/path/file') as f:
    my_lines = f.readlines()

Note that this approach has 2 downsides:

1) You store all the lines in memory. In the general case, this is a very bad idea. The file could be very large, and you could run out of memory. Even if it's not large, it is simply a waste of memory.

2) This does not allow processing of each line as you read them. So if you process your lines after this, it is not efficient (requires two passes rather than one).

A better approach for the general case would be the following:

with open('/your/path/file') as f:
    for line in f:
        process(line)

Where you define your process function any way you want. For example:

def process(line):
    if 'save the world' in line.lower():
         superman.save_the_world()

(The implementation of the Superman class is left as an exercise for you).

This will work nicely for any file size and you go through your file in just 1 pass. This is typically how generic parsers will work.

Peter Mortensen
20#
Peter Mortensen Reply to 2017-03-04 21:54:42Z

To my knowledge Python doesn't have a native array data structure. But it does support the list data structure which is much simpler to use than an array.

array = [] #declaring a list with name '**array**'
with open(PATH,'r') as reader :
    for line in reader :
        array.append(line)
Giovanni Gianni
21#
Giovanni Gianni Reply to 2017-11-26 07:58:50Z

A real easy way:

with open(file) as g:
    stuff = g.readlines()

If you want to make it a fully-fledged program, type this in:

file = raw_input ("Enter EXACT file name: ")
with open(file) as g:
    stuff = g.readlines()
print (stuff)
exit = raw_input("Press enter when you are done.")

For some reason, it doesn't read .py files properly.

Peter Mortensen
22#
Peter Mortensen Reply to 2017-03-04 21:56:36Z

Use this:

import pandas as pd
data = pd.read_csv(filename) # You can also add parameters such as header, sep, etc.
array = data.values

data is a dataframe type, and uses values to get ndarray. You can also get a list by using array.tolist().

Xolve
23#
Xolve Reply to 2016-05-04 06:19:11Z

How about:

fp = open("filename")
content = fp.read();
lines = content.split("\n")
Giovanni Gianni
24#
Giovanni Gianni Reply to 2017-11-26 07:56:21Z

Just use the splitlines() functions. Here is an example.

inp = "file.txt"
data = open(inp)
dat = data.read()
lst = dat.splitlines()
print lst
# print(lst) # for python 3

In the output you will have the list of lines.

user3394040
25#
user3394040 Reply to 2016-12-09 18:43:57Z

I'd do it like this.

lines = []
with open("myfile.txt") as f:
    for line in f:
        lines.append(line)
Jonathan Leffler
26#
Jonathan Leffler Reply to 2017-03-16 04:09:13Z

If you want to are faced with a very large / huge file and want to read faster (imagine you are in a Topcoder/Hackerrank coding competition), you might read a considerably bigger chunk of lines into a memory buffer at one time, rather than just iterate line by line at file level.

buffersize = 2**16
with open(path) as f: 
    while True:
        lines_buffer = f.readlines(buffersize)
        if not lines_buffer:
            break
        for line in lines_buffer:
            process(line)
Giovanni Gianni
27#
Giovanni Gianni Reply to 2018-01-12 19:36:40Z

Data into list

Let's read data from a text file

Text file content:

   line 1
   line 2
   line 3
  1. Open the cmd in the same dir (right click the mouse and choose cmd or powershell)
  2. run python and in the interpreter write:

The python script

>>> with open("myfile.txt", encoding="utf-8") as file:
...     x = [l.strip() for l in file]
>>> x
['line 1','line 2','line 3']

Using append

x = []
with open("myfile.txt") as file:
    for l in file:
        x.append(l.strip())

or...

>>> x = open("myfile.txt").read().splitlines()
>>> x
['line 1','line 2','line 3']

or...

>>> y = [x.rstrip() for x in open("my_file.txt")]
>>> y
['line 1','line 2','line 3']

Getting a text from a web page with python 3

Here there's a pratical example of a text grabbed from the net. The page contains just plain text. We just need to get rid of the \n \r and b' thing, in it to keep it clean to be printed. There's a convertion of bytes data to string data and then the string is splitted in lines or, better, each line is stored into an item of the list. when the print function is called, we pass each item of the list without the \r and \' and b' things that make the text less readeable.

from urllib.request import urlopen
testo = urlopen("https://www.gutenberg.org/files/11/11.txt").read()
testo = str(testo).split("\\n")
for l in testo[30:48]:
    print(l.replace("\\r","").replace("\\'","\'").replace("b'",""))

OUTPUT:

ALICE'S ADVENTURES IN WONDERLAND

Lewis Carroll

THE MILLENNIUM FULCRUM EDITION 3.0

CHAPTER I. Down the Rabbit-Hole

Alice was beginning to get very tired of sitting by her sister on the bank, and of having nothing to do: once or twice she had peeped into the book her sister was reading, but it had no pictures or conversations in it, 'and what is the use of a book,' thought Alice 'without pictures or conversations?'

Giovanni Gianni
28#
Giovanni Gianni Reply to 2017-11-26 08:00:58Z

You can just open your file for reading using

file1 = open("filename","r")
# and for reading use
lines = file1.readlines()
file1.close()

The list lines will contain all your lines as individual elements and you can call a specific element using lines["linenumber-1"] as python starts its counting from 0.

Pedram
29#
Pedram Reply to 2017-06-26 01:43:06Z

You can easily do it by the following piece of code:

lines = open(filePath).readlines()
jasonleonhard
30#
jasonleonhard Reply to 2017-08-29 23:53:59Z

Command line version

#!/bin/python3
import os
import sys
abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
filename = dname + sys.argv[1]
arr = open(filename).read().split("\n") 
print(arr)

Run with:

python3 somefile.py input_file_name.txt
Martin Thoma
31#
Martin Thoma Reply to 2018-01-16 19:51:03Z

Read and write text files with Python 2+3; works with unicode

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# Define data
lines = ['     A first string  ',
         'A unicode sample: €',
         'German: äöüß']

# Write text file
with open('file.txt', 'w') as fp:
    fp.write('\n'.join(lines))

# Read text file
with open('file.txt', 'r') as fp:
    read_lines = fp.readlines()
    read_lines = [line.rstrip('\n') for line in read_lines]

print(lines == read_lines)

Things to notice:

  • with is a so called context manager. It makes sure that the opened file is closed again.
  • All solutions here which simply make .strip() or .rstrip() will fail to reproduce the lines as they also strip the white space.

Common file endings

.txt

More advanced file writing / reading

  • CSV: Super simple format (read & write)
  • JSON: Nice for writing human-readable data; VERY commonly used (read & write)
  • YAML: YAML is a superset of JSON, but easier to read (read & write, comparison of JSON and YAML)
  • pickle: A Python serialization format (read & write)
  • MessagePack (Python package): More compact representation (read & write)
  • HDF5 (Python package): Nice for matrices (read & write)
  • XML: exists too *sigh* (read & write)

For your application, the following might be important:

  • Support by other programming languages
  • Reading / writing performance
  • Compactness (file size)

See also: Comparison of data serialization formats

In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python

You need to login account before you can post.

About| Privacy statement| Terms of Service| Advertising| Contact us| Help| Sitemap|
Processed in 0.475744 second(s) , Gzip On .

© 2016 Powered by mzan.com design MATCHINFO