Home Delete lines in a text file that contain a specific string

# Delete lines in a text file that contain a specific string

A Clockwork Orange
1#
A Clockwork Orange Published in 2011-03-23 19:46:07Z
 How would I use sed to delete all lines in a text file that contain a specific string?
2#
 To remove the line and print the output to standard out: sed '/pattern to match/d' ./infile  To directly modify the file: sed -i '/pattern to match/d' ./infile  To directly modify the file (and create a backup): sed -i.bak '/pattern to match/d' ./infile  For Mac OS X users: sed -i '' '/pattern/d' ./infile 
thanasisp
3#
 there are many other ways to delete lines with specific string besides sed awk awk '!/pattern/' file > temp && mv temp file  Ruby (1.9+) ruby -i.bak -ne 'print if not /test/' file  Perl perl -ni.bak -e "print unless /pattern/" file  Shell (bash3.2+) while read -r line do [[ ! $line =~ pattern ]] && echo "$line" done o mv o file  GNU grep grep -v "pattern" file > temp && mv temp file  and of course sed (printing the inverse is faster than actual deletion. ) sed -n '/pattern/!p' file 
DomainsFeatured
4#
 You can use sed to replace lines in place in a file. However, it seems to be much slower than using grep for the inverse into a second file and then moving the second file over the original. e.g. sed -i '/pattern/d' filename  or grep -v "pattern" filename > filename2; mv filename2 filename  The first command takes 3 times longer on my machine anyway.
Kjetil S.
5#
Kjetil S. Reply to 2018-01-08 12:07:39Z
 perl -i -nle'/regexp/||print' file1 file2 file3 perl -i.bk -nle'/regexp/||print' file1 file2 file3  The first command edits the file(s) inplace (-i). The second command does the same thing but keeps a copy or backup of the original file(s) by adding .bk to the file names (.bk can be changed to anything).
codeforester
6#
 The easy way to do it, with GNU sed: sed --in-place '/some string here/d' yourfile 
Bhuvanesh
7#
 You can use this also  grep -v 'pattern' filename  here -v will print only other than your pattern(that means Invert match)
Jahid
8#
 To get a inplace like result with grep you can do this: echo "$(grep -v "pattern" filename)" >filename  Community 9# Community Reply to 2017-04-13 12:36:30Z  You may consider using ex (which a standard UNIX command-based editor): ex +g/match/d -cwq file  where: + executes given Ex command (man ex), same as -c which executes wq (write and quit) g/match/d - Ex command to delete lines with given match, see: Power of g Above example is POSIX-compliant method for in-place editing a file as per this post at Unix.SE and POSIX specifications for ex. The difference with sed is that: sed is a Stream EDitor, not a file editor.BashFAQ unless you enjoy unportable code, I/O overhead and some other bad side effects. So basically some parameters (such as in-place/-i) are non-standard FreeBSD extensions and may not be available on other operating systems. Community 10# Community Reply to 2017-05-23 12:10:48Z  I was struggling with this on Mac. Plus, I needed to do it using variable replacement. So I used: sed -i '' "/$pattern/d" $file where $file is file where deletion is needed and $pattern is the pattern to be matched for deletion. Picked the '' from this comment. The thing to note here is use of double quotes in "/$pattern/d". Variable won't work when we use single quote.
JakeGould
11#
 SED: '/James\|John/d' -n '/James\|John/!p' AWK: '!/James|John/' /James|John/ {next;} {print} GREP: -v 'James\|John'
Shizzmo
12#
 echo -e "/thing_to_delete\ndd\033:x\n" | vim file_to_edit.txt 
 Just in case someone wants to do it for exact matches of strings, you can use -w flag in grep, w for whole. That is, for example if you want to delete the lines that have number 11 but keep the lines with number 111: -bash-4.1$head file 1 11 111 -bash-4.1$ grep -v "11" file 1 -bash-4.1\$ grep -w -v "11" file 1 111  Also works with -f flag if you want to exclude several exact patterns at once. If "blacklist" is a file with several patterns on each line that you want to delete from "file": grep -w -v -f blacklist file 
 I have made a small benchmark with a file which contains approximately 345 000 lines. The way with grep seems to be around 15 times faster than the sed method in this case. I have tried both with and without the setting LC_ALL=C, it does not seem change the timings significantly. The search string (CDGA_00004.pdbqt.gz.tar) is somewhere in the middle of the file. Here are the commands and the timings: time sed -i "/CDGA_00004.pdbqt.gz.tar/d" /tmp/input.txt real 0m0.711s user 0m0.179s sys 0m0.530s time perl -ni -e 'print unless /CDGA_00004.pdbqt.gz.tar/' /tmp/input.txt real 0m0.105s user 0m0.088s sys 0m0.016s time (grep -v CDGA_00004.pdbqt.gz.tar /tmp/input.txt > /tmp/input.tmp; mv /tmp/input.tmp /tmp/input.txt ) real 0m0.046s user 0m0.014s sys 0m0.019s