How to avoid duplicate entries in .bash_history

Question

The same command is listed in my history again and again. How can I avoid this? I know it's possible via HISTCONTROL but I haven't found the right way.

Solution 1

From the bash man page:

HISTCONTROL

A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines which begin with a space character are not saved in the history list. A value of ignoredups causes lines matching the previous history entry to not be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regardless of the value of HISTCONTROL.

So put the following line in your ~/.bashrc:

export HISTCONTROL=ignoreboth:erasedups

Solution 2

Stick this in your ~/.bashrc:

export HISTCONTROL=ignoredups

You could instead use ignoreboth. This it shorthand for both ignorespaces (commands starting with spaces) and ignoredups (duplicates).

I prefer ignoredups on its own as I find the default behaviour of ignoring commands with spaces at the front quite annoying when I copy a command off a website and it doesn't get saved because I accidentally copied in a space too.... But to each their own.

Solution 3

Putting this in ~/.bashrc will apply @alvin's solution across different sessions as wlell

HISTCONTROL=ignoredups:erasedups
shopt -s histappend
PROMPT_COMMAND="history -n; history -w; history -c; history -r; $PROMPT_COMMAND"

Source: Linux: Bash history: “ignoredups” and “erasedups” setting conflict with common history across sessions

Solution 4

export HISTCONTROL=erasedups

Solution 5

Add the following to your ~/.bashrc:

export HISTCONTROL=ignoredups

To do this, you can use this command:

nano ~/.bashrc

View more solutions

Comments

[karthick87]

Do you want me to add HISTCONTROL=ignoreboth in the bottom of ~/.bashrc?

[Isaiah]

@karthick It doesn't matter where you put it, the bottom is fine though.

[karthick87]

Still duplicates exist..What may be the problem??imgur.com/fXeLQ.png

[ck-]

On my installation of Ubuntu 12.04 I had to make this modification in ~/.bash_login Most of the time it would work if .bashrc is modified but that's not the case all the time.

[A.L]

@karthick87 quoted from another answer: A value of ignoredups causes lines matching the previous history entry to not be saved. It only removes consecutive identical lines.

[Eric Kigathi]

Don't forget to run these after you update your bash profile/rc files - source ~/.bashrc && source ~/.bash_profile

[felwithe]

Why does his use export HISTCONTROL and yours doesn't? What's the difference? It's working for me without export, I'm just curious

[Nour Wolf]

Good question! I would also like to know the answer. In general export makes the variable available to sub-processes. stackoverflow.com/q/1158091/552621 In the case of HISTCONTROL, PROMPT_COMMAND, etc, these variables are used by the shell itself so maybe exporting them (to children processes) is not necessary.