Is it possible to make writing to .bash_history immediate?

Question

I often open lots and lots of Terminals [Right now I have 7 open on this workspace] and I often search history with grep to find a command I've just written recently, but I don't want to hunt down the terminal and then scroll up and hunt for it more, etc.. Sometimes my terminals close without 'exit,' and everything I've written is lost [Sometimes I've needed something I'd written in a terminal that was killed].

So is there a way to make it so each terminal writes to .bash_history immediately? or at least once a minute, or something like that?

Solution 1

A simple solution as detailed in Update Bash History in Realtime.

It says to put those commands in the .bashrc config:

shopt -s histappend
PROMPT_COMMAND="history -a;$PROMPT_COMMAND"

The first command changes the .history file mode to append. And the second configures the history -a command to be run at each shell prompt. The -a immediately writes the current/new lines to the history file.

Related for zsh:

• How do you share history between terminals in zsh?

Solution 2

Try putting this into your .bashrc:

shopt -s histappend                      # append to history, don't overwrite it
export PROMPT_COMMAND="history -a; history -c; history -r; $PROMPT_COMMAND"

Credit here: https://stackoverflow.com/questions/103944/real-time-history-export-amongst-bash-terminal-windows/3055135

history -c clears the history of the running session. This will reduce the history counter by the amount of $HISTSIZE. history -r read the contents of $HISTFILE and insert them in to the current running session history. This will raise the history counter by the amount of lines in $HISTFILE.

I think it means that the commands are available almost immediately (you have one terminal, write echo 1, second terminal echo 2, first echo 3 and upon pressing down arrow twice, you should have echo 2 available. You must issue a command in a given terminal to have access to what has been written.

Solution 3

I have a large history file with about 100000 entries, and the variants that clear the history list and read the whole history file (using history -c and history -r) introduce a noticeable (maybe 0.2 second) delay before the prompt is displayed. Using history -n so that only new lines are read from the history file is faster:

shopt -s histappend
PROMPT_COMMAND='history -a;history -n'

PROMPT_COMMAND does not have to be exported because it is a shell variable.

Solution 4

A note for all other answers (which are basically the same thing):

Setting PROMPT_COMMAND="history -a;$PROMPT_COMMAND" in the .bashrc (or friends) is enough.

Also, you can manually run history -a whenever you want to "snapshot" the history in the current session.

The command shopt -s histappend is not needed because history -a always appends new lines to the file and never overwrites it. Also, at least as of Bash 4, histappend is the default behavior.

View more solutions

Comments

[Admin]

Normally I use control-r to reverse-search in the command history of one bash. More convenient than history | grep unless you need a regex.

[Admin]

@PeterCordes But that will only search in the history of that one terminal so it is going in the opposite direction of the question. ;)

[Matt]

well this kind of works. I tried some more testing and it seems to only work on the first command. does it work correctly for you?

[pabouk - Ukraine stay strong]

@artfulrobot: This command is a bash built-in so it is described in man bash. You can also use help history.

[Steven Lu]

I have my zsh configured to work like this: Each shell tracks its OWN history (and before that, the combined history), and every command is appended to combined history. This is different and better than always reloading history for every command for every shell, because I tend to have a different "thread of work" or "job" for each shell terminal. I think that is the difference between this with -r and -c vs just having -a.

[sup]

So what exactly are you using? history -c; history -r? I guess it is better for you, it would be better for me only sometimes, I often use several shells to do one job so I prefer to have the history as much combined as possible. I actually mind that command is written to history only after it finishes. typically, when I start a ssh session to one host and want to open another one while the first is still running, I have to retype it because it has not been written to history yet.

[Steven Lu]

I think I never stuck around long enough to figure out how to make bash simultaneously track parallel threads of history (per shell instance basis) while immediately writing history to the main history file. Zsh makes it much easier to do so, plus in addition I have defined a zsh function to write out a lot of fancy metadata to an enhanced-history file of my devising. It's become rather indispensible. Anyway, the mechanism there is precmd() and preexec() for zsh get sent args describing the entered command. It then becomes trivial to direct that to files. history -a might suffice

[Tobia]

Brilliant! No more lost history when the connection gets dropped!

[dragon788]

@StevenLu could you elaborate on your setup with a code snippet? I frequently use Tmux and each "pane" is for a specific task so I'd love to have them separately logged immediately and then only combine the history when opening a new pane where I may want to access those previous sessions.

[wisbucky]

This is wonderful. By appending to the file and reading from file, it will share history across multiple terminals.

[ubershmekel]

On my mac 10.13.6 it seemed it did not work without the shopt -s histappend. Also note I added it to my ~/.bash_profile

[Guss]

Likely there's another setting somewhere in your system that disabled histappend, as that is the default behavior. I would look for it, but I don't have access to a Mac.

[donquixote]

This significantly slows down every command in my terminal. Not worth it in the end. (I suppose it depends on history size)

[sup]

Hm, my history is 51k and it is super fast.

[R J]

The downside of history -c; history -r is that if you have another terminal open on the same computer, and switch to the other terminal to do something, press up, you'd see the last command from the previous terminal, not from this one. Very annoying.

[Jonathan Wren]

Macs have never shipped with Bash 4 (even the new ones). Apple doesn't like GPLv3 which Bash switched to with Bash 4, so they stopped updating Bash on MacOS a long time ago. This is also why the new MacOS is switching to Zsh as the default shell. Long story short, if you want Bash 4 on your Mac, you have to install it yourself.

[Warkwark]

@StevenLu I second the request to see your setup. It sounds like what I want but have been to lazy to set up.

[Steven Lu]

@RJ 's comment is probably exactly why I'm getting these requests for sharing my setup. Thanks for the requests. Unfortunately it's close to a decade since I set stuff up (which was definitely nontrivial) and I'd need to actually dive back into determining the correct settings to get this behavior right. Absolutely worthy of a blog post though, but I'm not set up for blogging at the moment, been putting it off. I know there's an existing blog post out there where someone has clearly laid out how to get this right. Let me look for one..

[Steven Lu]

@Warkwark I think it's as simple as having shopt -s histappend and PROMPT_COMMAND='history -a', mine has some other junk in the PROMPT_COMMAND function, but the only thing you need is to run history -a so it commits commands to history as you issue them. At this point I'm 100% confused by why shopt -s histappend is even there, though, since if you are appending it instantly then you definitely don't want it repeated once you quit. But the behavior I observe is that nothing gets added when you quit bash... So yeah. Mini blog in a comment.

[Steven Lu]

The logic of histappend still escapes me, but superuser.com/a/832091/98199 sums up the sane settings to use.

[Uzumaki D. Ichigo]

isn't histappend there by default?