Journalctl shows logs from about last hour only

Question

I have a problem with journalctl logs, which doesn't show me logs older than about one hour.

First log entry is from about one hour ago, instead of 2 days as requested.

$ date -u
Wed May 13 08:59:21 UTC 2020
$ sudo journalctl --utc --since="2 days ago" -n 1 --no-pager -u [email protected]
-- Logs begin at Tue 2020-03-17 17:30:22 UTC, end at Wed 2020-05-13 09:00:10 UTC. --
May 13 08:11:03 <log content>

Service's been turned on 5 days ago:

$ sudo systemctl status  [email protected]
...
   Active: active (running) since Thu 2020-05-07 15:45:47 CEST; 5 days ago
...

jorunalctd.conf doesn't look suspicions though:

$ cat /etc/systemd/journald.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See journald.conf(5) for details.

[Journal]
Storage=persistent
#Compress=yes
#Seal=yes
#SplitMode=uid
#SyncIntervalSec=5m
#RateLimitInterval=30s
#RateLimitBurst=1000
#SystemMaxUse=
#SystemKeepFree=
#SystemMaxFileSize=
#SystemMaxFiles=100
#RuntimeMaxUse=
#RuntimeKeepFree=
#RuntimeMaxFileSize=
#RuntimeMaxFiles=100
#MaxRetentionSec=
#MaxFileSec=1month
#ForwardToSyslog=yes
#ForwardToKMsg=no
#ForwardToConsole=no
#ForwardToWall=yes
#TTYPath=/dev/console
#MaxLevelStore=debug
#MaxLevelSyslog=debug
#MaxLevelKMsg=notice
#MaxLevelConsole=info
#MaxLevelWall=emerg

It's an repetitive pattern that I see logs from about last hour. What may be the cause?

edit

Unfortunately it's not a problem with pager size limit, as @Peleion mentioned.

Additional limiting by --until doesn't show any messages (and no -- No entries -- info).

$ sudo journalctl --utc --since="2 days ago" --until="yesterday" -n 1 --no-pager -u [email protected]  # with additional `--until` filter no messages are shown
-- Logs begin at Tue 2020-03-17 17:30:22 UTC, end at Thu 2020-05-14 09:14:04 UTC. --
$ sudo journalctl --utc --since="2 days ago" --no-pager -u [email protected]  | wc -l  # logs more than 1k entities
17661
$ sudo journalctl -p err --utc --since="2 days ago" -n 1 --no-pager -u [email protected]  # here we have explicit info that there're no entries
-- Logs begin at Tue 2020-03-17 17:30:22 UTC, end at Thu 2020-05-14 09:14:21 UTC. --
-- No entries --

edit2

We ended up logging data to separate file instead of using journald.

There was to much mysterious problems with missing logs.
All the time jorunal used about 2.5-3GB:

Archived and active journals take up 3.1G in the file system.

And file of user responsible for running workers had between 30 and 60MB:

/var/log/journal/<hash>/user-<user id>.journal

Solution

The journalctl pager defaults to the most recent 1,000 entries. This number of entries in an hour is unfortunately common on current Fedora installs. The journal entries are still there (up to the max journal file storage) but you can only see 1,000 at a time.

Use --since and --until together to specify a particular 1,000 entry chunk of time or --user to specify a particular user or any other option that limits the output to the 1,000 buffer with a greater time slice.

Archlinux documentation has a good introduction of options

Edit: It also appears your journal is not flushing to persistent storage. RH/Centos/Fedora has #Storage=auto as the default setting in /etc/systemd/journald.conf. You might try commenting yours out or setting it to auto and restarting systemd.

Comments

[Jakub Kuszneruk]

it's not the case, I've updated OP.

[Peleion]

Answer updated . . .

[Jakub Kuszneruk]

But I need storage to be persistent, because I need data be kept between machine reboots.