Samba4 issues with Time Machine: cannot create new backup on Samba share

Question

Note: originally posted on StackExchange. I'm not sure which location is more appropriate, my apologies if the duplication causes issues.

TL;DR: Time Machine cannot create a new backup on my shared drive, but can add to an existing backup.

I'm running macOS Catalina and my Time Machine backs up to a Debian 10 server with NetAtalk and Avahi. Since Mavericks macOS has preferred SMB, and given SMB is marginally faster I decided to switch to using SMB for the Time Machine shares. On a fresh AFP share I can start a new Time Machine backup in System Preferences and it will create a new .sparseimage without complaint.

If I use the exact same directory (/usr/local/smb), so same permissions etc, and create a samba share, when Time Machine attempts to create a new backup it give the error: "Time Machine couldn’t complete the backup to SERVER.local. The backup disk image could not be created."

If I first connect to the share with AFP and do the initial backup, I can then connect with SMB and add subsequent incremental backups without error. I thought maybe a permissions issue, but for debugging purposes I have /usr/local/smb set to 0777 and still get the error.

ls -la showing permissions of the share point:

drwxrwxrwx  5 root smbusers 4096 Apr  3 12:35 smb

I find the following possibly helpful error in the log:

Failed to create '/Volumes/.timemachine/SERVER._smb._tcp.local/DDE06691-7411-41DD-8419-24FEFC21CE29/TimeMachine Set A - SMB/8E394711-7E3F-520B-800C-192D4F680177.sparsebundle', results: {
}, error: 13 Permission denied

afp.conf:

[Global]
; Global server settings
vol preset = default_for_all
log file = /var/log/netatalk.log
uam list = uams_dhx2.so,uams_clrtxt.so
save password = no

[default_for_all]
file perm = 0664
directory perm = 0774
cnid scheme = dbd

[Time Machine Set A - AFP]
path = /usr/local/smb
time machine = yes
vol size limit = 4000000

I'm using some smb.conf options suggested in this GitHub: https://gist.github.com/ChloeTigre/4c2022c0d1a281deedba6f7539a2e3ae

smb.conf:

[global]

## Browsing/Identification ###

# Change this to the workgroup/NT-domain name your Samba server will part of
   workgroup = WORKGROUP
   wins support = yes

#### Debugging/Accounting ####

# This tells Samba to use a separate log file for each machine
# that connects
   log file = /var/log/samba/log.%m

# Cap the size of the individual log files (in KiB).
   max log size = 1000

# We want Samba to only log to /var/log/samba/log.{smbd,nmbd}.
# Append syslog@1 if you want important messages to be sent to syslog too.
   logging = file

# Do something sensible when Samba crashes: mail the admin a backtrace
   panic action = /usr/share/samba/panic-action %d

###MacOS compatability stuff
guest account = smbguest
min protocol = SMB2
map acl inherit = yes
vfs objects = catia fruit streams_xattr  
fruit:metadata = netatalk
fruit:model = MacSamba
fruit:posix_rename = yes 
fruit:veto_appledouble = yes

durable handles = yes
kernel oplocks = no
kernel share modes =no
posix locking = no
smb2 leases = yes


#Turned off for testing compatability
#fruit:wipe_intentionally_left_blank_rfork = yes 
#fruit:delete_empty_adfiles = yes 

####### Authentication #######

# Server role. Defines in which mode Samba will operate. Possible
# values are "standalone server", "member server", "classic primary
# domain controller", "classic backup domain controller", "active
# directory domain controller". 
#
# Most people will want "standalone server" or "member server".
# Running as "active directory domain controller" will require first
# running "samba-tool domain provision" to wipe databases and create a
# new domain.
   server role = standalone server

   obey pam restrictions = yes

# This boolean parameter controls whether Samba attempts to sync the Unix
# password with the SMB password when the encrypted SMB password in the
# passdb is changed.
   unix password sync = yes

# For Unix password sync to work on a Debian GNU/Linux system, the following
# parameters must be set (thanks to Ian Kahan <<[email protected]> for
# sending the correct chat script for the passwd program in Debian Sarge).
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .

# This boolean controls whether PAM will be used for password changes
# when requested by an SMB client instead of the program listed in
# 'passwd program'. The default is 'no'.
   pam password change = yes

# This option controls how unsuccessful authentication attempts are mapped
# to anonymous connections
   map to guest = bad user

######Security#######
security = user
valid users = @smbusers
username map = /etc/samba/users.map
guest ok = no

# Allow users who've been granted usershare privileges to create
# public shares, not just authenticated ones
   usershare allow guests = yes

#======================= Share Definitions =======================

[TimeMachine Set A - SMB]
path = /usr/local/smb
comment = SMB Time Machine Destination Set A
browsable = yes
writeable = yes
create mode = 0664     #tried turning this off, no fix
directory mode = 0777  #tried turning this off, no fix
vfs objects = catia fruit streams_xattr
fruit:aapl = yes
fruit:time machine = yes
#guest ok = yes
fruit:time machine max size = 3.9T  #tried turning this off, no fix
inherit acls = yes

Solution 1

Thought I'd post some additional info for people. My currently working smb.conf is as follows. Adding fruit:metadata = stream worked for only a short while for me, so below are the results of hours of additional testing.

Note this is on Debian 10 (Buster) so things like password change command will be different on other distributions. Also note that the order of modules in the setting vfs objects = catia fruit streams_xattr is significant and important.

# NOTE: Whenever you modify this file you should run the command
# "testparm" to check that you have not made any basic syntactic 
# errors. 
[global]
   workgroup = WORKGROUP
   min protocol = SMB2


   log file = /var/log/samba/log.%m
   max log size = 5000
   logging = file
   security = USER
   
   panic action = /usr/share/samba/panic-action %d

   server role = standalone server
   obey pam restrictions = yes
   unix password sync = yes

   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   username map = /etc/samba/users.map
   map to guest = bad user
   guest account = XXXYOURGUESTACCOUNT

# Time Machine settings
   vfs objects = catia fruit streams_xattr
   fruit:model = RackMac
   fruit:advertise_fullsync = true
   fruit:metadata = stream
   fruit:veto_appledouble = no
#default is yes, not necessary to specify fruit:posix_rename = no
#default is yes, not necessary to specify fruit:zero_file_id = yes
   fruit:wipe_intentionally_left_blank_rfork = yes
   fruit:delete_empty_adfiles = yes
   ea support = yes


#default is yes, not necessary to specify   fruit:aapl = yes

   # Make share visible to Windows
#Disabled for a Mac-Only network
#   lanman auth = no
#   ntlm auth = yes
#   wins support = yes
#   local master = yes
#   preferred master = yes

   # Allow symlinks
#   follow symlinks = yes
#   wide links = yes
#   unix extensions = no

#======================= Share Definitions =======================
[SHARED DRIVE]

   path = /PATH/TO/YOUR/SHARED/FOLDER
   valid users = @YOURGUESTUSERGROUP
   writable = yes
   durable handles = yes
   kernel oplocks = no
   kernel share modes = no
   posix locking = no
   vfs objects = catia fruit streams_xattr
#default is yes, not necessary to specify   ea support = yes
#default is yes, not necessary to specify browseable = yes
   read only = no
   inherit acls = yes
   fruit:time machine = yes
   fruit:metadata = stream
   fruit:locking = netatalk
   guest ok = yes

[Time Machine]
   path = /PATH/TO/YOUR/SHARED/FOLDER
   valid users = @YOURGUESTUSERGROUP
   writable = yes
   durable handles = yes
   kernel oplocks = no
   kernel share modes = no
   posix locking = no
   vfs objects = catia fruit streams_xattr
#default is yes, not necessary to specify   ea support = yes
   browseable = no
   read only = no
   inherit acls = yes
   fruit:time machine = yes
   fruit:metadata = stream
   fruit:locking = netatalk
   fruit:time machine max size = 1.9T

**EDIT: Turns out Samba 4.9 on Debian 10 DOES automatically advertise TimeMachine destinations, as long as there are no Samba-related advertisements in /etc/avahi/services. My system wasn't advertising the TimeMachine share because I also had a manual smb share setup for avahi. When I disabled BOTH, Samba properly advertised both.

*** OLD SEE ABOVE Also note that The Samba build with Debian 10 currently does not have built-in support for advertising Time Machine shares. Normally specifying fruit:time machine = yes will trigger Samba to advertise a Time Machine destination via Avahi, but this specific support is not built with the Samba included in Debian 10. You'll need to configure a .service definition in /etc/avahi/services/. I created time_machine_advert.service in that folder. You'll need to restart avahi to be sure it gets applied with sudo systemctl restart avahi. I have two Time Machine destinations. Multiples can be advertised within a single file by simply duplicating the txt record and incrementing dkX, i.e. dk1, dk2 etc.

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
 <name replace-wildcards="yes">%h</name>
 <service>
   <type>_adisk._tcp</type>
   <txt-record>sys=waMa=0,adVF=0x100</txt-record>
   <txt-record>dk0=adVN=SAMBA SHARE NAME EXACTLY,adVF=0x82</txt-record>
   <txt-record>dk1=adVN=SAMBA SHARE NAME #2 EXACTLY,adVF=0x82</txt-record>
 </service>
  <service>
    <type>_smb._tcp</type>
    <port>445</port>
  </service>
</service-group>

There are some additional useful sites for Samba configuration with regards to Time Machine here:

Samba team's official guide to Time Machine configuration, although this alone didn't get a working config for me: https://wiki.samba.org/index.php/Configure_Samba_to_Work_Better_with_Mac_OS_X

smb.conf reference: https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html

vfs_fruit (the Apple compatibility module for Samba) reference https://www.samba.org/samba/docs/current/man-html/vfs_fruit.8.html

Ken Murphy's smb.conf on GitHub that finally pushed me over the edge and got my setup working: https://github.com/KenMurphy/SambaConfigs/blob/master/smb.conf

A general Samba on Debian guide: https://www.antoneliasson.se/journal/time-machine-compatible-samba-on-debian-buster/

Solution 2

I encountered the same issue, having my Mac refusing to create the initial sparse bundle files on my samba server. After incorporating the hints on https://wiki.samba.org/index.php/Configure_Samba_to_Work_Better_with_Mac_OS_X it finally started the backup.

Although I can only assume, the following comment caught my attention: How to store OS X metadata: fruit:metadata = stream

You could try switching from netatalk to stream. Maybe using netatalk prevents time machine from storing its metadata.

Good luck, Kay

Comments

[Mike]

That was it, thank you very much! 'stream' is mentioned twice in the fruit:vfs man page, once for fruit:resource and once for fruit:metadata. The comments on fruit:resource warned about this option being experimental, and I confused it with fruit:metadata so I didn't try this option. Thanks a bunch!

[lickdragon]

Small note: Looks like Debian Buster Samba 4.9 does broadcast on avahi automatically if avahi-daemon is installed.

[Mike]

Yes, but not fully. See the comments above about this. Avahi-daemon as compiled for Debian 10 does not automatically broadcast Time Machine shares when fruit:timemachine = yes is set. It only broadcasts Samba shares automatically.

[lickdragon]

Using 'avahi-discover' I see an entry under 'Apple TimeMachine' which I didn't create and matches the hostname of the Samba timemachine server.

[Mike]

Interesting. I'm also running Samba 4.9 on my Debian 10 box and when I run avahi-discover I have no such entry unless I manually add it in /etc/avahi/services. Are you running a later, perhaps back ported Samba? Is your smb.conf identical to mine? I'd be curious what differences there are. Do you have a Mac? If so, does it see the Time Machine destination in System Preferences?

[Mike]

@lickdragon I'm very curious as to your setup. Can you paste your smb.conf and avahi-daemon.conf?

[Mike]

Nevermind @lickdragon, I figured it out. I had two .service files in /etc/avahi/services, one for the SMB share and one for the Time Machine. If either was active samba does not automatically advertise. If both are disabled, samba does indeed advertise the share and my Time Machine destinations.

[Ari 'APz' Sovijärvi]

Does this reliably work in the long run? I have very similar setup, that when smbd and avahi-daemon are restarted, works for maybe an hour or two, after which avahi stops advertising the Time machine shares. CUPS and others work as expected. I have no manual services defined either for Samba as it seems to add them on its own. This is on Debian 10.

[Mike]

It has been reliably working for several weeks for me, and persists across restarts and upgrades