In Console.app, how can I reveal to what <private> tags are actually referring?

Question

In the last several versions of MacOS, Console.app has become increasingly more censored, to the point of near uselessness.

Filenames and URLs have been replaced by <private>tags, making it impossible to track down and remedy errors.

While the following example isn't an error, nor is it the file permissions error I was having for literally years with cfprefsd, unable to track it down, it is an example as to how vague and unhelpful most messages logged with Console.app have become:

 com.apple.WebKit.Networking [19870 <private> <private>] start

How can I reveal to what the <private> tags refer in Console.app log messages?

Solution 1

Console.app can be made to display actual file paths and URLs instead of private by issuing the following command in Terminal.app:

    sudo log config --mode "private_data:on"

This causes messages logged to Console.app to display more specific and helpful information, like URLs and filenames, instead of just the cryptic <private> tags, but not retroactively. The error or condition will have to occur again for the previously censored data to be displayed.

So an essentially useless message like:

  com.apple.WebKit.Networking [19870 <private> <private>] start

would then be expanded to something like:

 com.apple.Webkit.Networking [19920 www.facebook.com:443 stream, pid: 5311, url: https://www.facebook.com/api/graphqlbatch/, tls] start

Since to leave private_data:on long-term may compromise privacy and security, the logging facility can be returned to its normal obscure level with this command:

sudo log config --mode "private_data:off"

once finished tracking down the desired event.

Solution 2

Solution for Catalina

You can add a .mobileprofile which will deprivatize the logs in Catalina 10.15.4.

I'll copy the answer here that user lx07 shared at: https://superuser.com/a/1532052/1091227 (Their post has images which I can't repost, so check it out for more detail.)

As described here Unified Logs: How to Enable Private Data you can create and install a configuration profile like this:

Profile to enable (reveal) private data

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>PayloadContent</key>
  <array>
    <dict>
      <key>PayloadDisplayName</key>
      <string>ManagedClient logging</string>
      <key>PayloadEnabled</key>
      <true/>
      <key>PayloadIdentifier</key>
      <string>com.apple.logging.ManagedClient.1</string>
      <key>PayloadType</key>
      <string>com.apple.system.logging</string>
      <key>PayloadUUID</key>
      <string>ED5DE307-A5FC-434F-AD88-187677F02222</string>
      <key>PayloadVersion</key>
      <integer>1</integer>
      <key>System</key>
      <dict>
        <key>Enable-Private-Data</key>
        <true/>
      </dict>
    </dict>
  </array>
  <key>PayloadDescription</key>
  <string>Enable Unified Log Private Data logging</string>
  <key>PayloadDisplayName</key>
  <string>Enable Unified Log Private Data</string>
  <key>PayloadIdentifier</key>
  <string>C510208B-AD6E-4121-A945-E397B61CACCF</string>
  <key>PayloadRemovalDisallowed</key>
  <false/>
  <key>PayloadScope</key>
  <string>System</string>
  <key>PayloadType</key>
  <string>Configuration</string>
  <key>PayloadUUID</key>
  <string>D30C25BD-E0C1-44C8-830A-964F27DAD4BA</string>
  <key>PayloadVersion</key>
  <integer>1</integer>
</dict>
</plist>

Save the file as YourProfileName.mobileconfig. If you don't need to sign it or deploy it you can just double-click and as a .mobileconfig it will automatically add to Profiles in System Preferences once you authenticate.

Monitoring unlocking Users and Groups in System Preferences on macOS Catalina 10.15.3 with (as suggested in the linked article) this command gives the following results:

sudo log stream --predicate '(subsystem == "com.apple.opendirectoryd") && (senderImagePath == "\/System\/Library\/OpenDirectory\/Modules\/PlistFile.bundle\/Contents\/MacOS\/PlistFile")'

• Without profile loaded <private> data (in this case the user unlocking) is redacted.

• With the profile loaded the previous <private> data is visible.

Comments

[Marek H]

As per mjtsai it seems this solution doesn't work on Catalina mjtsai.com/blog/2019/11/21/catalinas-log-cant-be-unprivatise‌​d

[pmdj]

This works brilliantly, thank you for sharing!

[MrTemple]

Cheers! I wish I'd found the solution ages ago.

[Chris]

This no longer works in Catalina. Apple has decided mere users shouldn't be able to view their logs. In Catalina 10.15.2 there was a hack to get around this by making the system think you work for Apple, but that was patched in the next release. The configuration profile is the only real solution now

[Chris]

I don't get Apple. The old private_data:on method was easy to toggle on/off and had to be re-enabled after every reboot. With the this new method, most people will just leave private logging on all the time. That's not a huge deal, but increases the danger if your logs get snooped. For example: by viewing mDNSResponder logs an otherwise low-privilege process can now see your historical dns queries/browsing habits. While trying to improve security, they've forced users to use a less secure/more error prone workaround. Seems worse for security/privacy overall. Or am I missing something?