How to configure squid4 as transparent proxy?

Question

There are many posts how to do that, most for older versions.

According to this post, to setup squid as transparent server for port 80 it should be as simple as this:

//squid.config
http_port 3128 transparent
http_port 80 accel #vhost option is deprecated
http_port 443 accel #not forget https 

but can't put it to work. i can't figure out if is my ip tables configuration, or the squid configuration.

Squid runs on the same pc as the gateway and dhcp server.

The iptables configuration:

# Generated by xtables-save v1.8.2 on Sun Feb  2 14:05:24 2020
*mangle
:PREROUTING ACCEPT [512813:147258305]
:INPUT ACCEPT [505693:146550975]
:FORWARD ACCEPT [5559:319150]
:OUTPUT ACCEPT [485369:200427691]
:POSTROUTING ACCEPT [486362:200704832]
COMMIT
# Completed on Sun Feb  2 14:05:24 2020
# Generated by xtables-save v1.8.2 on Sun Feb  2 14:05:24 2020
*filter
:INPUT ACCEPT [505677:146546207]
:FORWARD DROP [5532:317382]
:OUTPUT ACCEPT [485351:200422918]
-A FORWARD -p icmp -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 80 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 443 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 25 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 110 -j ACCEPT
-A FORWARD -p tcp -m tcp --dport 993 -j ACCEPT
-A FORWARD -p icmp -j ACCEPT
COMMIT
# Completed on Sun Feb  2 14:05:24 2020
# Generated by xtables-save v1.8.2 on Sun Feb  2 14:05:24 2020
*nat
:PREROUTING ACCEPT [55608:3516287]
:INPUT ACCEPT [48615:2816337]
:POSTROUTING ACCEPT [4525:272304]
:OUTPUT ACCEPT [17086:1270605]
-A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 127.1.1.0:3128
-A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A POSTROUTING -o eth0 -j MASQUERADE
-A POSTROUTING -o wlan0 -j MASQUERADE
COMMIT

The squid.config file:

acl LocalNet src 192.168.4.0/24 #my network . 
http_access allow LocalNet

acl localnet src 0.0.0.1-0.255.255.255
#acl localnet src 10.0.0.0/8
#acl localnet src 100.64.0.0/10
#acl localnet src 169.254.0.0/16
#acl localnet src 172.16.0.0/12
#acl localnet src 192.168.0.0/16
acl localnet src fc00::/7
acl localnet src fe80::/10
acl SSL_ports port 443
acl Safe_ports port 80  #http
acl Safe_ports port 21  #ftp
acl Safe_ports port 443 #https
acl Safe_ports port 70  #gopher
acl Safe_ports port 210 #wais
acl Safe_ports port 1025-65535  #unregisted ports
acl Safe_ports port 280 #http-mgmt
acl Safe_ports port 488 #gss-http
acl Safe_ports port 591 #filemaker
acl Safe_ports port 777 #multilining http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
#include /etc/squid/conf.d/*
http_access allow localhost
http_access allow all

#for explicit proxy
#http_port 3128
#http_port 3129 tproxy

#for transparent proxy
http_port 3128 transparent
http_port 80 accel
http_port 443 accel

#cache
cache_dir ufs /var/spool/squid 100 16 256
coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320
cache_effective_user proxy
cache_effective_group proxy

#log file
logformat timeread %tl %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
access_log daemon:/var/log/squid/access.log timeread

The squid start log:

squid[23403]: Starting Squid Cache version 4.6 for arm-unknown-linux-gnueabihf...
squid[23403]: Service Name: squid
squid[23403]: Process ID 23403
squid[23403]: Process Roles: worker
squid[23403]: With 1024 file descriptors available
squid[23403]: Initializing IP Cache...
squid[23403]: DNS Socket created at [::], FD 5
squid[23403]: DNS Socket created at 0.0.0.0, FD 9
squid[23403]: Adding domain Home from /etc/resolv.conf
squid[23403]: Adding nameserver 192.168.1.254 from /etc/resolv.conf
squid[23403]: Logfile: opening log daemon:/var/log/squid/access.log
squid[23403]: Logfile Daemon: opening log /var/log/squid/access.log
squid[23403]: Unlinkd pipe opened on FD 15
squid[23403]: Local cache digest enabled; rebuild/rewrite every 3600/3600 sec
squid[23403]: Store logging disabled
squid[23403]: Swap maxSize 102400 + 262144 KB, estimated 28041 objects
squid[23403]: Target number of buckets: 1402
squid[23403]: Using 8192 Store buckets
squid[23403]: Max Mem  size: 262144 KB
squid[23403]: Max Swap size: 102400 KB
squid[23403]: Rebuilding storage in /var/spool/squid (clean log)
squid[23403]: Using Least Load store dir selection
squid[23403]: Set Current Directory to /var/spool/squid
squid[23403]: Finished loading MIME types and icons.
squid[23403]: HTCP Disabled.
squid[23403]: Pinger socket opened on FD 22
squid[23403]: Squid plugin modules loaded: 0
squid[23403]: Adaptation support is off.
squid[23403]: Accepting NAT intercepted HTTP Socket connections at local=[::]:3128 remote=[::] FD 18 flags=41
squid[23403]: Accepting reverse-proxy HTTP Socket connections at local=[::]:80 remote=[::] FD 19 flags=9
squid[23403]: Accepting reverse-proxy HTTP Socket connections at local=[::]:443 remote=[::] FD 20 flags=9
squid[23403]: Done reading /var/spool/squid swaplog (0 entries)
squid[23403]: Store rebuilding is 0.00% complete
squid[23403]: Finished rebuilding storage from disk.
squid[23403]:         0 Entries scanned
squid[23403]:         0 Invalid entries.
squid[23403]:         0 With invalid flags.
squid[23403]:         0 Objects loaded.
squid[23403]:         0 Objects expired.
squid[23403]:         0 Objects cancelled.
squid[23403]:         0 Duplicate URLs purged.
squid[23403]:         0 Swapfile clashes avoided.
squid[23403]:   Took 0.05 seconds (  0.00 objects/sec).
squid[23403]: Beginning Validation Procedure
squid[23403]:   Completed Validation Procedure
squid[23403]:   Validated 0 Entries
squid[23403]:   store_swap_size = 0.00 KB
squid[23403]: storeLateRelease: released 0 objects

Update

iptables configuration update below, according to @Piotr P. Karwasz. Also i add log that can be see at /var/log/messages.

# Generated by xtables-save v1.8.2 on Wed Feb  5 18:05:20 2020
*mangle
:PREROUTING ACCEPT [26643:2488050]
:INPUT ACCEPT [26417:2421007]
:FORWARD ACCEPT [203:57945]
:OUTPUT ACCEPT [46685:14572738]
:POSTROUTING ACCEPT [47031:14661263]
COMMIT
# Completed on Wed Feb  5 18:05:20 2020
# Generated by xtables-save v1.8.2 on Wed Feb  5 18:05:20 2020
*filter
:INPUT ACCEPT [26404:2417368]
:FORWARD ACCEPT [7:448]
:OUTPUT ACCEPT [46671:14569146]
-A INPUT -j LOG
-A FORWARD -j LOG
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o eth0 -j ACCEPT
-A OUTPUT -j LOG
COMMIT
# Completed on Wed Feb  5 18:05:20 2020
# Generated by xtables-save v1.8.2 on Wed Feb  5 18:05:20 2020
*nat
:PREROUTING ACCEPT [612:69658]
:INPUT ACCEPT [577:59792]
:POSTROUTING ACCEPT [26:2979]
:OUTPUT ACCEPT [543:44473]
-A PREROUTING -j LOG
-A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A POSTROUTING -j LOG
-A POSTROUTING -o eth0 -j MASQUERADE
-A OUTPUT -j LOG
-A OUTPUT -o lo -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
COMMIT
# Completed on Wed Feb  5 18:05:20 2020
# Warning: iptables-legacy tables present, use iptables-legacy-save to see them

For clarity, here the command i use to filter for dns query hits:

tail -f /var/log/messages |grep -wi --color  -w 'UDP.*DPT=53'

Update 2

Bind 9 was blocking the dns querys. even stoped. Guess is not a regular service. I changed bind9 settings to allow dns querys from anyplace.

Now i got hits like this on the log, for port 53:

Feb  5 18:36:43 MyLinuxBox kernel: [11771.211903] IN=wlan0 OUT= MAC=b8:27:eb:35:e4:5b:48:43:7c:04:7e:55:08:00 SRC=192.168.42.19 DST=192.168.42.1 LEN=59 TOS=0x00 PREC=0x00 TTL=255 ID=43761 PROTO=UDP SPT=63906 DPT=5

It seams the dns query pass through the linux-box, from wlan0 to eth0, as is supposed too, but the response doesn't reach the client, resulting in a dns query timeout.

It is possible bind9 is still messing with the query.

Solution 1

Your iptables nat table should look like this:

-A OUTPUT -o lo -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A PREROUTING -i wlan0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
-A POSTROUTING -o eth0 -j MASQUERADE

This way both the web traffic from the wireless network and the machine itself will be sent to Squid. No masquerading is needed when sending packets to the wlan0 interface.

With a working Squid you can also delete the rule:

-A FORWARD -p tcp -m tcp --dport 80 -j ACCEPT

in the filter table.

Edit: Your DNS resolution problem comes from the restrictive FORWARD policy you have. At first you might start by allowing all outgoing traffic and the return packets:

-A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -o eth0 -j ACCEPT

and only when you know what kind of services you need, replace the second rule with more specific ones, e.g.:

-A FORWARD -o eth0 -p icmp -m icmp --icmp-type echo-request -m comment --comment ping -j ACCEPT
-A FORWARD -o eth0 -p udp -m udp --dport 53 -m comment --comment DNS -j ACCEPT
-A FORWARD -o eth0 -p tcp -m tcp --dport 53 -m comment --comment DNS -j ACCEPT
-A FORWARD -o eth0 -p tcp -m multiport --dports 25,465,587 -m comment --comment SMTP -j ACCEPT
-A FORWARD -o eth0 -p tcp -m multiport --dports 110,995 -m comment --comment POP3 -j ACCEPT
-A FORWARD -o eth0 -p tcp -m multiport --dports 143,993 -m comment --comment IMAP -j ACCEPT 

Solution 2

If your Squid proxy runs on your gateway, you would need the following iptables to redirect the traffic on HTTP/HTTPS to Squid (note here ens33 is the NIC facing the local network), see complete tutorial at https://docs.diladele.com/tutorials/transparent_proxy_debian/final.html.

# redirect HTTP to locally installed Squid instance
-A PREROUTING -i ens33 -p tcp --dport 80 -j REDIRECT --to-ports 3126

# redirect HTTPS to locally installed Squid instance
-A PREROUTING -i ens33 -p tcp --dport 443 -j REDIRECT --to-ports 3127

If you need to run your Squid on a separate box than your router/gateway, you'd need to use policy based routing, see tutorial at https://docs.diladele.com/tutorials/policy_based_routing_squid/index.html.

Comments

[Piotr P. Karwasz]

Which of your network interfaces is the external one (the one with a public IP)? If you are testing on the same machine as the router, the PREROUTING chain will never be hit.

[Miguel]

The eth0 has the public ip. Squid runs o a pc that as the 2 interfaces and routes traffic through him. The linux box should work as an hotspot.

[Piotr P. Karwasz]

Your WiFi clients are using Bind9 as their DNS server (the DHCP server provides the configuration). Therefore you need to set allow-recursion { localhost; 192.168.42.0/24; }; in the options section of Bind's config and keep it running. If everything passes through Bind, there is no need to accept DNS traffic in the FORWARD chain.

[Miguel]

can't put it to work with this configuration too. Squid is blocking the way, with access denied. Yet, if i send the traffic to port 3128, by setting proxy manually, it works.

[Piotr P. Karwasz]

Sorry, the OUTPUT rule I gave previously causes a redirection loop. It should be fixed now (with the -o lo option).

[Miguel]

It works, but dns query won't pass througth.I can only get sites by ip.

[Piotr P. Karwasz]

Yes, the devices on your WiFi network must be allowed to query a DNS server (port 53 of both udp and tcp), unless you install a DNS resolver like unbound on the router. I edited the answer with some details.

[Miguel]

I clear all rules, changed every default to accept, add the rules you mention. Still didn't work, so i stop the squid but still not working. I added log to every rule and then look for UDP packets on port 53. I only see hits coming from the localhost. i will update the post

[Miguel]

Made some progress. Somehow the bind9 service was blocking requests that come from the wlan0. When i set allow dns query from any place, requests start to appear on eth0, coming from wlan0. But still got dns timeout, so i thing the responses is not going back to the sender.. i will post details on question again.