Transparent with Squid Proxy
Install Squid, SquidGuard and Sarg
yum install squidGuard sarg squid-helpers squid-debuginfo squid perl-Crypt-OpenSSL openssl httpd httpd-tools mod_ssl gd gd-devel make perl-GDConfiguration Squid
1. Make Certificates
cd /var/opt/squid
mkdir certs
cd certs
openssl req -new -newkey rsa:2048 -days 36500 -nodes -x509 -keyout proxyCA.pem -out proxyCA.pem
chown squid:squid ../certs
cd ..
mkdir -p ssldb
/usr/lib64/squid/ssl_crtd -s /var/opt/squid/ssldb/ssl_db -M 1002. Kernel Parameter
configure in /etc/sysctl.conf
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.accept_source_route = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 68719476736
kernel.shmall = 4294967296
kernel.core_uses_pid=1
kernel.ctrl-alt-del = 0
kernel.kptr_restrict = 2
kernel.randomize_va_space = 2
kernel.sysrq = 0
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.rp_filter = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.log_martians = 1
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_timestamps = 0
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.all.accept_source_route = 0
net.ipv6.conf.default.accept_redirects = 0
net.ipv6.conf.default.accept_source_route = 0
vm.swappiness = 10
fs.file-max = 65535
net.core.rmem_default = 262144
net.core.rmem_max = 262144
net.core.wmem_default = 262144
net.core.wmem_max = 262144
net.ipv4.tcp_rmem = 4096 87380 8388608
net.ipv4.tcp_wmem = 4096 65536 8388608
net.ipv4.tcp_mem = 4096 4096 4096
net.ipv4.tcp_low_latency = 1
net.core.netdev_max_backlog = 4000
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_max_syn_backlog = 16384
net.ipv4.tcp_ecn = 0and then reload parameter
sysctl -p3. IP Tables
replace file iptables in /etc/sysconfig/
*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-t nat -A PREROUTING -p tcp -m tcp --dport 443 -j DNAT --to-destination 10.32.16.222:3127
-t nat -A PREROUTING -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.32.16.222:3128
-t nat -A POSTROUTING -j MASQUERADE
COMMIT
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [3455:321198]
:LOGGING - [0:0]
-A INPUT -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -j ACCEPT
-A INPUT -p tcp -s 10.32.5.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.15.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.6.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.7.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.8.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.9.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.10.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.11.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.13.0/24 -j ACCEPT
-A INPUT -p tcp -s 10.32.14.0/24 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m multiport --dports 80,443,3127,3128,8081 -j ACCEPT
-A INPUT -p tcp -m state --state NEW,ESTABLISHED -m multiport --dports 3000,3001 -j ACCEPT
-A INPUT -j LOGGING
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
-A LOGGING -m limit --limit 5/min -j LOG --log-prefix "Incoming Dropped: " --log-level 5
-A INPUT -j DROP
COMMIT4. Routing
create file route-eth0 in /etc/sysconfig/network-scripts/
10.32.5.0/24 via 10.32.16.221
10.32.15.0/24 via 10.32.16.221
10.32.6.0/24 via 10.32.16.221
10.32.7.0/24 via 10.32.16.221
10.32.8.0/24 via 10.32.16.221
10.32.9.0/24 via 10.32.16.221
10.32.10.0/24 via 10.32.16.221
10.32.11.0/24 via 10.32.16.221
10.32.13.0/24 via 10.32.16.221
10.32.14.0/24 via 10.32.16.2215. Squid Config
Configuration main squid.conf in /etc/squid/
...
cache_dir ufs /var/spool/squid 400000 32 512
coredump_dir /var/spool/squid
http_port 8181
http_port 3128 intercept
https_port 3127 intercept ssl-bump cert=/var/opt/squid/certs/proxyCA.pem
sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/opt/squid/ssldb/ssl_db -M 100
...details can be download : Link
Details:
1. Port 8181 is using for trapping if there is bypassing squid server
2. Port 3128 is using in traffic http
3. Port 3127 is using in traffic https
6. Create Cache in Squid
move to directory /var/spool/squid/
cd /var/spool/squid/
squid -z
chown squid:squid -R /var/spool/squid7. Manage Service Squid
service squid startor
/etc/init.d/squid startenable startup
chkconfig squid onConfiguration SquidGuard
1. Configure Blacklist Site
Download Blacklist list in URL
Extract All file blacklist in /var/squidGuard/db/blacklist
Listing blacklist :
....
adult
blacklist
malware
ddos
filesharing
mixed_adult
updatesites
whitelist
...2. Create ACL SquidGuard
Common Pattern ACL is Source, Destination and ACL
Source :
src mgt {
ip 10.32.14.0/255.255.255.0
}Destination :
dest job {
domainlist job/domains
urllist job/urls
redirect https://squidshare.waajtmaka.com/job.html?
}ACL :
acl {
mgt {
pass !job all
}Information above is the mean IP segmen 10.32.14.0/24 deny access web JOB database, but anothet that is allowed.
3. Generate Destination
Many Destination DB blacklist, for generate can be do, following :
cd /var/squidGuard/db/blacklistsand copy this script to /var/squidGuard/db/blacklists
Copy result hasil.txt to /etc/squid/squidGuard.conf after the end line source Configuration existing :
############# TIME RULES ####################
time free {
weekly mtwhf 08:00-09:00 12:00-13:00 18:00-20:00
}
........
############# SOURCE ADDRESSES ####################
src ga {
ip 10.32.11.0/255.255.255.0
}
........
############# DESTINATION ADDRESSES ####################
dest dewasa {
domainlist dewasa/domains
urllist dewasa/urls
expressionlist dewasa/expressions
redirect https://squidshare.wajatmaka.com/banned.html?
}
........
############# ACCESS CONTROL LIST ####################
acl {
ga within free {
pass boleh !manga !blacklist !malware !job !banned !ddos !job !malware !porn !dewasa !iklan !weapons !social_networks !adult !chat any
}
.......
default {
pass none
redirect https://squidshare.wajatmaka.com/workinghour.html?
}
}Configuration Full can be download
4. Add Configuration SquidGuard in Squid
add in /etc/squid/squid.conf after end line acl
url_rewrite_program /usr/bin/squidGuard -c /etc/squid/squidGuard.conf5. Generate DB SquidGuard
squidGuard -b -d -C all
chown squid:squid -R /var/squidGuard/6. Manage Service SquidGuard
chkconfig squidGuard on
service squid restartor
/etc/init.d/squid restartConfiguration SARG
1. Create VHost for SARG
enable mod ssl in apache, for checking
httpd -t -D DUMP_MODULES | grep ssladd virtualhost in /etc/httpd/conf.d/ssl.conf
############### DOCUMENT ROOT ###################
DocumentRoot "/home/squid/sarg/reports"
ServerName squidshare.roomit.com
DirectoryIndex at_domains_index.html index.html index.html.var index.shtml index.php index.htm
\
Options -Indexes -Includes
Order deny,allow
deny from 10.32.6.0/24
deny from 10.32.14.0/24
deny from 10.32.9.0/24
deny from 10.32.7.0/24
deny from 10.32.13.0/24
deny from 10.32.11.0/24
allow from 10.32.5.0/24
\
........
Full Configuration can be download
2. Secure VHost for SARG
add paramater in /etc/httpd/conf.d/ssl.conf before line VirtualHost
Header always set Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"
Header edit Set-Cookie ^(.*)$ $1;HttpOnly;Secure
Header always append X-Frame-Options SAMEORIGIN
ServerTokens Prod
ServerSignature Off
FileETag None
TraceEnable offAdd reports SARG with symbolic link
ln -s /home/squid/sarg/reports /var/www/html3. Optimize Configuration SARG
Change configuration in /etc/sarg/sarg.conf
access_log /var/log/squid/access.log
output_dir /home/squid/sarg/reports
date_format e
overwrite_report yes4. Change IP to Hostname in SARG
Create file wajatmaka.usertab in /etc/sarg/, list of name people, the following :
......
10.32.6.6 dev.DinaFarhah-PC
10.32.6.7 dev.GiginGinanjar-PC
.......
add parameter /etc/sarg/sarg.conf
usertab /etc/sarg/wajatmaka.usertab5. Generate Report in SARG
Before generate restart apache
/etc/init.d/httpd restartGenerate report
sarg -xcheck using browser, access url https://localhost/
Customize Page Banned in Squid
Download Link
make symbolic link
ln -sf /var/www/html/page-error/banned.html /var/www/html/reports/
ln -sf /var/www/html/page-error/job.html /var/www/html/reports/
ln -sf /var/www/html/page-error/iklan.html /var/www/html/reports/
ln -sf /var/www/html/page-error/workinghour.html /var/www/html/reports/
ln -s /var/www/html/page-error/img/squid-icon.png /var/www/html/reports/images/For manage redirecting we can manage in /etc/squid/squidGuard.conf. In Destination add line
redirect https://squidshare.wajatmaka.com/banned.html?using ? in end rows.
Optimize Parameter in Squid
We can tuning squid change value paramater following :
cache_mem 100 MB
maximum_object_size_in_memory 50 KB
memory_cache_mode always
minimum_object_size 0 KB
maximum_object_size 200 MB
memory_pools off
quick_abort_min 0 KB
quick_abort_max 0 KB
log_icp_queries off
cache_replacement_policy heap LFUDA
memory_replacement_policy heap LFUDA
half_closed_clients off
max_filedesc 4096
redirect_children 37
max_filedesc 65535
client_db off
hosts_file /etc/hostsFor Limit File header will be downloading :
########### SOURCE ###########
acl developer src 10.32.6.0/24 # VLAN 6
acl telco src 10.32.9.0/24 # VLAN 9
acl marketing src 10.32.13.0/24 # VLAN 11
acl hrd src 10.32.8.0/24 # VLAN 8
acl finance src 10.32.7.0/24 # VLAN 7
acl ga src 10.32.11.0/24 # VLAN 10
########### BANDWIDTH LIMIT FOR DOWNLOAD ########
reply_body_max_size 300 MB developer
reply_body_max_size 300 MB telco
reply_body_max_size 100 MB marketing
reply_body_max_size 100 MB hrd
reply_body_max_size 100 MB finance
reply_body_max_size 100 MB gaSquidGuard Managenment
1. Add User IP
If there any workstation/server cannot be access internet, but can only access website http, example can be access http://mirror.centos.org. we can add in /etc/squid/squidGuard.conf. Classification of ip, example we add ip 10.32.15.101 so this is using source qca. add in end of source qca
src qca {
.....
ip 10.32.15.41
ip 10.32.15.70
ip 10.32.15.242
ip 10.32.15.136
......
10.32.15.101
}Reload squid
/etc/init.d/squid reloador
squid -k reconfigure -f /etc/squid/squid.conf2. Block Domains
example, we will block website http://localhost going to /var/squidGuard/db/blacklists/ add in file vi blacklist/domains in end of file rows without http://
.......
youtube.com
blog.wajatmaka.comso, update db using
squidGuard -C blacklist/domainsReload squid
/etc/init.d/squid reloador
squid -k reconfigure -f /etc/squid/squid.conf3. Unblock Domains
example, we will allow website https://cimbniaga.com going to /var/squidGuard/db/blacklists/
grep "cimbniaga" */*and output is
blacklistlist/domains:cimbniaga.comsearch and remove cimbniaga.com in file vi blacklist/domains.
.......
cimbniaga.com
youtube.com
.......so, update db using
squidGuard -C blacklist/domainsReload squid
/etc/init.d/squid reloador
squid -k reconfigure -f /etc/squid/squid.confSquid Monitoring, Problem And Solving
ree RAM must greater then 230 MB, if not squid will collapse
free -m | grep Mem | awk -F" " '{print $4,"MB"}'Number of squidGuard child for link redirection must be less then 24, if child greater then 24 then RAM usage will increase. If squidguard child is limited, then request will be queued for url filtering.
[root@squidshare senops]# cat /var/log/squid/cache.log| grep -i warning
| grep -i busy2017/04/25 10:38:13 kid1| WARNING: All 40/40 redirector
processes are busy.2017/04/25 10:46:32 kid1| WARNING: All 40/40
redirector processes are busy.[root@squidshare senops]# cat /var/log/squid/cache.log| grep -i warning | grep -i pending2017/04/25
10:38:13 kid1| WARNING: 40 pending requests queued2017/04/25 10:46:32
kid1| WARNING: 40 pending requests queuedSquid listening Port list :
[root@squidshare blacklists]# netstat -tulpna | grep LISTEN
tcp 0 0 10.32.16.222:2812 0.0.0.0:* LISTEN 15511/monit
tcp 0 0 127.0.0.1:199 0.0.0.0:* LISTEN 29185/snmpd
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 23213/httpd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 6119/sshd
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 1716/ntop
tcp 0 0 0.0.0.0:3001 0.0.0.0:* LISTEN 1716/ntop
tcp 0 0 10.32.16.222:443 0.0.0.0:* LISTEN 23213/httpd
tcp 0 0 :::8181 :::* LISTEN 26568/(squid-1)
tcp 0 0 :::22 :::* LISTEN 6119/sshd
tcp 0 0 :::3127 :::* LISTEN 26568/(squid-1)
tcp 0 0 :::3128 :::* LISTEN 26568/(squid-1)Important PORT : 3128, 3127,8181
If SQUID COLLAPSE or DOWN and restart or stop/start SQUID is failed, then stop squid, kill all squid process, flush RAM and reduce redirect_children to 20 child in /etc/squid/squid.conf.
Stop squid :
/etc/init.d/squid stopKill all squid process :
pgrep squid | xargs kill -9Flush RAM :
/usr/bin/wjt-clear-cacheor
sync;echo 3 > /proc/sys/vm/drop_cachesstart service squid/etc/init.d/squid startor
service squid startScript tunning in Link
Crontab
AILTO=""
*/2 * * * * /usr/sbin/ntpdate -u 10.32.16.237
55 23 * * * /usr/bin/sarg -x
#### CHECK OPTIMIZE CACHE RAM ######
* * * * * /usr/bin/wjt-check-ram
* * * * * /usr/bin/wjt-check-children
#### SCHEDULE AUTOMATIC OPTIMIZE CACHE RAM #####
### MALAM ###
58 23 * * * /usr/bin/wjt-clear-cache
59 23 * * * /usr/bin/wjt-check-children
### SIANG ###
30 12 * * * /usr/bin/wjt-clear-cache
33 12 * * * /usr/bin/wjt-check-children
### PAGI ###
30 8 * * * /usr/bin/wjt-clear-cache
33 8 * * * /usr/bin/wjt-check-children
#### SNMP PERFORMANCE SERVER MONITORING ####
*/5 * * * * /etc/snmp/local-snmp-cronjob > /dev/null 2>&1
*/10 * * * * /etc/snmp/vmstat-cron /var/local/snmp/vmstat
*/10 * * * * /etc/snmp/iostat-cron /var/local/snmp/iostat
#### INTEGRITY FILE #####
12 1 * * * /usr/local/sbin/samhain -t