终极安全代理与Tor
如今，在不断增长的Web 2.0环境中，您可能希望有一些匿名性，并使用其他IP地址比您自己的IP。 或者，为了一些特殊目的 - 几个IP或更多的IP，频繁更改。 所以没有人能跟踪你。 存在一个解决方案，它被称为Tor Project ，或者简单地说是Tor。 有很多文章和howtos给你的想法如何工作，我不会在这里描述洋葱路线及其原则，我宁愿告诉你，实际上如何拉出最大限度。
所以，让我们开始吧。
tor的主要缺点，其所有的好处如安全（但是，制造商建议不要依靠其强大的匿名性，因为有些情况下，当有可能跟踪年轻的h4x0r ...）是：
1.足够长的会话（每个出口节点一个连接60秒，这意味着您将有一个外部IP一分钟）
2.缓慢的性能（要求需要20秒才能完成，什么让冲浪网站有很多元素是灾难）
所有的请求通过一个节点，并且可能路由可以计算你的请求的到达。
为了打这三个，我们要用：
1. 8 tor进程，每个使用独立的spool目录
2. 8个进程，每个进程配置为分别进行通话。
第一个Squid，恶意软件域黑名单，将配置8个循环缓存对等体。 （Squid）
鲱鱼，Squid作为父母。 （反病毒代理，使用clamav :)）
第二个Squid，将使用havp作为父（squid-out）。 用户将连接到这个。
模式：
好的，所以你得到它，它是怎么样的。 现在我们来做所有必要的配置。
对于Squid - 约2Gb的光盘缓存就足够了。
对于squid-out，最好有大约12-15 Gb的光盘缓存来存储干净的元素。
所以你需要至少20 Gb硬盘。
现在关于RAM。
每个tor过程消耗约6 Mb的Ram。 6x8 = 48〜64 Mb的RAM用于Tor进程。
Privoxy每次吃约2 Mb。 2x8 = 16〜32Mb的RAM。
所以，128 Mb的RAM将足够用于Tor和Privoxy进程。
Squid-in将具有大约128 Mb的内存空间。 我们假设它吃了256 Mb。
同样的Squid出来。 我们得到了768 Mb的RAM总数。 如果它是一个真正的系统 - 我们假设1 Gb硬件RAM
已安装。
中央处理器。
正如你所知，防病毒扫描是非常CPU要求。
我使用了2.0 GHz P4。
我在10位用户的并发负载下测试了该系统，现在显示它已经耗尽了。
最后，我们有：
CPU：2.0 GHz
RAM：1.0 Gb
HDD：20 Gb
二手软件：
Privoxy
privoxy --version
Privoxy版本3.0.8（http://www.privoxy.org/）
Tor
tor --version
Oct 11 19:49:27.394 [notice] Tor v0.1.2.19. This is experimental software. Do not rely on it for strong anonymity.
Tor version 0.1.2.19.
Squid
squid -v
Squid Cache：Version 2.7.STABLE3 配置选项：'--prefix = / usr''--host = x86_64-pc-linux-gnu''--mandir = / usr / share / man' '--infodir = / usr / share / info''--datadir = / usr / share''--sysconfdir = / etc''--localstatedir = / var / lib' '--sysconfdir = / etc / squid''--libexecdir = / usr / libexec / squid''--localstatedir = / var' '--datadir = / usr / share / squid''--enable-auth = basic，digest，ntlm''--enable-removal-policies = lru，heap' '--enable-digest-auth-helpers = password' '--enable-basic-auth-helpers = SASL，PAM，SMB，多域NTLM，getpwnam，NCSA，MSNT' '--enable-external-acl-helpers = wbinfo_group，ip_user，session，unix_group' '--enable-ntlm-auth-helpers = SMB，fakeauth''--enable-ident-lookups''--enable-useragent-log' '--enable-cache-digests''--enable-delay-pools''--enable-referer-log''--enable-arp-acl''--with-pthreads' '--with-large-files''--enable-htcp''--enable-carp''--enable-follow-x-forwarding-for''--with-maxfd = 8192' '--enable-snmp''--enable-ssl''--enable-storeio = ufs，diskd，coss，aufs，null''--enable-async-io' '--enable-linux-netfilter''--libdir = / usr / lib64''--build = x86_64-pc-linux-gnu' 'build_alias = x86_64-pc-linux-gnu''host_alias = x86_64-pc-linux-gnu''CC = x86_64-pc-linux-gnu-gcc' 'CFLAGS = -march = k8 -O2 -pipe -fomit-frame-pointer''LDFLAGS = -Wl，-O1'
Tor配置
/etc/tor/torrc
SocksPort 9050 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor
/etc/tor/torrc2
SocksPort 9150 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor2 ControlPort 9151
/etc/tor/torrc3
SocksPort 9250 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor3 ControlPort 9251
/etc/tor/torrc4
SocksPort 9350 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor4 ControlPort 9351
/etc/tor/torrc5
SocksPort 9450 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor5 ControlPort 9451
/etc/tor/torrc6
SocksPort 9550 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor6 ControlPort 9551
/etc/tor/torrc7
SocksPort 9650 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor7 ControlPort 9651
/etc/tor/torrc8
SocksPort 9750 # what port to open for local application connections SocksBindAddress 127.0.0.1 # accept connections only from localhost AllowUnverifiedNodes middle,rendezvous Log notice syslog RunAsDaemon 1 User tor Group tor DataDirectory /var/lib/tor8 ControlPort 9751
privoxy的配置文件
/etc/privoxy/config
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8118 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9050 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config2
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy2 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8129 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9150 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config3
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy3 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8230 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9250 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config4
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy4 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8321 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9350 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config5
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy5 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8421 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9450 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config6
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy6 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8522 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9550 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config7
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy7 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8623 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9650 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
/etc/privoxy/config8
trust-info-url http://www.example.com/why_we_block.html trust-info-url http://www.example.com/what_we_allow.html confdir /etc/privoxy logdir /var/log/privoxy8 actionsfile standard.action # Internal purpose, recommended actionsfile default.action # Main actions file actionsfile user.action # User customizations filterfile default.filter logfile logfile listen-address 127.0.0.1:8724 toggle 1 enable-remote-toggle 0 enable-remote-http-toggle 0 enable-edit-actions 0 enforce-blocks 0 buffer-limit 4096 forward-socks4a / 127.0.0.1:9750 . forwarded-connect-retries 0 accept-intercepted-requests 0 allow-cgi-request-crunching 0 split-large-forms 0
启动脚本
Privoxy：/etc/rc.d/privoxy（Arch linux）
#!/bin/bash
# source application-specific settings
[ -f /etc/conf.d/privoxy ] && . /etc/conf.d/privoxy
. /etc/rc.conf
. /etc/rc.d/functions
#PID=`pidof -o %PPID /usr/sbin/privoxy`
start() {
stat_busy "Starting Privoxy, squid, havp"
chown privoxy:privoxy /var/log/privoxy
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy/$i
chmod 660 /var/log/privoxy/$i
done
chown privoxy:privoxy /var/log/privoxy2
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy2/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy2/$i
chmod 660 /var/log/privoxy2/$i
done
chown privoxy:privoxy /var/log/privoxy3
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy3/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy3/$i
chmod 660 /var/log/privoxy3/$i
done
chown privoxy:privoxy /var/log/privoxy4
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy4/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy4/$i
chmod 660 /var/log/privoxy4/$i
done
chown privoxy:privoxy /var/log/privoxy5
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy5/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy5/$i
chmod 660 /var/log/privoxy5/$i
done
chown privoxy:privoxy /var/log/privoxy6
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy6/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy6/$i
chmod 660 /var/log/privoxy6/$i
done
chown privoxy:privoxy /var/log/privoxy7
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy7/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy7/$i
chmod 660 /var/log/privoxy7/$i
done
chown privoxy:privoxy /var/log/privoxy8
# create missing logfiles
for i in logfile jarfile; do
touch /var/log/privoxy8/$i
chgrp $PRIVOXY_GROUP /var/log/privoxy8/$i
chmod 660 /var/log/privoxy8/$i
done
#[ -z "$PID" ] && /usr/sbin/privoxy $PRIVOXY_ARGS 2>/dev/null
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy.pid /etc/privoxy/config
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy2.pid /etc/privoxy/config2
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy3.pid /etc/privoxy/config3
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy4.pid /etc/privoxy/config4
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy5.pid /etc/privoxy/config5
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy6.pid /etc/privoxy/config6
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy7.pid /etc/privoxy/config7
/usr/sbin/privoxy $PRIVOXY_ARGS --pidfile /var/run/privoxy8.pid /etc/privoxy/config8
havp -c /etc/havp/havp.config
squid -f /etc/squid/squid-in.conf -DYC
squid -f /etc/squid/squid-out.conf -DYC
if [ $? -gt 0 ]; then
stat_fail
else
add_daemon privoxy
stat_done
fi
}
stop() {
stat_busy "Stopping Privoxy, Squid, Havp"
#[ ! -z "$PID" ] && kill $PID &> /dev/null
killall privoxy
killall squid
killall havp
if [ $? -gt 0 ]; then
stat_fail
else
rm_daemon privoxy
stat_done
fi
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
$0 stop
sleep 1
$0 start
;;
*)
echo "usage: $0 {start|stop|restart}"
esac
exit 0
Tor启动脚本：/etc/rc.d/tor（Arch linux）
#!/bin/bash
. /etc/rc.conf
. /etc/rc.d/functions
#PID=`pidof -o %PPID /usr/bin/tor`
tor='/usr/bin/tor'
case "$1" in
start)
stat_busy "Starting Tor Daemon"
#[ -z "$PID" ] && /usr/bin/tor &>/dev/null
${tor} -f /etc/tor/torrc
${tor} -f /etc/tor/torrc2
${tor} -f /etc/tor/torrc3
${tor} -f /etc/tor/torrc4
${tor} -f /etc/tor/torrc5
${tor} -f /etc/tor/torrc6
${tor} -f /etc/tor/torrc7
${tor} -f /etc/tor/torrc8
if [ $? -gt 0 ]; then
stat_fail
else
add_daemon tor
stat_done
fi
;;
stop)
stat_busy "Stopping Tor Daemon"
#[ ! -z "$PID" ] && kill $PID &> /dev/null
killall tor
if [ $? -gt 0 ]; then
stat_fail
else
rm_daemon tor
stat_done
fi
;;
restart)
$0 stop
sleep 3
$0 start
;;
*)
echo "usage: $0 {start|stop|restart}"
esac
exit 0
# vim: ft=sh ts=2 sw=2
Squid-in配置
/etc/squid/squid-in.conf
acl all src all acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl malware_domains url_regex '/etc/squid/Malware-domains.txt' http_access deny malware_domains http_access allow localhost http_access deny all icp_access deny all http_port 3400 icp_port 0 hierarchy_stoplist cgi-bin ? refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 cache_peer localhost parent 8118 0 round-robin no-query cache_peer localhost2 parent 8129 0 round-robin no-query cache_peer localhost3 parent 8230 0 round-robin no-query cache_peer localhost4 parent 8321 0 round-robin no-query cache_peer localhost5 parent 8421 0 round-robin no-query cache_peer localhost6 parent 8522 0 round-robin no-query cache_peer localhost7 parent 8623 0 round-robin no-query cache_peer localhost8 parent 8724 0 round-robin no-query never_direct allow all always_direct deny all acl apache rep_header Server ^Apache broken_vary_encoding allow apache forwarded_for off coredump_dir /var/cache/squid-in cache_dir ufs /var/cache/squid-in 100 16 256 pid_filename /var/run/squid-in.pid access_log /var/log/squid/access.squid-in.log cache_store_log /var/log/squid/store.squid-in.log cache_log /var/log/squid/cache.squid-in.log
Squid-out配置
/etc/squid/squid-out.conf
acl all src all acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localnet http_access allow localhost http_access deny all icp_access deny all http_port 3128 icp_port 0 hierarchy_stoplist cgi-bin ? refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320 cache_peer localhost parent 3410 0 round-robin no-query never_direct allow all always_direct deny all acl apache rep_header Server ^Apache broken_vary_encoding allow apache forwarded_for off coredump_dir /var/cache/squid-out cache_dir ufs /var/cache/squid-out 100 16 256 pid_filename /var/run/squid-out.pid access_log /var/log/squid/access.squid-out.log cache_store_log /var/log/squid/store.squid-out.log cache_log /var/log/squid/cache.squid-out.log
Havp防病毒代理配置
/etc/havp/havp.config
USER havp GROUP havp DAEMON true PIDFILE /tmp/havp.pid SERVERNUMBER 4 MAXSERVERS 256 SCANTEMPFILE /tmp/havp-XXXXXX TEMPDIR /tmp DBRELOAD 60 PARENTPROXY localhost PARENTPORT 3400 FORWARDED_IP false X_FORWARDED_FOR false PORT 3410 BIND_ADDRESS 127.0.0.1 ENABLECLAMLIB true ENABLECLAMD false ENABLEFPROT false ENABLEAVG false ENABLEAVESERVER false ENABLESOPHIE false ENABLETROPHIE false ENABLENOD32 false ENABLEAVAST false ENABLEARCAVIR false ENABLEDRWEB false
域列表更新脚本（可从cron启动）
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Author: Bohdan Turkynewych, AKA Gh0st, 2006-2008, tb0hdan[at]gmail[dot]com
# Distributed under the terms of GPLv2
# Parts of code from neutron jabber bot project, http://code.google.com/p/neutron/
from urllib2 import Request as urllib2_Request, urlopen as urllib2_urlopen
from sys import exit as sys_exit
from re import search as re_search
from time import ctime as time_ctime, time as time_time
from re import compile as re_compile
def t_conv(timestamp):
reply = ''
seconds = timestamp % 60
minutes = (timestamp / 60) % 60
hours = (timestamp / 3600) % 60
days = timestamp / 216000
if days: reply += str(days) + ' day(s) '
if hours: reply += str(hours) + ' hour(s) '
if minutes: reply += str(minutes) + ' minute(s) '
reply += str(seconds) + ' second(s)'
return reply
def uptime():
global BOOTUP_TIMESTAMP
if BOOTUP_TIMESTAMP:
idletime = int(time_time() - BOOTUP_TIMESTAMP)
reply = t_conv(idletime)
else:
reply = 'Unknown'
return reply
strip_tags = re_compile(r'<[^]+>')
def decode(text):
data = text.replace('<br>','\n').replace(' ', ' ').replace('<', '<').replace('>', '>').replace('"', '"').replace('<br>','\n').replace('
<li>','\r\n')
return strip_tags.sub('', data)
def get_data(message):
reply=[]
try:
for line in message.split('\n'):
if re_search('wbr',line):
#
od = re_search('',line)
d0main = line[od.end():]
d0main = d0main[:re_search('',d0main).start()].strip()
#
if d0main == '-':
od = re_search('-',line)
bookurl = line[od.end():]
bookurl = bookurl[:re_search('',bookurl).start()].strip()
bookurl = decode(bookurl)
else:
bookurl = decode(d0main)
d0main = ''
stripper = '[/|:]'
if re_search(stripper,bookurl):
#d0main = bookurl
od = re_search(stripper,bookurl)
d0main = bookurl[od.end():]
d0main = bookurl[:re_search(stripper,bookurl).start()].strip()
else:
d0main = bookurl
if not d0main in reply:
reply.append(d0main)
return reply
except:
raise
def update_list(filename):
req2 = urllib2_Request('http://www.malwaredomainlist.com/mdl.php?sort=Domain&ascordesc=DESC&search=&colsearch=All&quantity=All')
req2.add_header = ('User-agent', 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.9) Gecko/20071030 SeaMonkey/1.1.6')
print '[+] Update process started at ' + time_ctime()
try:
r = urllib2_urlopen(req2)
target = r.read()
urls = get_data(target)
except:
raise
print '[-] Error occured while fetching / parsing data.'
sys_exit(1)
print '[+] Got ' + str(len(urls)) + ' domains/ips.'
try:
file = open(filename,'w')
for line in urls:
file.write(line + '\n')
file.close()
except:
print '[-] Error occured while flushing data to file'
sys_exit(5)
elapsed = uptime()
print '[+] Update process finished at ' + time_ctime() + '. Took ' + elapsed
BOOTUP_TIMESTAMP = time_time()
update_list('/etc/squid/Malware-domains.txt')
玩的开心！
PS未来优化还需要8个havp实例，因为减速。
PSS您可以从系统中排除havp和squid-out代理，并且只能使用squid-in。 那会快得多，
