当前位置：首页 → 问答吧 → 搭建pf on freebsd 实现负载均衡出现的问题。

搭建pf on freebsd 实现负载均衡出现的问题。

时间：2010-08-13

来源：互联网

本帖最后由 adastudy 于 2010-08-13 17:31 编辑

最近公司让我测试一下lvs on freebsd ,pf on freebsd
lvs on freebsd 6.1 初步测试（DR模式）成功
pf on freebsd 负载均衡（NAT模式）也是测试成功
但在测试pf on freebsd 负载均衡（重定向）具体我也不知道叫什么模式，出现问题，无法正常请求http。经过抓包分析，发贴来求助
B(实体服务 http)
/
拓扑 (客户端访问)D-----A (pf director)
\
C(实体服务 http)

复制代码

D ip:192.168.1.234
A  :192.168.1.249
B:192.168.1.51
C:192.168.1.52

四台都处于同一物理网段 192.168.1.x/24

D是宿主机(当客户端)
A,B,C都是vmware的guest os

A是运行pf 的freebsd 8.0 (当director)
B,C是(运行nginx，当具体服务机)

pf设置的策略为当A收到来自80,443的请求，会重定向到B，C

以下是配置的内容:

outing# pfctl -vnf /etc/pf.conf
lop_if = "lo0"
ext_if = "em0"
int_if = "em1"
web_ports = "{ 80, 443 }"
web_servers = "{ 192.168.1.51,192.168.1.52 }"
set block-policy drop
set skip on { lo0 }
scrub in on em0 all fragment reassemble
nat on em0 inet from 10.0.0.0/24 to any -> 192.168.1.249
rdr on em0 inet proto tcp from any to any port = http -> { 192.168.1.51, 192.168.1.52 } round-robin
rdr on em0 inet proto tcp from any to any port = https -> { 192.168.1.51, 192.168.1.52 } round-robin
pass in all flags S/SA keep state
pass out all flags S/SA keep state

实际配置文件

lop_if="lo0"
ext_if="em0" //pf server 的外网口 ip为192.168.1.249
int_if="em1" //pf server内网口
web_ports = "{ 80, 443 }" //定义 port of services
web_servers = "{ 192.168.1.51,192.168.1.52 }" //定义两台实体服务机 B:192.168.1.51 C:192.168.1.52
#################################################
#
#options ,scrub and NAT
#
#################################################
set block-policy drop
set skip on $lop_if
scrub in on $ext_if
nat on $ext_if from $int_if:network to any -> $ext_if //定义实体服务的内网口的数据包走nat
################################################################################
# Redirection
################################################################################
#rdr on $ext_if proto tcp from any to any port 80 -> $web_servers round-robin sticky-address
#rdr on $ext_if proto tcp from any to $ext_carp port $web_ports -> $web_servers round-robin
rdr on $ext_if proto tcp from any to any port $web_ports -> $web_servers round-robin //定义凡是属于服务组的请求rdr到 web server组
################################################################################
# Filtering Rules
################################################################################
pass in all
pass out all

复制代码

抓包结果

2:36:40.677031 IP 192.168.1.234.1472 > 192.168.1.249.80: Flags [S], seq 2852452265, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:40.680101 IP 192.168.1.234.1472 > 192.168.1.51.80: Flags [S], seq 2852452265, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:40.680841 IP 192.168.1.51.80 > 192.168.1.234.1472: Flags [S.], seq 1793702654, ack 2852452266, win 65535, options [mss 1460,nop,wscale 1,sackOK,eol], length 0
02:36:40.928738 IP 192.168.1.234.1473 > 192.168.1.249.80: Flags [S], seq 3795059101, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:40.929378 IP 192.168.1.234.1473 > 192.168.1.52.80: Flags [S], seq 3795059101, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:40.930026 IP 192.168.1.52.80 > 192.168.1.234.1473: Flags [S.], seq 1581156137, ack 3795059102, win 65535, options [mss 1460,nop,wscale 1,sackOK,eol], length 0
02:36:43.570606 IP 192.168.1.234.1472 > 192.168.1.249.80: Flags [S], seq 2852452265, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:43.571034 IP 192.168.1.234.1472 > 192.168.1.51.80: Flags [S], seq 2852452265, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0
02:36:43.572853 IP 192.168.1.51.80 > 192.168.1.234.1472: Flags [S.], seq 1793702654, ack 2852452266, win 65535, options [mss 1460,nop,wscale 1,sackOK,eol], length 0
02:36:43.771705 IP 192.168.1.234.1473 > 192.168.1.249.80: Flags [S], seq 3795059101, win 65535, options [mss 1460,nop,wscale 1,nop,nop,sackOK], length 0

复制代码

这是抓包结果，我的hosts主机ip是　192.168.1.234(客户端)，pf director是1.249 两台负载均衡的服务机子是1.51,1.52

从上面抓包的结果可以看到

1.249接收到1.234的一个80请求包(syn)之后，接着1.51也收到一个同样序列号的syn(应该是转发方式)，接着1.51就回应syn一个ack

但是仅接着1.234并没有再次确认1.51的ack，这个http请求就终止在这个地方了，为什么？

作者: adastudy 发布时间: 2010-08-13

从抓包上看,D向A:80提出请求,由B或C直接应答,D应该会认为这些包是非法的.

作者: congli 发布时间: 2010-08-14

D经过A之后,源IP应该改为A的IP.否则B或C会直接应答,因为在同一局域网.

作者: congli 发布时间: 2010-08-14