自定义H111协议分析与请求走私漏洞研究<\/h1>

1. H111协议概述<\/h2>

H111是一种自定义的HTTP协议变体，用于在代理服务器前后端之间传输HTTP请求和响应。该协议采用二进制格式进行序列化和反序列化，主要特点包括：<\/p>

使用固定长度的字段表示可变长度数据<\/li>
采用大端序(网络字节序)进行数据编码<\/li>
支持连接复用和pipeline处理<\/li>

设计上类似于HTTP但采用二进制格式<\/li> <\/ul>

2. H111协议格式详解<\/h2>

2.1 请求格式<\/h3>

H111协议的请求报文由以下部分组成，按顺序排列：<\/p>

字段<\/th> 长度<\/th> 描述<\/th> <\/tr> <\/thead>

方法长度<\/td> 2字节 (uint16)<\/td> 方法名称的长度(如GET、POST)<\/td> <\/tr>

方法名称<\/td> N字节<\/td> 方法名称的实际数据<\/td> <\/tr>

URI长度<\/td> 2字节 (uint16)<\/td> URI的长度<\/td> <\/tr>

URI<\/td> M字节<\/td> URI的实际数据(如\/path)<\/td> <\/tr>

请求头数量<\/td> 2字节 (uint16)<\/td> 请求头的总数<\/td> <\/tr>

请求头<\/td> 变长<\/td> 每个请求头的键值对<\/td> <\/tr>

请求体长度<\/td> 2字节 (uint16)<\/td> 请求体的长度<\/td> <\/tr>

请求体<\/td>

K字节<\/td>

请求体的实际数据<\/td> <\/tr> <\/tbody> <\/table>

2.2 请求头格式<\/h3>

每个请求头由以下部分组成：<\/p>

字段<\/th> 长度<\/th> 描述<\/th> <\/tr> <\/thead>

键长度<\/td> 2字节 (uint16)<\/td> 键名的长度<\/td> <\/tr>

键数据<\/td> 变长<\/td> 键名的实际数据<\/td> <\/tr>

值长度<\/td> 2字节 (uint16)<\/td> 值的长度<\/td> <\/tr>

值数据<\/td>

变长<\/td>

值的实际数据<\/td> <\/tr> <\/tbody> <\/table>

2.3 响应格式<\/h3>

H111协议的响应报文格式与请求类似，但包含状态码字段：<\/p>

字段<\/th>	长度<\/th>	描述<\/th> <\/tr> <\/thead>
状态码<\/td>	2字节 (uint16)<\/td>	HTTP状态码<\/td> <\/tr>
状态文本长度<\/td>	2字节 (uint16)<\/td>	状态文本的长度<\/td> <\/tr>
状态文本<\/td>	N字节<\/td>	状态文本的实际数据<\/td> <\/tr>
响应头数量<\/td>	2字节 (uint16)<\/td>	响应头的总数<\/td> <\/tr>
响应头<\/td>	变长<\/td>	每个响应头的键值对<\/td> <\/tr>
响应体长度<\/td>	2字节 (uint16)<\/td>	响应体的长度<\/td> <\/tr>
响应体<\/td>	K字节<\/td>	响应体的实际数据<\/td> <\/tr> <\/tbody> <\/table> 3. 协议实现细节<\/h2> 3.1 序列化过程<\/h3> H111协议的序列化过程主要涉及以下步骤：<\/p> 写入方法信息<\/strong>：<\/p> 使用`binary.Write<\/code>写入2字节的方法长度(uint16)<\/li>` `写入方法名称的字节数据<\/li> <\/ul> <\/li>` 写入URI信息<\/strong>：<\/p> 使用binary.Write<\/code>写入2字节的URI长度(uint16)<\/li> 写入URI的字节数据<\/li> <\/ul> <\/li> 写入请求头<\/strong>：<\/p> 写入2字节的请求头数量(uint16)<\/li> 对于每个请求头：写入2字节的键长度(uint16)<\/li> 写入键数据<\/li> 写入2字节的值长度(uint16)<\/li> 写入值数据<\/li> <\/ul> <\/li> <\/ul> <\/li> 写入请求体<\/strong>：<\/p> 写入2字节的请求体长度(uint16)<\/li> 写入请求体数据<\/li> <\/ul> <\/li> <\/ol> 3.2 反序列化过程<\/h3> H111协议的反序列化过程与序列化相反：<\/p> 读取方法信息<\/strong>：<\/p> 使用binary.Read<\/code>读取2字节的方法长度(uint16)<\/li> 使用io.ReadFull<\/code>读取方法名称数据<\/li> <\/ul> <\/li> 读取URI信息<\/strong>：<\/p> 读取2字节的URI长度(uint16)<\/li> 读取URI数据<\/li> <\/ul> <\/li> 读取请求头<\/strong>：<\/p> 读取2字节的请求头数量(uint16)<\/li> 对于每个请求头：读取2字节的键长度(uint16)<\/li> 读取键数据<\/li> 读取2字节的值长度(uint16)<\/li> 读取值数据<\/li> <\/ul> <\/li> <\/ul> <\/li> 读取请求体<\/strong>：<\/p> 读取2字节的请求体长度(uint16)<\/li> 读取请求体数据<\/li> <\/ul> <\/li> <\/ol> 4. 安全漏洞分析<\/h2> 4.1 整数溢出漏洞<\/h3> H111协议存在严重的安全漏洞，主要问题在于：<\/p> 长度字段使用uint16<\/strong>：<\/p> 所有长度字段(方法长度、URI长度、请求头数量、键值长度、请求体长度)都使用uint16类型<\/li> uint16的最大值为65535(0xFFFF)<\/li> <\/ul> <\/li> 缺乏溢出检查<\/strong>：<\/p> 当实际长度超过65535时会发生整数溢出<\/li> 例如：长度为65536(0x10000)会被截断为0(0x0000)<\/li> <\/ul> <\/li> 数据截断问题<\/strong>：<\/p> 当长度溢出为0时，协议解析器会读取0字节数据<\/li> 剩余数据会被"搁置"，可能被误认为是下一个请求的一部分<\/li> <\/ul> <\/li> <\/ol> 4.2 请求走私攻击<\/h3> 利用上述整数溢出漏洞，可以构造请求走私攻击：<\/p> 攻击原理<\/strong>：<\/p> 构造一个GET请求，但包含大量数据(超过65535字节)<\/li> 由于请求体长度溢出为0，代理服务器会认为没有请求体<\/li> 后端服务器可能以不同方式解析，导致请求拆分<\/li> <\/ul> <\/li> 攻击步骤<\/strong>：<\/p> 发送一个精心构造的GET请求，请求体恰好65536字节<\/li> 前65535字节会被正常处理<\/li> 第65536字节会被认为是下一个请求的开始<\/li> 可以隐藏恶意请求在溢出的数据中<\/li> <\/ul> <\/li> 实际利用<\/strong>：<\/p> \/\/ 示例攻击代码(概念验证) <\/span><\/span><\/span><\/span>func<\/span> sendMaliciousRequest<\/span>(conn<\/span> net<\/span>.Conn<\/span>) { <\/span><\/span> \/\/ 构造恶意请求 <\/span><\/span><\/span><\/span> method<\/span> :=<\/span> "GET"<\/span> <\/span><\/span> uri<\/span> :=<\/span> "\/"<\/span> <\/span><\/span> body<\/span> :=<\/span> make([]byte<\/span>, 65536<\/span>) \/\/ 刚好触发溢出 <\/span><\/span><\/span><\/span> \/\/ 填充body前部为正常数据 <\/span><\/span><\/span><\/span> copy(body<\/span>[:100<\/span>], []byte("normal data"<\/span>)) <\/span><\/span> \/\/ 在body后部隐藏恶意请求 <\/span><\/span><\/span><\/span> copy(body<\/span>[65400<\/span>:], []byte("POST \/admin HTTP\/1.1\r\n..."<\/span>)) <\/span><\/span> <\/span><\/span> \/\/ 序列化请求 <\/span><\/span><\/span><\/span> binary<\/span>.Write<\/span>(conn<\/span>, binary<\/span>.BigEndian<\/span>, uint16(len(method<\/span>))) <\/span><\/span> conn<\/span>.Write<\/span>([]byte(method<\/span>)) <\/span><\/span> binary<\/span>.Write<\/span>(conn<\/span>, binary<\/span>.BigEndian<\/span>, uint16(len(uri<\/span>))) <\/span><\/span> conn<\/span>.Write<\/span>([]byte(uri<\/span>)) <\/span><\/span> binary<\/span>.Write<\/span>(conn<\/span>, binary<\/span>.BigEndian<\/span>, uint16(0<\/span>)) \/\/ 0个header <\/span><\/span><\/span><\/span> binary<\/span>.Write<\/span>(conn<\/span>, binary<\/span>.BigEndian<\/span>, uint16(len(body<\/span>))) \/\/ 会溢出为0 <\/span><\/span><\/span><\/span> conn<\/span>.Write<\/span>(body<\/span>) <\/span><\/span>} <\/span><\/span><\/code><\/pre><\/li> <\/ol> 5. 防御措施<\/h2> 针对H111协议的漏洞，建议采取以下防御措施：<\/p> 长度字段升级<\/strong>：<\/p> 将uint16改为uint32或uint64，支持更大长度<\/li> 或使用可变长度编码(如varint)<\/li> <\/ul> <\/li> 添加溢出检查<\/strong>：<\/p> 在序列化和反序列化时检查长度是否超过最大值<\/li> 如果超过则返回错误，而不是静默截断<\/li> <\/ul> <\/li> 严格协议验证<\/strong>：<\/p> 验证方法是否合法(如只允许GET、POST等)<\/li> 验证URI格式是否合法<\/li> 验证请求头数量是否合理<\/li> <\/ul> <\/li> 连接处理改进<\/strong>：<\/p> 实现更严格的连接复用逻辑<\/li> 添加请求超时机制<\/li> 限制单个连接的最大请求数<\/li> <\/ul> <\/li> 输入净化<\/strong>：<\/p> 对所有输入数据进行严格验证和净化<\/li> 拒绝任何不符合协议规范的请求<\/li> <\/ul> <\/li> <\/ol> 6. 实现示例<\/h2> 以下是H111协议的安全实现示例：<\/p> \/\/ 安全读取长度字段 <\/span><\/span><\/span><\/span>func<\/span> readLength<\/span>(r<\/span> io<\/span>.Reader<\/span>) (int<\/span>, error<\/span>) { <\/span><\/span> var<\/span> length<\/span> uint16<\/span> <\/span><\/span> if<\/span> err<\/span> :=<\/span> binary<\/span>.Read<\/span>(r<\/span>, binary<\/span>.BigEndian<\/span>, &<\/span>length<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> 0<\/span>, err<\/span> <\/span><\/span> } <\/span><\/span> if<\/span> length<\/span> > MaxAllowedLength<\/span> { <\/span><\/span> return<\/span> 0<\/span>, fmt<\/span>.Errorf<\/span>("length %d exceeds maximum allowed %d"<\/span>, length<\/span>, MaxAllowedLength<\/span>) <\/span><\/span> } <\/span><\/span> return<\/span> int(length<\/span>), nil<\/span> <\/span><\/span>} <\/span><\/span> <\/span><\/span>\/\/ 安全写入长度字段 <\/span><\/span><\/span><\/span>func<\/span> writeLength<\/span>(w<\/span> io<\/span>.Writer<\/span>, length<\/span> int<\/span>) error<\/span> { <\/span><\/span> if<\/span> length<\/span> > MaxAllowedLength<\/span> { <\/span><\/span> return<\/span> fmt<\/span>.Errorf<\/span>("length %d exceeds maximum allowed %d"<\/span>, length<\/span>, MaxAllowedLength<\/span>) <\/span><\/span> } <\/span><\/span> return<\/span> binary<\/span>.Write<\/span>(w<\/span>, binary<\/span>.BigEndian<\/span>, uint16(length<\/span>)) <\/span><\/span>} <\/span><\/span> <\/span><\/span>\/\/ 安全读取H111请求 <\/span><\/span><\/span><\/span>func<\/span> ReadH111Request<\/span>(r<\/span> io<\/span>.Reader<\/span>) (*<\/span>H111Request<\/span>, error<\/span>) { <\/span><\/span> req<\/span> :=<\/span> &<\/span>H111Request<\/span>{} <\/span><\/span> <\/span><\/span> \/\/ 读取方法 <\/span><\/span><\/span><\/span> methodLen<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read method length: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> method<\/span> :=<\/span> make([]byte<\/span>, methodLen<\/span>) <\/span><\/span> if<\/span> _<\/span>, err<\/span> :=<\/span> io<\/span>.ReadFull<\/span>(r<\/span>, method<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read method: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> req<\/span>.Method<\/span> = string(method<\/span>) <\/span><\/span> <\/span><\/span> \/\/ 读取URI <\/span><\/span><\/span><\/span> uriLen<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read URI length: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> uri<\/span> :=<\/span> make([]byte<\/span>, uriLen<\/span>) <\/span><\/span> if<\/span> _<\/span>, err<\/span> :=<\/span> io<\/span>.ReadFull<\/span>(r<\/span>, uri<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read URI: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> req<\/span>.URI<\/span> = string(uri<\/span>) <\/span><\/span> <\/span><\/span> \/\/ 读取请求头 <\/span><\/span><\/span><\/span> headerCount<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read header count: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> if<\/span> headerCount<\/span> > MaxHeaderCount<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("header count %d exceeds maximum %d"<\/span>, headerCount<\/span>, MaxHeaderCount<\/span>) <\/span><\/span> } <\/span><\/span> <\/span><\/span> req<\/span>.Headers<\/span> = make(map<\/span>[string<\/span>]string<\/span>) <\/span><\/span> for<\/span> i<\/span> :=<\/span> 0<\/span>; i<\/span> < headerCount<\/span>; i<\/span>++<\/span> { <\/span><\/span> \/\/ 读取键 <\/span><\/span><\/span><\/span> keyLen<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read header key length: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> key<\/span> :=<\/span> make([]byte<\/span>, keyLen<\/span>) <\/span><\/span> if<\/span> _<\/span>, err<\/span> :=<\/span> io<\/span>.ReadFull<\/span>(r<\/span>, key<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read header key: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> <\/span><\/span> \/\/ 读取值 <\/span><\/span><\/span><\/span> valueLen<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read header value length: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> value<\/span> :=<\/span> make([]byte<\/span>, valueLen<\/span>) <\/span><\/span> if<\/span> _<\/span>, err<\/span> :=<\/span> io<\/span>.ReadFull<\/span>(r<\/span>, value<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read header value: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> <\/span><\/span> req<\/span>.Headers<\/span>[string(key<\/span>)] = string(value<\/span>) <\/span><\/span> } <\/span><\/span> <\/span><\/span> \/\/ 读取请求体 <\/span><\/span><\/span><\/span> bodyLen<\/span>, err<\/span> :=<\/span> readLength<\/span>(r<\/span>) <\/span><\/span> if<\/span> err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read body length: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> if<\/span> bodyLen<\/span> > 0<\/span> { <\/span><\/span> req<\/span>.Body<\/span> = make([]byte<\/span>, bodyLen<\/span>) <\/span><\/span> if<\/span> _<\/span>, err<\/span> :=<\/span> io<\/span>.ReadFull<\/span>(r<\/span>, req<\/span>.Body<\/span>); err<\/span> !=<\/span> nil<\/span> { <\/span><\/span> return<\/span> nil<\/span>, fmt<\/span>.Errorf<\/span>("read body: %w"<\/span>, err<\/span>) <\/span><\/span> } <\/span><\/span> } <\/span><\/span> <\/span><\/span> return<\/span> req<\/span>, nil<\/span> <\/span><\/span>} <\/span><\/span><\/code><\/pre>7. 总结<\/h2> H111协议作为一种自定义的HTTP协议变体，在追求性能的同时牺牲了安全性。其主要的漏洞源于：<\/p> 使用固定长度的uint16表示所有长度字段<\/li> 缺乏必要的边界检查和溢出处理<\/li> 对异常情况的处理不够健壮<\/li> <\/ol> 通过分析这个案例，我们可以得到以下安全开发经验：<\/p> 在设计二进制协议时，长度字段的选择要谨慎<\/li> 必须对所有输入进行严格的验证和边界检查<\/li> 整数溢出是常见的安全问题，需要特别注意<\/li> 协议设计要考虑各种异常情况和边界条件<\/li> 性能优化不能以牺牲安全性为代价<\/li> <\/ol> 对于使用类似协议的开发者，建议立即检查并修复相关实现，添加必要的安全措施，防止请求走私等攻击的发生。<\/p>

自定义H111协议分析与请求走私漏洞研究<\/h1>

2. H111协议格式详解<\/h2>

3. 协议实现细节<\/h2>

4. 安全漏洞分析<\/h2>