MCP-Finder: 基于AI的智能目录扫描与敏感信息收集工具开发指南<\/h1>

1. 工具概述<\/h2>
MCP-Finder是一款创新的网络安全扫描工具，巧妙地将传统目录扫描技术与现代AI大模型分析能力相结合，通过整合MCP协议中的streamable-http方案，为安全研究人员提供了更智能、更高效的漏洞发现解决方案。<\/p>

核心特点<\/h3>

双引擎设计<\/strong>：传统扫描+AI分析<\/li>
流式传输<\/strong>：基于MCP协议的streamable-http方案<\/li>
敏感数据收集<\/strong>：特定状态码(如500)页面信息泄露收集<\/li>
AI增强分析<\/strong>：理解上下文关系，发现潜在安全问题<\/li>

自动化报告<\/strong>：结构化漏洞报告，包含修复建议<\/li> <\/ul>
2. 环境搭建<\/h2>
2.1 项目初始化<\/h3>
uv init MCP-Finder <\/span><\/span>cd MCP-Finder <\/span><\/span>uv venv <\/span><\/span><\/code><\/pre>2.2 依赖安装<\/h3> # 通过脚本安装最新版pip<\/span> <\/span><\/span>curl https:\/\/bootstrap.pypa.io\/get-pip.py -o get-pip.py <\/span><\/span>python get-pip.py <\/span><\/span> <\/span><\/span># 激活虚拟环境<\/span> <\/span><\/span>source .venv\/bin\/activate <\/span><\/span> <\/span><\/span># 安装MCP和dirsearch所需依赖<\/span> <\/span><\/span>uv add "mcp[cli]"<\/span> <\/span><\/span>uv add requests <\/span><\/span>pip install setuptools <\/span><\/span> <\/span><\/span># 退出虚拟环境<\/span> <\/span><\/span>deactivate <\/span><\/span> <\/span><\/span># 安装dirsearch依赖<\/span> <\/span><\/span>cd dirsearch <\/span><\/span>python -m pip3 install -r requirements.txt <\/span><\/span><\/code><\/pre>3. 项目架构<\/h2> MCP-Finder\/ ├── main.py # 主程序入口 ├── results\/ # 存放扫描结果 │ └── Finder_*.txt # 按时间戳命名的JSON结果文件 └── dirsearch-master\/ # dirsearch源码目录 └── dirsearch.py # dirsearch主程序 <\/code><\/pre> 4. 核心功能实现<\/h2> 4.1 MCP服务初始化<\/h3> from<\/span> mcp.server.fastmcp import<\/span> FastMCP <\/span><\/span> <\/span><\/span># 基本配置<\/span> <\/span><\/span>MCP =<\/span> FastMCP("MCP-Finder"<\/span>, port=<\/span>"8001"<\/span>) <\/span><\/span> <\/span><\/span># 高级配置<\/span> <\/span><\/span>MCP =<\/span> FastMCP( <\/span><\/span> name=<\/span>"MCP-Finder"<\/span>, <\/span><\/span> port=<\/span>8001<\/span>, <\/span><\/span> host=<\/span>"0.0.0.0"<\/span>, # 允许外部访问<\/span> <\/span><\/span> debug=<\/span>True<\/span> # 开启调试模式<\/span> <\/span><\/span>) <\/span><\/span><\/code><\/pre>4.2 Streamable HTTP传输<\/h3> if<\/span> __name__ ==<\/span> "__main__"<\/span>: <\/span><\/span> MCP.<\/span>run("streamable-http"<\/span>) <\/span><\/span><\/code><\/pre>流式传输特性<\/strong>：<\/p> 需要维持持久连接进行分块数据传输<\/li> 服务器可能间歇性推送数据<\/li> 单个请求的完整响应周期被拉长<\/li> 短超时会中断传输<\/li> <\/ul> 4.3 工具函数注册<\/h3> 同步工具示例<\/h4> @mcp<\/span>.<\/span>tool <\/span><\/span>def<\/span> calculate_distance<\/span>(lat1: float, lon1: float, lat2: float, lon2: float) -><\/span> float: <\/span><\/span> """Calculate the distance between two coordinates."""<\/span> <\/span><\/span> return<\/span> 42.5<\/span> <\/span><\/span><\/code><\/pre>异步工具示例<\/h4> @mcp<\/span>.<\/span>tool <\/span><\/span>async<\/span> def<\/span> fetch_weather<\/span>(city: str) -><\/span> dict: <\/span><\/span> """Retrieve current weather conditions for a city."""<\/span> <\/span><\/span> async<\/span> with<\/span> aiohttp.<\/span>ClientSession() as<\/span> session: <\/span><\/span> async<\/span> with<\/span> session.<\/span>get(f<\/span>"https:\/\/api.example.com\/weather\/<\/span>{<\/span>city}<\/span>"<\/span>) as<\/span> response: <\/span><\/span> response.<\/span>raise_for_status() <\/span><\/span> return<\/span> await<\/span> response.<\/span>json() <\/span><\/span><\/code><\/pre>4.4 核心扫描功能<\/h3> 扫描工具函数<\/h4> @MCP<\/span>.<\/span>tool() <\/span><\/span>async<\/span> def<\/span> scan<\/span>(url: str) -><\/span> dict: <\/span><\/span> """ <\/span><\/span><\/span> 执行网站目录扫描，返回结构化扫描结果 <\/span><\/span><\/span> Args: <\/span><\/span><\/span> url (str): 目标网站URL，需包含协议头(如http\/https) <\/span><\/span><\/span> Returns: <\/span><\/span><\/span> dict: 包含以下键的字典: <\/span><\/span><\/span> - status_200: 200状态的有效路径列表 <\/span><\/span><\/span> - status_500: 500状态的有效结果列表 <\/span><\/span><\/span> - status_other: 非200和500状态码的有效结果列表 <\/span><\/span><\/span> - stat_counts: 各类状态码统计 <\/span><\/span><\/span> - result_path: 结果文件路径 <\/span><\/span><\/span> """<\/span> <\/span><\/span> # 生成时间戳文件名<\/span> <\/span><\/span> timestamp =<\/span> datetime.<\/span>now().<\/span>strftime("%Y%m<\/span>%d<\/span>_%H%M%S"<\/span>) <\/span><\/span> filename =<\/span> f<\/span>"Finder_<\/span>{<\/span>timestamp}<\/span>.txt"<\/span> <\/span><\/span> outpath =<\/span> f<\/span>".\/results\/<\/span>{<\/span>filename}<\/span>"<\/span> <\/span><\/span> <\/span><\/span> # 定义dirsearch命令<\/span> <\/span><\/span> command =<\/span> [ <\/span><\/span> "python3"<\/span>, ".\/dirsearch-master\/dirsearch.py"<\/span>, <\/span><\/span> "-u"<\/span>, url, <\/span><\/span> "-o"<\/span>, outpath, <\/span><\/span> "--output-formats=json"<\/span>, <\/span><\/span> ] <\/span><\/span> <\/span><\/span> # 异步执行扫描<\/span> <\/span><\/span> try<\/span>: <\/span><\/span> process =<\/span> await<\/span> asyncio.<\/span>create_subprocess_exec( <\/span><\/span> *<\/span>command, <\/span><\/span> stdout=<\/span>asyncio.<\/span>subprocess.<\/span>PIPE, <\/span><\/span> stderr=<\/span>asyncio.<\/span>subprocess.<\/span>PIPE, <\/span><\/span> ) <\/span><\/span> stdout, stderr =<\/span> await<\/span> asyncio.<\/span>wait_for(process.<\/span>communicate(), timeout=<\/span>300.0<\/span>) <\/span><\/span> <\/span><\/span> if<\/span> process.<\/span>returncode !=<\/span> 0<\/span>: <\/span><\/span> raise<\/span> RuntimeError<\/span>(f<\/span>"Command failed with error: <\/span>{<\/span>stderr.<\/span>decode()}<\/span>"<\/span>) <\/span><\/span> <\/span><\/span> print(f<\/span>"Scan completed, results saved to <\/span>{<\/span>outpath}<\/span>"<\/span>) <\/span><\/span> return<\/span> process_results(outpath) <\/span><\/span> <\/span><\/span> except<\/span> Exception<\/span> as<\/span> e: <\/span><\/span> print(f<\/span>"An error occurred: <\/span>{<\/span>e}<\/span>"<\/span>) <\/span><\/span> raise<\/span> <\/span><\/span><\/code><\/pre>结果处理函数<\/h4> def<\/span> process_results<\/span>(file_path: str) -><\/span> dict: <\/span><\/span> """ <\/span><\/span><\/span> 处理dirsearch扫描生成的JSON结果文件 <\/span><\/span><\/span> Args: <\/span><\/span><\/span> file_path (str): JSON格式的扫描结果文件路径 <\/span><\/span><\/span> Returns: <\/span><\/span><\/span> dict: 分类后的扫描结果 <\/span><\/span><\/span> """<\/span> <\/span><\/span> status_200 =<\/span> [] <\/span><\/span> stats_other =<\/span> [] <\/span><\/span> stats_500 =<\/span> [] <\/span><\/span> stats =<\/span> {} <\/span><\/span> <\/span><\/span> try<\/span>: <\/span><\/span> with<\/span> open(file_path, 'r'<\/span>, encoding=<\/span>'utf-8'<\/span>) as<\/span> f: <\/span><\/span> data =<\/span> json.<\/span>load(f) <\/span><\/span> results =<\/span> data.<\/span>get('results'<\/span>, []) <\/span><\/span> <\/span><\/span> for<\/span> result in<\/span> results: <\/span><\/span> url =<\/span> result.<\/span>get('url'<\/span>) <\/span><\/span> status_code =<\/span> result.<\/span>get('status'<\/span>) <\/span><\/span> content_length =<\/span> result.<\/span>get('contentLength'<\/span>) <\/span><\/span> <\/span><\/span> # 状态码统计<\/span> <\/span><\/span> stats[status_code] =<\/span> stats.<\/span>get(status_code, 0<\/span>) +<\/span> 1<\/span> <\/span><\/span> <\/span><\/span> # 分类处理<\/span> <\/span><\/span> if<\/span> status_code ==<\/span> 404<\/span>: <\/span><\/span> continue<\/span> <\/span><\/span> elif<\/span> status_code ==<\/span> 200<\/span>: <\/span><\/span> status_200.<\/span>append({ <\/span><\/span> "url"<\/span>: url, <\/span><\/span> "status"<\/span>: status_code, <\/span><\/span> "length"<\/span>: content_length <\/span><\/span> }) <\/span><\/span> elif<\/span> status_code ==<\/span> 500<\/span>: <\/span><\/span> stats_500.<\/span>append({ <\/span><\/span> "url"<\/span>: url, <\/span><\/span> "status"<\/span>: status_code, <\/span><\/span> "length"<\/span>: content_length <\/span><\/span> }) <\/span><\/span> else<\/span>: <\/span><\/span> stats_other.<\/span>append({ <\/span><\/span> "url"<\/span>: url, <\/span><\/span> "status"<\/span>: status_code, <\/span><\/span> "length"<\/span>: content_length <\/span><\/span> }) <\/span><\/span> <\/span><\/span> except<\/span> Exception<\/span> as<\/span> e: <\/span><\/span> print(f<\/span>"Error processing file: <\/span>{<\/span>e}<\/span>"<\/span>) <\/span><\/span> <\/span><\/span> return<\/span> { <\/span><\/span> "status_200"<\/span>: status_200, <\/span><\/span> "status_500"<\/span>: stats_500, <\/span><\/span> "status_other"<\/span>: stats_other, <\/span><\/span> "stat_counts"<\/span>: stats, <\/span><\/span> "result_path"<\/span>: file_path <\/span><\/span> } <\/span><\/span><\/code><\/pre>4.5 敏感信息收集<\/h3> 500状态页面信息收集<\/h4> @MCP<\/span>.<\/span>tool() <\/span><\/span>async<\/span> def<\/span> get_500_message<\/span>(url_path: str) -><\/span> str: <\/span><\/span> """ <\/span><\/span><\/span> 对500状态码页面提取错误信息 <\/span><\/span><\/span> Args: <\/span><\/span><\/span> url_path (str): 目标URL <\/span><\/span><\/span> Returns: <\/span><\/span><\/span> str: 错误信息或错误字典 <\/span><\/span><\/span> """<\/span> <\/span><\/span> async<\/span> with<\/span> httpx.<\/span>AsyncClient() as<\/span> client: <\/span><\/span> try<\/span>: <\/span><\/span> response =<\/span> await<\/span> client.<\/span>get(url_path, timeout=<\/span>5.0<\/span>) <\/span><\/span> response.<\/span>raise_for_status() <\/span><\/span> return<\/span> response.<\/span>text <\/span><\/span> except<\/span> httpx.<\/span>HTTPStatusError as<\/span> e: <\/span><\/span> return<\/span> {"error"<\/span>: f<\/span>"请求失败: <\/span>{<\/span>e}<\/span>"<\/span>} <\/span><\/span> except<\/span> Exception<\/span> as<\/span> e: <\/span><\/span> return<\/span> {"error"<\/span>: f<\/span>"请求失败: <\/span>{<\/span>str(e)}<\/span>"<\/span>} <\/span><\/span><\/code><\/pre>敏感信息检测<\/h4> # 敏感信息检测模式<\/span> <\/span><\/span>SENSITIVE_PATTERNS =<\/span> [ <\/span><\/span> r<\/span>"(?i)password\s*[=:]\s*['<\/span>\"<\/span>]?\w+"<\/span>, <\/span><\/span> r<\/span>"(?i)api[_-]?key\s*[=:]\s*['<\/span>\"<\/span>]?\w+"<\/span>, <\/span><\/span> r<\/span>"-----BEGIN (RSA|OPENSSH) PRIVATE KEY-----"<\/span> <\/span><\/span>] <\/span><\/span> <\/span><\/span>@MCP<\/span>.<\/span>tool() <\/span><\/span>async<\/span> def<\/span> sensitive_seek<\/span>(url_path: str) -><\/span> dict: <\/span><\/span> """ <\/span><\/span><\/span> 对URL路径进行敏感信息检测 <\/span><\/span><\/span> Args: <\/span><\/span><\/span> url_path (str): 目标URL路径 <\/span><\/span><\/span> Returns: <\/span><\/span><\/span> dict: 包含敏感信息检测结果 <\/span><\/span><\/span> """<\/span> <\/span><\/span> async<\/span> with<\/span> httpx.<\/span>AsyncClient() as<\/span> client: <\/span><\/span> try<\/span>: <\/span><\/span> sensitive_msg =<\/span> [] <\/span><\/span> response =<\/span> await<\/span> client.<\/span>get(url_path, timeout=<\/span>5.0<\/span>) <\/span><\/span> response.<\/span>raise_for_status() <\/span><\/span> content =<\/span> response.<\/span>text <\/span><\/span> <\/span><\/span> for<\/span> pattern in<\/span> SENSITIVE_PATTERNS: <\/span><\/span> if<\/span> re.<\/span>search(pattern, content): <\/span><\/span> msg =<\/span> re.<\/span>findall(pattern, content) <\/span><\/span> sensitive_msg.<\/span>append(msg) <\/span><\/span> <\/span><\/span> return<\/span> { <\/span><\/span> "url"<\/span>: url_path, <\/span><\/span> "sensitive_msg"<\/span>: sensitive_msg <\/span><\/span> } <\/span><\/span> <\/span><\/span> except<\/span> Exception<\/span> as<\/span> e: <\/span><\/span> return<\/span> {"error"<\/span>: f<\/span>"请求失败: <\/span>{<\/span>str(e)}<\/span>"<\/span>} <\/span><\/span><\/code><\/pre>5. 高级功能<\/h2> 5.1 深度递归扫描<\/h3> command =<\/span> [ <\/span><\/span> ...<\/span> <\/span><\/span> "--recursive"<\/span>, # 启用递归扫描<\/span> <\/span><\/span> "--max-recursion-depth"<\/span>, "3"<\/span> # 控制递归深度<\/span> <\/span><\/span>] <\/span><\/span><\/code><\/pre>5.2 提示词优化<\/h3> 请使用已有的MCP工具对网站https:\/\/xxx.xxx.com\/进行全面扫描，具体要求如下: <\/span><\/span> <\/span><\/span>1.<\/span> 目录与页面扫描自动遍历网站常见目录与页面，重点关注可能存在版本目录、备份目录、敏感文件等信息泄露风险。 <\/span><\/span> <\/span><\/span>2.<\/span> 500响应处理 <\/span><\/span>对所有返回500状态码的页面，自动调用`get_500_message`工具函数获取详细错误内容，并作出信息泄露和利用分析。 <\/span><\/span> <\/span><\/span>3.<\/span> 输出结构化报告 <\/span><\/span>以Markdown表格形式输出以下字段: <\/span><\/span>-<\/span> 目录\/页面路径 <\/span><\/span>-<\/span> HTTP状态码 <\/span><\/span>-<\/span> 危害描述 <\/span><\/span>-<\/span> 利用方法 <\/span><\/span>-<\/span> 修复建议 <\/span><\/span><\/code><\/pre>6. 实战应用<\/h2> 6.1 启动服务<\/h3> python main.py <\/span><\/span><\/code><\/pre>6.2 扫描报告示例<\/h3> 工具可以生成美观易读的扫描报告，包含:<\/p> 漏洞类型<\/li> 风险等级<\/li> 受影响URL<\/li> 修复建议<\/li> 危害描述<\/li> 利用方法<\/li> <\/ul> 7. 未来计划<\/h2> 增加更多漏洞类型的检测能力<\/li> 实现完整的漏洞扫描自动化流程<\/li> 优化AI模型的分析准确性<\/li> 扩展对更多Web框架的支持<\/li> 增强分布式扫描能力<\/li> <\/ol> 8. 最佳实践<\/h2> 异常处理<\/strong>：对扫描、文件读写、网络请求等环节增加细致的异常捕获与日志输出<\/li> 异步优化<\/strong>：充分利用asyncio提升大规模扫描性能<\/li> 模块化设计<\/strong>：将功能模块化，便于维护和扩展<\/li> 数据存储<\/strong>：使用时间戳命名结果文件，便于查找和管理<\/li> Token优化<\/strong>：通过文件中转减少直接输入LLM的数据量<\/li> <\/ol> 9. 注意事项<\/h2> 流式传输需要维持持久连接，避免设置过短的超时时间<\/li> 大规模扫描时注意资源占用和性能优化<\/li> AI分析结果需要与原始扫描数据对比验证<\/li> 敏感信息检测规则需要定期更新维护<\/li> 递归扫描深度需要合理设置以避免过度请求<\/li> <\/ol>

MCP-Finder: 基于AI的智能目录扫描与敏感信息收集工具开发指南<\/h1>

1. 工具概述<\/h2> MCP-Finder是一款创新的网络安全扫描工具，巧妙地将传统目录扫描技术与现代AI大模型分析能力相结合，通过整合MCP协议中的streamable-http方案，为安全研究人员提供了更智能、更高效的漏洞发现解决方案。<\/p>

2. 环境搭建<\/h2>

4. 核心功能实现<\/h2>

4.3 工具函数注册<\/h3>

4.4 核心扫描功能<\/h3>

4.5 敏感信息收集<\/h3>

5. 高级功能<\/h2>

6. 实战应用<\/h2>

1. 工具概述<\/h2>
MCP-Finder是一款创新的网络安全扫描工具，巧妙地将传统目录扫描技术与现代AI大模型分析能力相结合，通过整合MCP协议中的streamable-http方案，为安全研究人员提供了更智能、更高效的漏洞发现解决方案。<\/p>