GitHub代码泄露监控工具开发指南<\/h1>

背景与原理<\/h2>

GitHub作为开源代码托管平台，经常存在敏感信息泄露风险，包括：<\/p>

用户名、密码、数据库连接信息<\/li>
内网IP地址<\/li>

开发者个人信息（身高、体重、年龄等）<\/li> <\/ul>

本工具通过自动化监控GitHub代码仓库，及时发现包含敏感信息的代码提交。<\/p>

开发环境与依赖<\/h2>

系统环境<\/strong>：<\/p>

MacOS 10.12.6（也可在其他支持Python的系统运行）<\/li>
Python 3.6.5+<\/li> <\/ul>
所需Python库<\/strong>：<\/p>

requests<\/code>：HTTP请求<\/li>
lxml<\/code>：HTML\/XML解析<\/li>
csv<\/code>：CSV文件操作<\/li>
tqdm<\/code>：进度条显示<\/li>
email<\/code>\/smtplib<\/code>：邮件发送<\/li>
configparser<\/code>：配置文件解析<\/li>
time<\/code>：时间控制<\/li> <\/ul> 核心功能实现<\/h2> 1. GitHub登录机制<\/h3> GitHub登录流程：<\/p> 访问https:\/\/github.com\/login<\/code>获取登录页面<\/li> 提取authenticity_token<\/code>值<\/li> 向https:\/\/github.com\/session<\/code>提交POST请求<\/li> <\/ol> 关键代码：<\/p> def<\/span> login_github<\/span>(username, password): <\/span><\/span> login_url =<\/span> 'https:\/\/github.com\/login'<\/span> <\/span><\/span> session_url =<\/span> 'https:\/\/github.com\/session'<\/span> <\/span><\/span> try<\/span>: <\/span><\/span> s =<\/span> requests.<\/span>session() <\/span><\/span> resp =<\/span> s.<\/span>get(login_url).<\/span>text <\/span><\/span> dom_tree =<\/span> etree.<\/span>HTML(resp) <\/span><\/span> key =<\/span> dom_tree.<\/span>xpath('\/\/input[@name="authenticity_token"]\/@value'<\/span>) <\/span><\/span> user_data =<\/span> { <\/span><\/span> 'commit'<\/span>: 'Sign in'<\/span>, <\/span><\/span> 'utf8'<\/span>: '✓'<\/span>, <\/span><\/span> 'authenticity_token'<\/span>: key, <\/span><\/span> 'login'<\/span>: username, <\/span><\/span> 'password'<\/span>: password <\/span><\/span> } <\/span><\/span> s.<\/span>post(session_url, data=<\/span>user_data) <\/span><\/span> s.<\/span>get('https:\/\/github.com\/settings\/profile'<\/span>) # 验证登录<\/span> <\/span><\/span> return<\/span> s <\/span><\/span> except<\/span>: <\/span><\/span> print('产生异常，请检查网络设置及用户名和密码'<\/span>) <\/span><\/span><\/code><\/pre>2. 代码搜索与解析<\/h3> 搜索流程：<\/p> 构造搜索URL：https:\/\/github.com\/search?p=[页码]&q=[关键词]&type=Code<\/code><\/li> 使用XPath解析返回的HTML页面<\/li> 提取关键信息：仓库URL<\/li> 用户名<\/li> 上传时间<\/li> 文件名<\/li> <\/ul> <\/li> <\/ol> 关键代码：<\/p> def<\/span> hunter<\/span>(gUser, gPass, keyword, payloads): <\/span><\/span> sensitive_list =<\/span> [] <\/span><\/span> tUrls =<\/span> [] <\/span><\/span> try<\/span>: <\/span><\/span> with<\/span> open('leak.csv'<\/span>, 'w'<\/span>, encoding=<\/span>'utf-8'<\/span>, newline=<\/span>''<\/span>) as<\/span> csv_file: <\/span><\/span> writer =<\/span> csv.<\/span>writer(csv_file) <\/span><\/span> writer.<\/span>writerow(['URL'<\/span>, 'Username'<\/span>, 'Upload Time'<\/span>, 'Filename'<\/span>]) <\/span><\/span> <\/span><\/span> s =<\/span> login_github(gUser, gPass) <\/span><\/span> for<\/span> page in<\/span> tqdm(range(1<\/span>, 6<\/span>)): # 搜索1-5页<\/span> <\/span><\/span> search_code =<\/span> f<\/span>'https:\/\/github.com\/search?p=<\/span>{<\/span>page}<\/span>&q=<\/span>{<\/span>keyword}<\/span>&type=Code'<\/span> <\/span><\/span> resp =<\/span> s.<\/span>get(search_code) <\/span><\/span> dom_tree_code =<\/span> etree.<\/span>HTML(resp.<\/span>text) <\/span><\/span> <\/span><\/span> # XPath提取信息<\/span> <\/span><\/span> Urls =<\/span> dom_tree_code.<\/span>xpath('\/\/div[@class="d-inline-block col-10"]\/a[2]\/@href'<\/span>) <\/span><\/span> users =<\/span> dom_tree_code.<\/span>xpath('\/\/a[@class="text-blod"]\/text()'<\/span>) <\/span><\/span> datetime =<\/span> dom_tree_code.<\/span>xpath('\/\/relative-time\/text()'<\/span>) <\/span><\/span> filename =<\/span> dom_tree_code.<\/span>xpath('\/\/div[@class="d-inline-block col-10"]\/a[2]\/text()'<\/span>) <\/span><\/span> <\/span><\/span> for<\/span> i in<\/span> range(len(Urls)): <\/span><\/span> full_url =<\/span> 'https:\/\/github.com'<\/span> +<\/span> Urls[i] <\/span><\/span> tUrls.<\/span>append(full_url) <\/span><\/span> writer.<\/span>writerow([full_url, users[i], datetime[i], filename[i]]) <\/span><\/span> <\/span><\/span> # 检查原始代码中的敏感信息<\/span> <\/span><\/span> raw_url =<\/span> 'https:\/\/raw.githubusercontent.com'<\/span> +<\/span> Urls[i].<\/span>replace('\/blob'<\/span>, ''<\/span>) <\/span><\/span> code =<\/span> requests.<\/span>get(raw_url).<\/span>text <\/span><\/span> for<\/span> payload in<\/span> payloads: <\/span><\/span> if<\/span> payload in<\/span> code: <\/span><\/span> leak_info =<\/span> f<\/span>"命中的Payload为: <\/span>{<\/span>payload}<\/span>\n<\/span>{<\/span>full_url}<\/span>\n\n<\/span>代码如下:<\/span>\n<\/span>{<\/span>code}<\/span>\n\n<\/span>"<\/span> <\/span><\/span> sensitive_list.<\/span>append(leak_info) <\/span><\/span> return<\/span> sensitive_list <\/span><\/span> except<\/span> Exception<\/span> as<\/span> e: <\/span><\/span> print(e) <\/span><\/span><\/code><\/pre>3. 邮件预警系统<\/h3> 邮件功能特点：<\/p> 支持多个收件人<\/li> 可发送HTML格式邮件<\/li> 附带CSV格式的搜索结果附件<\/li> <\/ul> 关键代码：<\/p> def<\/span> send_warning<\/span>(host, username, password, sender, receivers, content): <\/span><\/span> def<\/span> _format_addr<\/span>(s): <\/span><\/span> name, addr =<\/span> parseaddr(s) <\/span><\/span> return<\/span> formataddr((Header(name, 'utf-8'<\/span>).<\/span>encode(), addr)) <\/span><\/span> <\/span><\/span> msg =<\/span> MIMEMultipart() <\/span><\/span> msg['From'<\/span>] =<\/span> _format_addr(f<\/span>'Github安全监控<<\/span>{<\/span>sender}<\/span>>'<\/span>) <\/span><\/span> msg['To'<\/span>] =<\/span> ', '<\/span>.<\/span>join(receivers) <\/span><\/span> msg['Subject'<\/span>] =<\/span> Header('Github敏感信息泄露通知'<\/span>, 'utf-8'<\/span>).<\/span>encode() <\/span><\/span> <\/span><\/span> # 邮件正文<\/span> <\/span><\/span> msg.<\/span>attach(MIMEText(f<\/span>''' <\/span><\/span><\/span> Dear all <\/span><\/span><\/span> <\/span><\/span><\/span> 请注意，怀疑Github上已经上传敏感信息！以下是可能存在敏感信息的仓库！ <\/span><\/span><\/span> <\/span><\/span><\/span> <\/span>{<\/span>content}<\/span> <\/span><\/span><\/span> '''<\/span>, 'plain'<\/span>, 'utf-8'<\/span>)) <\/span><\/span> <\/span><\/span> # 添加附件<\/span> <\/span><\/span> with<\/span> open('leak.csv'<\/span>, 'rb'<\/span>) as<\/span> f: <\/span><\/span> m =<\/span> MIMEBase('excel'<\/span>, 'csv'<\/span>, filename=<\/span>'leak.csv'<\/span>) <\/span><\/span> m.<\/span>add_header('Content-Disposition'<\/span>, 'attachment'<\/span>, filename=<\/span>'leak.csv'<\/span>) <\/span><\/span> m.<\/span>add_header('Content-ID'<\/span>, '<0>'<\/span>) <\/span><\/span> m.<\/span>add_header('X-Attachment-ID'<\/span>, '0'<\/span>) <\/span><\/span> m.<\/span>set_payload(f.<\/span>read()) <\/span><\/span> encoders.<\/span>encode_base64(m) <\/span><\/span> msg.<\/span>attach(m) <\/span><\/span> <\/span><\/span> try<\/span>: <\/span><\/span> server =<\/span> smtplib.<\/span>SMTP(host, 25<\/span>) <\/span><\/span> server.<\/span>login(username, password) <\/span><\/span> server.<\/span>sendmail(sender, receivers, msg.<\/span>as_string()) <\/span><\/span> print('邮件发送成功！'<\/span>) <\/span><\/span> except<\/span> Exception<\/span> as<\/span> err: <\/span><\/span> print(err) <\/span><\/span> finally<\/span>: <\/span><\/span> server.<\/span>quit() <\/span><\/span><\/code><\/pre>4. 配置文件管理<\/h3> 配置文件格式(info.ini<\/code>)：<\/p> [KEYWORD]<\/span> <\/span><\/span>keyword<\/span> =<\/span> your main keyword here<\/span> <\/span><\/span> <\/span><\/span>[EMAIL]<\/span> <\/span><\/span>host<\/span> =<\/span> Email server<\/span> <\/span><\/span>user<\/span> =<\/span> Email User<\/span> <\/span><\/span>password<\/span> =<\/span> Email password<\/span> <\/span><\/span> <\/span><\/span>[SENDER]<\/span> <\/span><\/span>sender<\/span> =<\/span> The email sender<\/span> <\/span><\/span> <\/span><\/span>[RECEIVER]<\/span> <\/span><\/span>receiver1<\/span> =<\/span> Email receiver No.1<\/span> <\/span><\/span>receiver2<\/span> =<\/span> Email receiver No.2<\/span> <\/span><\/span> <\/span><\/span>[Github]<\/span> <\/span><\/span>user<\/span> =<\/span> Github Username<\/span> <\/span><\/span>password<\/span> =<\/span> Github Password<\/span> <\/span><\/span> <\/span><\/span>[PAYLOADS]<\/span> <\/span><\/span>p1<\/span> =<\/span> Payload 1<\/span> <\/span><\/span>p2<\/span> =<\/span> Payload 2<\/span> <\/span><\/span>p3<\/span> =<\/span> Payload 3<\/span> <\/span><\/span>p4<\/span> =<\/span> Payload 4<\/span> <\/span><\/span>p5<\/span> =<\/span> Payload 5<\/span> <\/span><\/span>p6<\/span> =<\/span> Payload 6<\/span> <\/span><\/span><\/code><\/pre>配置文件读取代码：<\/p> config =<\/span> configparser.<\/span>ConfigParser() <\/span><\/span>config.<\/span>read('info.ini'<\/span>) <\/span><\/span> <\/span><\/span># 读取GitHub凭据<\/span> <\/span><\/span>g_User =<\/span> config['Github'<\/span>]['user'<\/span>] <\/span><\/span>g_Pass =<\/span> config['Github'<\/span>]['password'<\/span>] <\/span><\/span> <\/span><\/span># 读取邮件配置<\/span> <\/span><\/span>host =<\/span> config['EMAIL'<\/span>]['host'<\/span>] <\/span><\/span>m_User =<\/span> config['EMAIL'<\/span>]['user'<\/span>] <\/span><\/span>m_Pass =<\/span> config['EMAIL'<\/span>]['password'<\/span>] <\/span><\/span>m_sender =<\/span> config['SENDER'<\/span>]['sender'<\/span>] <\/span><\/span> <\/span><\/span># 读取收件人列表<\/span> <\/span><\/span>receivers =<\/span> [config['RECEIVER'<\/span>][k] for<\/span> k in<\/span> config['RECEIVER'<\/span>]] <\/span><\/span> <\/span><\/span># 读取搜索关键词和payload<\/span> <\/span><\/span>keyword =<\/span> config['KEYWORD'<\/span>]['keyword'<\/span>] <\/span><\/span>payloads =<\/span> [config['PAYLOADS'<\/span>][key] for<\/span> key in<\/span> config['PAYLOADS'<\/span>]] <\/span><\/span><\/code><\/pre>使用建议<\/h2> 搜索策略<\/strong>：<\/p> 主关键词：公司域名、邮箱后缀、员工姓名等<\/li> 辅助payload：password、username、database等敏感关键词<\/li> <\/ul> <\/li> 运行频率<\/strong>：<\/p> 建议每天运行2次<\/li> 避免频繁请求触发GitHub反爬机制<\/li> <\/ul> <\/li> 部署方式<\/strong>：<\/p> 使用Linux的crontab定时任务<\/li> 示例crontab配置（每天运行两次）： 0 9,21 * * * \/usr\/bin\/python3 \/path\/to\/github_monitor.py <\/code><\/pre> <\/li> <\/ul> <\/li> <\/ol> 完整代码获取<\/h2> 项目已开源在GitHub： https:\/\/github.com\/Hell0W0rld0\/Github-Hunter<\/a><\/p> 注意事项<\/h2> GitHub可能会更新页面结构，需要定期维护XPath表达式<\/li> 工具仅用于企业安全自查，请勿用于非法用途<\/li> 建议使用专用监控账号，避免使用个人GitHub账号<\/li> 搜索结果可能包含误报，需要人工确认<\/li> <\/ol> 通过本工具，企业可以及时发现并处理GitHub上的敏感信息泄露，降低安全风险。<\/p>