.DS_Store文件泄漏利用工具: ds_store_exp

.DS_Store是Mac下Finder用来保存如何展示 文件/文件夹 的数据文件,每个文件夹下对应一个。

如果开发/设计人员将.DS_Store上传部署到线上环境,可能造成文件目录结构泄漏,特别是备份文件、源代码文件。

ds_store_exp 是一个.DS_Store 文件泄漏利用脚本,它解析.DS_Store文件并递归地下载文件到本地: https://github.com/lijiejie/ds_store_exp

DS_Store parser is based on ds_store 1.1.0 。

一个示例

python ds_store_exp.py http://hd.zj.qq.com/themes/galaxyw/.DS_Store

hd.zj.qq.com/
└── themes
    └── galaxyw
        ├── app
        │   └── css
        │       └── style.min.css
        ├── cityData.min.js
        ├── images
        │   └── img
        │       ├── bg-hd.png
        │       ├── bg-item-activity.png
        │       ├── bg-masker-pop.png
        │       ├── btn-bm.png
        │       ├── btn-login-qq.png
        │       ├── btn-login-wx.png
        │       ├── ico-add-pic.png
        │       ├── ico-address.png
        │       ├── ico-bm.png
        │       ├── ico-duration-time.png
        │       ├── ico-pop-close.png
        │       ├── ico-right-top-delete.png
        │       ├── page-login-hd.png
        │       ├── pic-masker.png
        │       └── ticket-selected.png
        └── member
            ├── assets
            │   ├── css
            │   │   ├── ace-reset.css
            │   │   └── antd.css
            │   └── lib
            │       ├── cityData.min.js
            │       └── ueditor
            │           ├── index.html
            │           ├── lang
            │           │   └── zh-cn
            │           │       ├── images
            │           │       │   ├── copy.png
            │           │       │   ├── localimage.png
            │           │       │   ├── music.png
            │           │       │   └── upload.png
            │           │       └── zh-cn.js
            │           ├── php
            │           │   ├── action_crawler.php
            │           │   ├── action_list.php
            │           │   ├── action_upload.php
            │           │   ├── config.json
            │           │   ├── controller.php
            │           │   └── Uploader.class.php
            │           ├── ueditor.all.js
            │           ├── ueditor.all.min.js
            │           ├── ueditor.config.js
            │           ├── ueditor.parse.js
            │           └── ueditor.parse.min.js
            └── static
                ├── css
                │   └── page.css
                ├── img
                │   ├── bg-table-title.png
                │   ├── bg-tab-say.png
                │   ├── ico-black-disabled.png
                │   ├── ico-black-enabled.png
                │   ├── ico-coorption-person.png
                │   ├── ico-miss-person.png
                │   ├── ico-mr-person.png
                │   ├── ico-white-disabled.png
                │   └── ico-white-enabled.png
                └── scripts
                    ├── js
                    └── lib
                        └── jquery.min.js

21 directories, 48 files

QNetworkReply获取Content-Length的方法(PyQT)

使用pyqt webkit写headless 动态爬虫,需要捕获到所有的HTTP request和response(包括JS异步请求)。

QNetworkReply 没有返回content-length相关的方法,rawHeaderList实际上并不包括content-length。

一个可行的方法,是在readyRead信号触发时,size()方法获取到body的大小。示例代码如下,请注意self.reply_lst这个列表是必不可少的:

import sys
from PyQt5.Qt import QApplication
from PyQt5.QtCore import QUrl
from PyQt5.Qt import QWebView, QWebPage
from PyQt5.QtNetwork import QNetworkAccessManager


class Manager(QNetworkAccessManager):
    def __init__(self, parent=None):
        QNetworkAccessManager.__init__(self, parent)
        self.finished.connect(self._finished)
        self.reply_lst = []

    def _finished(self, reply):
        print '[%s bytes] %s' % (reply.content_length, reply.url().toString())
        self.reply_lst.remove(reply)

    def createRequest(self, operation, request, body=None):
        _url = request.url()
        reply = super(Manager, self).createRequest(operation, request, body)
        reply.readyRead.connect(self.read_read)
        self.reply_lst.append(reply)
        return reply

    def read_read(self):
        self.sender().content_length = self.sender().size()

def app_quit():
    app.quit()

if __name__ == "__main__":
    app = QApplication(['-platform', 'minimal'])
    browser = QWebView()
    page = QWebPage()
    manager = Manager()
    page.setNetworkAccessManager(manager)
    browser.setPage(page)
    browser.load(QUrl('https://www.lijiejie.com/'))
    browser.loadFinished.connect(app_quit)
    sys.exit(app.exec_())

最后,拿到所有URL的content-length如下,可以把它写到爬虫的结果中:

[499 bytes] https://www.lijiejie.com/
[2082 bytes] https://s6.cnzz.com/stat.php?id=3804994&web_id=3804994&show=pic
[11414 bytes] https://www.lijiejie.com/wp-includes/js/wp-emoji-release.min.js?ver=4.7.1
[20172 bytes] https://www.lijiejie.com/wp-content/plugins/crayon-syntax-highlighter/css/min/crayon.min.css?ver=_2.7.2_beta
[2850 bytes] https://www.lijiejie.com/wp-content/plugins/crayon-syntax-highlighter/themes/github/github.css?ver=_2.7.2_beta
[86 bytes] https://www.lijiejie.com/wp-content/plugins/crayon-syntax-highlighter/fonts/courier-new.css?ver=_2.7.2_beta
[374 bytes] https://www.lijiejie.com/wp-content/plugins/wp-pagenavi/pagenavi-css.css?ver=2.70
[27597 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/style.min.css?ver=0.4
[10056 bytes] https://www.lijiejie.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1
[3678 bytes] https://www.lijiejie.com/wp-content/plugins/dynamic-to-top/js/libs/jquery.easing.js?ver=1.3
[4214 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/library/js/drop-downs.min.js?ver=20110920
[22337 bytes] https://www.lijiejie.com/wp-content/plugins/crayon-syntax-highlighter/js/min/crayon.min.js?ver=_2.7.2_beta
[1192 bytes] https://www.lijiejie.com/wp-content/plugins/dynamic-to-top/js/dynamic.to.top.min.js?ver=3.5
[1398 bytes] https://www.lijiejie.com/wp-includes/js/wp-embed.min.js?ver=4.7.1
[6108 bytes] https://www.lijiejie.com/wp-includes/js/jquery/jquery.js?ver=1.12.4
[765 bytes] https://c.cnzz.com/core.php?web_id=3804994&show=pic&t=z
[3197 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/images/bg.jpg
[2236 bytes] https://www.lijiejie.com/wp-content/plugins/crayon-syntax-highlighter/css/images/toolbar/buttons.png
[576 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/images/quotes.png
[9766 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/images/header.png
[7210 bytes] https://www.lijiejie.com/wp-content/uploads/2014/10/stom.tencent.com_.png
[719 bytes] https://icon.cnzz.com/img/pic.gif
[43 bytes] https://hzs9.cnzz.com/stat.htm?id=3804994&r=&lg=zh-cn&ntime=none&cnzz_eid=1834513944-1484813777-&showp=1920x1080&t=李劼杰的博客&h=1&rnd=1792993998
[43 bytes] https://cnzz.mmstat.com/9.gif?abc=1&rnd=2124379566
[142 bytes] https://www.lijiejie.com/wp-content/themes/retro-fitted/images/bullet.png
[2516 bytes] https://www.lijiejie.com/wp-content/uploads/2017/01/eping.png
[3282 bytes] https://www.lijiejie.com/wp-content/uploads/2014/10/stom.tencent.com_upfile.png

eping — 为ping增加urlparse

做渗透测试或提交漏洞单时,常常用到ping,来确认内外网IP,或者寻找资产的归属。

然而ping并不支持url parse。我们写个简单的extended ping脚本。

vim /usr/bin/eping

#!/usr/bin/env python

# Extented Ping

import sys
import urlparse
import platform
import os


if __name__ == '__main__':
    if len(sys.argv) == 1:
        print '  Usage: python %s URL' % sys.argv[0]
        sys.exit(0)
    target = sys.argv[1].lower()
    if not target.startswith('http'):
        target = 'http://%s' % target
    host = urlparse.urlparse(target, 'http').hostname
    param_count = '-c4 ' if platform.system() != 'Windows' else ''
    os.system('ping %s%s' % (param_count, host))

chmod +x /usr/bin/eping

root@srvScanner-test-dev001:~# eping https://www.lijiejie.com/requests-exceptions-invalidschema-missing-dependencies-for-socks-support/
PING www.lijiejie.com (47.88.12.133) 56(84) bytes of data.
64 bytes from 47.88.12.133: icmp_seq=1 ttl=52 time=138 ms
64 bytes from 47.88.12.133: icmp_seq=2 ttl=52 time=138 ms
64 bytes from 47.88.12.133: icmp_seq=3 ttl=52 time=138 ms
64 bytes from 47.88.12.133: icmp_seq=4 ttl=52 time=138 ms

--- www.lijiejie.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 138.204/138.387/138.683/0.317 ms

Windows下使用pyinstaller生成一个exe文件,copy到C:\Windows\System32

pyinstaller –onefile eping.py

Windows 64 bit executable  https://www.lijiejie.com/wp-content/uploads/2017/01/eping.zip 

requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support

requests (2.12.4) 在Windows下使用socks5代理时,出现报错:

File "D:\Python27\lib\site-packages\requests-2.12.4-py2.7.egg\requests\adapters.py", line 43, in SOCKSProxyManager
 raise InvalidSchema("Missing dependencies for SOCKS support.")
requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support.

debug 发现是找不到模块 ‘requests.packages.urllib3.contrib.socks’,但其实socks.py文件是存在的。

解决的办法是安装 win_inet_pton

pip install win-inet-pton

之后, 在windows下再配置使用socks5代理就正常了。