09月27, 2020

Fiddler创建自定义规则

通过在Fiddler中创建custom rule, 可以添加自定义命令,还可以在HTTP请求时增加拦截方法,保存response数据。

Fiddler的自定义规则采用JS DotNET语言,也就是支持基本的JS语法和对象,比如字符串,正串表达式, encodeURICompoenent等方法,但是也与DotNET对象兼容,访问IO和JSON序列化时必须使用DOTNET对象;同时,Fiddler还提供了一些内置的对象,可以与UI交互

技巧一:JSON序列化

Fiddler.WebFormats.JSON.JsonEncode(responseJSON.JSONObject['wareInfo'])

技巧二:保存文件

if (!Directory.Exists(path))
                {
                    // Try to create the directory.
                    Directory.CreateDirectory(path);
                }
                fso = new ActiveXObject("Scripting.FileSystemObject");
                file = fso.OpenTextFile(path+fileName,8 ,true, true);
                file.writeLine(jsonString);
                file.writeLine("\n");
                file.close();

下面是我的脚本


    static function OnBeforeResponse(oSession: Session) {
        if (m_Hide304s && oSession.responseCode == 304) {
            oSession["ui-hide"] = "true";
        }
        /* data crawler */
        // JD.com
        if  (oSession.HostnameIs( "api.m.jd.com" ) && oSession.uriContains( "/client.action?functionId=")){

            var input = oSession.url.replace("api.m.jd.com/client.action?functionId=","")

            var pattern = "(.*)&.*"
            var functionId= input.Split("&")[0]
            FiddlerObject.StatusText=functionId;
            // 京东好货
            if (functionId=="discZdmGoodsList" || functionId=="discSearchExecute"){  
                var  destFolder =  "d:/dataset/jdhw/" ;  
                // parse query string "st"
                var s_qs = (oSession.url + "?").split("?")[1];
                var querystring = HttpUtility.ParseQueryString(s_qs);
                var st = querystring.Get("st");

                if ((oSession.responseCode == 200) && 
                    oSession.oResponse.headers.ExistsAndContains("Content-Type", "application/json;charset=UTF-8")) 
                { 
                    oSession.SaveResponseBody(destFolder + st);
                }
            }
            // 商品搜索
            if (functionId=="search"){  
                var  destFolder =  "d:/dataset/jdsearch/" ;  
                var rbodyText=decodeURIComponent(oSession.GetRequestBodyAsString())
                rbodyText=rbodyText.slice(5,rbodyText.length).replace("}&","}")
                FiddlerObject.log(rbodyText)
                var rqbs = Fiddler.WebFormats.JSON.JsonDecode(rbodyText)
                var keyword = rqbs.JSONObject['keyword']
                var pvid = rqbs.JSONObject['pvid']
                var page = rqbs.JSONObject['page']
                // 显示keyword
                FiddlerObject.StatusText= keyword

                //准备输出数据
                var jsonString = oSession.GetResponseBodyAsString();
                var responseJSON = Fiddler.WebFormats.JSON.JsonDecode(jsonString);
                var jsonString=Fiddler.WebFormats.JSON.JsonEncode(responseJSON.JSONObject['wareInfo'])
                FiddlerObject.log(jsonString)
                // 保存文件到本地
                var fso;
                var file;
                var path=destFolder+keyword+"/"
                var fileName = pvid+"-"+page
                if (!Directory.Exists(path))
                {
                    // Try to create the directory.
                    Directory.CreateDirectory(path);
                }
                fso = new ActiveXObject("Scripting.FileSystemObject");
                file = fso.OpenTextFile(path+fileName,8 ,true, true);
                file.writeLine(jsonString);
                file.writeLine("\n");
                file.close();
            }
            if (functionId=="search" || functionId=="wareBusiness"){  
                var  destFolder =  "d:/dataset/jdsku/" ;  

            }
        }
    }

参考资料

本文链接:http://57km.cc/post/create custom rules in fiddler.html

-- EOF --

Comments