Bug #2771
503 response in showtime.httpReq(url) causes error
Status: | Fixed | Start date: | 08/07/2015 | |
---|---|---|---|---|
Priority: | Normal | Due date: | ||
Assignee: | % Done: | 100% | ||
Category: | API | |||
Target version: | - | |||
Found in version: | 4.9.524.g52f7d | Platform: | Linux |
Description
In movian when showtime.httpReq(url) gets 503 response it causes the plugin to exit.
Dump:
kisscartoon [ERROR]: Error (HTTP request failed http://kisscartoon.me/CartoonList/LatestUpdate -- HTTP error: 503) at /project/repo/checkout/src/ecmascript/es_io.c:412 kisscartoon [ERROR]: STACK DUMP: Error: HTTP request failed http://kisscartoon.me/CartoonList/LatestUpdate -- HTTP error: 503 /project/repo/checkout/src/ecmascript/es_io.c:412 anon native strict preventsyield anon showtime/http:101 tailcalled browseItems file:///home/pisek/workspace/showtime-plugin-kisscartoon/kisscartoon/kisscartoon.js:47 anon file:///home/pisek/workspace/showtime-plugin-kisscartoon/kisscartoon/kisscartoon.js:60 preventsyield apply native strict preventsyield anon showtime/page:366 preventsyield
It should not work like that as there are many pages that respond with 503 (with custom error-page) together with Refresh header url which reloads content from the dynamically created, proper site (eg. CloudFlare).
Associated revisions
ecmascript: Add 'noFail' option
Fixes #2771
History
#1
Updated by Leonid Protasov almost 7 years ago
- Status changed from New to Fixed
- % Done changed from 0 to 100
Please use at least 4.10 and if the same is there - use following code:
var ok = false; try { httpReq here ok = true; } catch(err) { } if (!ok) ...
#2
Updated by Michal Pisarski over 6 years ago
The error is still there.
The response headers are not attached to any field in
err...
#3
Updated by Michal Pisarski over 6 years ago
The error is still there.
The response headers are not attached to any field in err...
#4
Updated by Michal Pisarski over 6 years ago
Object.getOwnPropertyNames(err) shows only message field which shows:
HTTP request failed http://192.168.0.11:9091/transmission/rpc -- HTTP error: 409
#5
Updated by Andreas Smas over 6 years ago
- Status changed from Fixed to Accepted
This could be improved.
#6
Updated by Andreas Smas over 6 years ago
- Status changed from Accepted to Fixed
Applied in changeset git|59277a6bd77b5da257f32ed504465dcf91e9414b.
#7
Updated by Andreas Smas over 6 years ago
Added noFail option
example:
var http =require('showtime/http'); var doc = http.request('https://movian.tv/foobarfoo', { noFail: true }); print("Response object:"); print('Status: ' + doc.statuscode); print(doc.toString()); for (var hdr in doc.headers) { print(hdr + ": " + doc.headers[hdr]); }
yields:
Response object: Status: 404 <!DOCTYPE html> <html> <head> <meta charset="utf-8" /> <title>Redmine 404 error</title> <style> body {font-family: Trebuchet MS, Georgia, "Times New Roman", serif; color: #303030; margin: 10px;} h1 {font-size:1.5em;} p {font-size:0.8em;} </style> </head> <body> <h1>Page not found</h1> <p>The page you were trying to access doesn't exist or has been removed.</p> <p><a href="javascript:history.back()">Back</a></p> </body> </html> CF-RAY: 2385bb0218b216a6-ARN X-Rack-Cache: miss Cache-Control: private X-Runtime: 0.002521 X-Request-Id: ad6d315c910b2c92be58cdd11c7efe1b Status: 404 Not Found Connection: keep-alive Transfer-Encoding: chunked Content-Type: text/html; charset=utf-8 Date: Tue, 20 Oct 2015 15:24:35 GMT Server: cloudflare-nginx
#8
Updated by Andreas Smas over 6 years ago
I also added the statuscode
property of the response object which might come in handy.
There's also a question if Movian should transparently handle Refresh: 0,URL -tag similar to a 3xx redirect?
#9
Updated by Michal Pisarski over 6 years ago
Great! Thanks!
#10
Updated by Michal Pisarski over 6 years ago
And about this:
<meta http-equiv="refresh" content="0;URL='http://dupa.com/'" />
IMHO yes, but there should be an option to forceNoRedirect or sth so that one can read this redirection page - there might be some important stuff there like session id or some important header etc.
#11
Updated by Buksa ... over 6 years ago
cf antibot
source https://github.com/Anorov/cloudflare-scrape/blob/master/cfscrape/__init__.py
@respond = http.request(BASE_URL, {
method: 'GET',
headers: {
'User-Agent': USER_AGENT
},
noFail: true
})
if (respond.statuscode === 503) {
match = /name="jschl_vc" value="(\w+)"/.exec(respond);
jschl_vc = match != null ? match[1] : "";
match = /name="pass" value="([^\n]+?)"/.exec(respond);
pass = match != null ? match[1] : "";
var match = /setTimeout\(function\(\)\{\s+(var t,r,a,f[^\n]+?\r?\n[\s\S]+?a\.value =[^\n]+?)\r?\n/.exec(respond);
result = match != null ? match[1] : "";
result = result.replace(/a\.value =(.+?) \+ .+?;/g, "$1");
result = result.replace(/\s{3,}[a-z](?: = |\.).+/g, "");
jschl_answer = (+(eval(result)) + +BASE_URL.split("/")[2].length)
setTimeout(function() {
respond = http.request('http://online.anidub.com/cdn-cgi/l/chk_jschl?jschl_vc=' + jschl_vc + '&pass=' + pass + '&jschl_answer=' + jschl_answer, {
//method: 'GET',
headers: {
'Referer': BASE_URL,
'User-Agent': USER_AGENT
},
noFail: true
})
}, 4000);
}
print(respond)@