Bug #2187

ST's http client should replace spaces in urls to %20

Added by Leonid Protasov almost 7 years ago. Updated almost 7 years ago.

Status:FixedStart date:05/28/2014
Priority:ImmediateDue date:
Assignee:Andreas Smas% Done:

100%

Category:API
Target version:4.8
Found in version:All Platform:Linux

Description

Can be reproducible in tree.tv plugin on trailers.

Associated revisions

Revision 0fd91a84
Added by Andreas Smas almost 7 years ago

httpclient: Escape inproperly encoded redirect URLs

Fixes #2187

Change included in version 4.7.41

Revision b8ae9be0
Added by Andreas Smas almost 7 years ago

httpclient: Rework Location header escape handling

Fixes #2187

Change included in version 4.7.43

History

#1 Updated by Leonid Protasov almost 7 years ago

  • Subject changed from ST can't open/play url in item videoparams if it contains spaces to http client fails on some redirections

http client should escape url part of the 'Location:' it gets from HTTP/1.1 302 Moved Temporarily.

navigator [INFO]: Opening http://balancer.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4
HTTP [DEBUG]: Connected to balancer.3tv.im:80 (id=39)
HTTP [DEBUG]: > GET /play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4 HTTP/1.1
HTTP [DEBUG]: > Range: bytes=0-4095
HTTP [DEBUG]: > User-Agent: Showtime Linux/x86_64 4.7.36.g71516.dirty
HTTP [DEBUG]: > Connection: keep-alive
HTTP [DEBUG]: > Accept: */*
HTTP [DEBUG]: > Accept-Encoding: identity
HTTP [DEBUG]: > Host: balancer.3tv.im
HTTP [DEBUG]: http://balancer.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4: Response:
HTTP [DEBUG]: < HTTP/1.1 302 Moved Temporarily
HTTP [DEBUG]: < Server: nginx
HTTP [DEBUG]: < Date: Thu, 29 May 2014 06:49:28 GMT
HTTP [DEBUG]: < Content-Type: text/html
HTTP [DEBUG]: < Content-Length: 154
HTTP [DEBUG]: < Location: http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking Bad Season 2 Promo.mp4
HTTP [DEBUG]: < 
HTTP [DEBUG]: http://balancer.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4: Following redirect to http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking Bad Season 2 Promo.mp4
HTTP [DEBUG]: Parking connection to balancer.3tv.im:80 (id=39)
HTTP [DEBUG]: Connected to st1.3tv.im:80 (id=40)
HTTP [DEBUG]: > GET /play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking+Bad+Season+2+Promo.mp4 HTTP/1.1
HTTP [DEBUG]: > Range: bytes=0-4095
HTTP [DEBUG]: > User-Agent: Showtime Linux/x86_64 4.7.36.g71516.dirty
HTTP [DEBUG]: > Connection: keep-alive
HTTP [DEBUG]: > Accept: */*
HTTP [DEBUG]: > Accept-Encoding: identity
HTTP [DEBUG]: > Host: st1.3tv.im
HTTP [DEBUG]: http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking Bad Season 2 Promo.mp4: Response:
HTTP [DEBUG]: < HTTP/1.1 403 Forbidden
HTTP [DEBUG]: < Server: nginx/1.6.0
HTTP [DEBUG]: < Date: Thu, 29 May 2014 06:52:30 GMT
HTTP [DEBUG]: < Content-Type: text/html
HTTP [DEBUG]: < Content-Length: 168
HTTP [DEBUG]: < 
HTTP [DEBUG]: Disconnected from st1.3tv.im:80 (id=40) Request destroyed

If you escape "http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking Bad Season 2 Promo.mp4" to "http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4" it plays those links ok...

Reproducible on trailers in tree.tv plugin.

#2 Updated by Leonid Protasov almost 7 years ago

What I see chrome browser does if you supply the links that contain spaces:
(http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking Bad Season 2 Promo.mp4).replace(/\s/g, '%20') ->

http://st1.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4

I met that problems when supplying links to icons etc. So to fix that globally - ST's http client should always check any links for spaces and replace them to %20...

#3 Updated by Leonid Protasov almost 7 years ago

  • Subject changed from http client fails on some redirections to ST's http client should replace spaces in urls to %20

#4 Updated by Leonid Protasov almost 7 years ago

Found good answer here: http://stackoverflow.com/questions/497908/are-urls-allowed-to-have-a-space-in-them

As per RFC 1738:

Unsafe:

Characters can be unsafe for a number of reasons. The space character is unsafe because significant spaces may disappear and insignificant spaces may be introduced when URLs are transcribed or typeset or subjected to the treatment of word-processing programs. The characters "<" and ">" are unsafe because they are used as the delimiters around URLs in free text; the quote mark (""") is used to delimit URLs in some systems. The character "#" is unsafe and should always be encoded because it is used in World Wide Web and in other systems to delimit a URL from a fragment/anchor identifier that might follow it. The character "%" is unsafe because it is used for encodings of other characters. Other characters are unsafe because gateways and other transport agents are known to sometimes modify such characters. These characters are "{", "}", "|", "\", "^", "~", "[", "]", and "`".

All unsafe characters must always be encoded within a URL. For example, the character "#" must be encoded within URLs even in systems that do not normally deal with fragment or anchor identifiers, so that if the URL is copied into another system that does use them, it will not be necessary to change the URL encoding.

#5 Updated by Andreas Smas almost 7 years ago

This is what I see:

HTTP [DEBUG]: Connected to balancer.3tv.im:80 (id=4)
HTTP [DEBUG]: > GET /play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4 HTTP/1.1
HTTP [DEBUG]: > Range: bytes=0-4095
HTTP [DEBUG]: > User-Agent: Showtime Linux/x86_64 4.7.37.ge783d.dirty
HTTP [DEBUG]: > Connection: keep-alive
HTTP [DEBUG]: > Accept: */*
HTTP [DEBUG]: > Accept-Encoding: identity
HTTP [DEBUG]: > Host: balancer.3tv.im
HTTP [DEBUG]: http://balancer.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4: Response:
HTTP [DEBUG]: < HTTP/1.1 302 Moved Temporarily
HTTP [DEBUG]: < Server: nginx
HTTP [DEBUG]: < Date: Thu, 29 May 2014 08:05:28 GMT
HTTP [DEBUG]: < Content-Type: text/html
HTTP [DEBUG]: < Content-Length: 154
HTTP [DEBUG]: < Connection: keep-alive
HTTP [DEBUG]: < Location: http://stream-ru.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4
HTTP [DEBUG]: < 
HTTP [DEBUG]: http://balancer.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ%2C1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4: Following redirect to http://stream-ru.3tv.im/play/1/ROFXNzsFQvHE-iK7I5IRIQ,1401389365/files/15/3347/treilerBreaking%20Bad%20Season%202%20Promo.mp4

So it seems correctly escaped.

I could implement an escaper but it's hard to know if it works correctly unless I can test it against something that's known to be broken.

#6 Updated by Andreas Smas almost 7 years ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

Also available in: Atom PDF