Page Index Toggle Pages: [1] 2  ReplyAdd Poll Send Topic
Hot Topic (More than 10 Replies) ; gets url encoded with some clients, + doesn't (Read 10527 times)
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
; gets url encoded with some clients, + doesn't
Sep 17th, 2014 at 4:51pm
Mark & QuoteQuote  
In the error log I see messages that, in Security.pm, YaBB can't parse thread id, which is obviously url encoded as it contains %3b, which is semicolon.

Reverse happens with attachment names that contain '+', which doesn't get url encoded in the client, even though it should and instead gets converted as " " (space).

Dunno if this is bug or security feature, so posting here...
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #1 - Sep 17th, 2014 at 5:39pm
Mark & QuoteQuote  
I'm going to call it a security feature (at least that attachment one as I'm pretty sure - not 100% sure -  YaBB doesn't allow '+' in uploaded file names).

Also, the only time I see %3b in the error log here is when someone has inserted a url into a query string.

Of course, we have no way of knowing what client was being used to do this but if it were a major issue or one that breaks the url, someone would have complained to YaBBForum about their users not being able to read messages.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #2 - Sep 17th, 2014 at 5:43pm
Mark & QuoteQuote  
The last time I saw %3b in a URL was when it was followed by "start=all"... I'm pretty sure YaBB used to allow "+" in uploaded file names in previous versions.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #3 - Sep 17th, 2014 at 6:15pm
Mark & QuoteQuote  
'start=all' should only appear in conjunction with 'board='. So a 'num=' in conjunction with a 'start=all' was entered from the address bar.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #4 - Sep 17th, 2014 at 8:53pm
Mark & QuoteQuote  
Dandello wrote on Sep 17th, 2014 at 6:15pm:
'start=all' should only appear in conjunction with 'board='. So a 'num=' in conjunction with a 'start=all' was entered from the address bar.


It really doesn't make sense as the same URL pattern comes from different IP subnets, but not all cause the error message, only if the "b" in the URL encoded query string is lowercase.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #5 - Sep 17th, 2014 at 9:38pm
Mark & QuoteQuote  
So you're saying YaBB sees '%3b' and tosses an error but doesn't on '%3B' - if it's in conjunction with 'start=all'? Or some clients are encoding the semi-colon to '%3b' and some aren't.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #6 - Sep 18th, 2014 at 3:33am
Mark & QuoteQuote  
Dandello wrote on Sep 17th, 2014 at 9:38pm:
So you're saying YaBB sees ';' and tosses an error but doesn't on ';' - if it's in conjunction with 'start=all'? Or some clients are encoding the semi-colon to ';' and some aren't.


As semicolon is reserved character, all clients should URL encode it... It seems some encoder implementations encode using lowercase letters, some encode using uppercase letters. The problem is that for some reason the lowercase versions don't always get decoded in Perl code... Where the "start=all" comes from, I suspect it has to do with old links that were "cached" or bookmarked about 7 years ago when the forum was using older version of YaBB.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #7 - Sep 18th, 2014 at 5:37am
Mark & QuoteQuote  
Well, I see that my browser decodes them the same...  Grin
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #8 - Sep 18th, 2014 at 10:40am
Mark & QuoteQuote  
Dandello wrote on Sep 18th, 2014 at 5:37am:
Well, I see that my browser decodes them the same...  Grin


My browser had no problems with decoding, but it seems Apache has troubles accepting ";" in URL, even if it is URL encoded... It just says "=" can't be there without "&"... So both ";" and "=" get URL encoded inside Apache even though ";" is already URL encoded by the browser.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #9 - Sep 18th, 2014 at 2:48pm
Mark & QuoteQuote  
Monni wrote on Sep 18th, 2014 at 10:40am:
but it seems Apache has troubles accepting ";" in URL, even if it is URL encoded


But it's not consistent. That's what's irksome. So I don't think it's an Apache issue, I think it's a browser issue as it's encoding (or not decoding) something that shouldn't be visibly encoded at all. My suspicion is there was/is a browser or harvester that mis-encoded query strings and that bad code is still in search engines and/or bookmarks.

I just tested the
Code
Select All
%3bstart=all 

on YaBBForum and it went through just fine.

Oh, BTW from Wikipedia (http://en.wikipedia.org/wiki/Query_string)
Quote:
SPACE is encoded as '+' or "%20"

So if YaBB accepted attachments with '+' in them in the past it was a bug then. I doubt there's an 'in YaBB' fix for it.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #10 - Sep 18th, 2014 at 3:06pm
Mark & QuoteQuote  
Dandello wrote on Sep 18th, 2014 at 2:48pm:
Monni wrote on Sep 18th, 2014 at 10:40am:
but it seems Apache has troubles accepting ";" in URL, even if it is URL encoded


But it's not consistent. That's what's irksome. So I don't think it's an Apache issue, I think it's a browser issue as it's encoding (or not decoding) something that shouldn't be visibly encoded at all. My suspicion is there was/is a browser or harvester that mis-encoded query strings and that bad code is still in search engines and/or bookmarks.

I just tested the
Code
Select All
;start=all 

on YaBBForum and it went through just fine.

Oh, BTW from Wikipedia (http://en.wikipedia.org/wiki/Query_string)
Quote:
SPACE is encoded as '+' or " "

So if YaBB accepted attachments with '+' in them in the past it was a bug then. I doubt there's an 'in YaBB' fix for it.


1. I can see from Apache access log that the incoming URL is correct, but instead of code 200, it throws 301, which restarts the query string parsing... That's where it goes wrong. I tried to force NoEscape in Apache directory-level config, but don't know if it is the 100% working solution...

2. If I try to access attachments containing "+" with Chrome, I can see the attachment, but with clients that use strict standards following URL encoding, it doesn't work.
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #11 - Sep 18th, 2014 at 5:53pm
Mark & QuoteQuote  
Occasionally 'invisible' characters slip in that can't be seen in text editors or logs but Apache and Perl see as part of the encoding that REALLY mess things up.

A 301 is a redirect, so - since it's throwing a YaBB error eventually - the parsing error is happening after the redirect.

When you see these errors, how old is the message they're trying to get to? I'm betting they're pretty old ones.

And what's the exact error message YaBB is giving?
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #12 - Sep 18th, 2014 at 7:18pm
Mark & QuoteQuote  
Dandello wrote on Sep 18th, 2014 at 5:53pm:
Occasionally 'invisible' characters slip in that can't be seen in text editors or logs but Apache and Perl see as part of the encoding that REALLY mess things up.

A 301 is a redirect, so - since it's throwing a YaBB error eventually - the parsing error is happening after the redirect.

When you see these errors, how old is the message they're trying to get to? I'm betting they're pretty old ones.

And what's the exact error message YaBB is giving?


April 9, 2009

$error_txt{'only_numbers_allowed'}
Thread ID: '1239226869%3bstart=all'
  
Back to top
IP Logged
 
Paste Member Name in Quick Reply Box Dandello
Forum Administrator
YaBB Modder
*****
Offline


I love YaBB 2.7!

Posts: 2234
Location: The Land of YaBB
Joined: Feb 12th, 2014
Gender: Female
Mood: Annoyed
Zodiac sign: Virgo
Re: ; gets url encoded with some clients, + doesn't
Reply #13 - Sep 18th, 2014 at 8:01pm
Mark & QuoteQuote  
That's not from YaBB's error log, is it? Because if it is there's something more seriously wrong with that error string that a mis-encoded semicolon.

And if all those errors are as old as the date indicates, I personally wouldn't worry about it.
  

Perfection is not possible. Excellence, however, is excellent.
Back to top
WWW  
IP Logged
 
Paste Member Name in Quick Reply Box Monni
Language
***
Offline


Min izāmō

Posts: 413
Location: Kaarina, Finland
Joined: Jul 16th, 2014
Gender: Male
Mood: Frustrated
Zodiac sign: Pisces
Re: ; gets url encoded with some clients, + doesn't
Reply #14 - Sep 19th, 2014 at 3:41am
Mark & QuoteQuote  
Dandello wrote on Sep 18th, 2014 at 8:01pm:
That's not from YaBB's error log, is it? Because if it is there's something more seriously wrong with that error string that a mis-encoded semicolon.

And if all those errors are as old as the date indicates, I personally wouldn't worry about it.


Eh... in English that error string would be "This field only accepts numbers from 0-9"... But that you should know already... Didn't make sense to post the Finnish version of the string as that would be what reads in error log.

Anyways... if the hack I made in .htaccess works for most non-malicious users, I'm not too worried about the cases where it doesn't work for malicious users Wink
  
Back to top
IP Logged
 
Page Index Toggle Pages: [1] 2 
ReplyAdd Poll Send Topic
Bookmarks: del.icio.us Digg Facebook Google LinkedIn reddit Twitter Yahoo
; gets url encoded with some clients, + doesn't

Please type the characters exactly as they appear in the image,
without the first 2 and last 2 characters.
The characters must be typed in the same order,
and they are case-sensitive.
Open Preview Preview

You can resize the textbox by dragging the right or bottom border.
Off Topic Comment Insert Spoiler
Insert Hyperlink Insert FTP Link Insert Image Insert E-mail Insert Media Insert Table Insert Table Row Insert Table Column Insert Horizontal Rule Insert Teletype Insert Code Insert Quote Edited Superscript Subscript Insert List /me - my name Insert Marquee Insert Timestamp No Parse
Bold Italicized Underline Insert Strikethrough Highlight
                       
Change Text Color
Insert Preformatted Text Left Align Centered Right Align
resize_wb
resize_hb







Max 5000 characters. Remaining characters:
Text size: %
More Smilies
View All Smilies
Collapse additional features Collapse/Expand additional features Smiley Wink Cheesy Grin Angry Sad Shocked Cool Huh Roll Eyes Tongue Embarrassed Lips Sealed Undecided Kiss Cry