In one of my flash applications I take in some text (HTML) and do some processing (convert to plaintext). Since this was written in AS2, which doesn’t have regular expressions or a String.replace() function we came up with a simple alternative. split().join(). This worked exactly as we wanted, replacing the text passed to split() with the text passed to join(). Part of this conversion was to compress 3 or more newlines into only 2 newlines. This would be easy with a regular expression but with only split().join() I had to run this through a while loop in order to compress 3 or more newlines into only 2 newlines. This was the code:
while (str.indexOf("\n\n\n") != -1) {
str = str.split("\n\n\n").join("\n\n");
}A very simple loop that, if it works correctly, will compress any number of 3 or more consecutive newlines into only 2 newlines. Needless to say it did work correctly in our AS2 application.
In January of this year I started converting our AS2 application to AS3 and Flex. Since we still had need of this conversion and the code should have worked fine in AS3 I copied it over and all seemed well. However, after testing for a while we started to get hangs of the application at a certain point. One of our developers traced it to an infinite loop….the while loop above. He then debugged into the loop, found that the array returned by split() had an empty string in it, and added some code to remove an empty string, should one be found.
When I saw the commit I was baffled. This looked like added code to solve a problem that didn’t exist. After all, this code had worked fine in AS2 and it was self-evident that the number of newlines would always reduce, as 3 newlines were always replaced by 2……or were they?
After a back and forth with the developer in question I had a small reproduction script which would loop infinitely, despite the fact that it was replacing 3 newlines with 2. I decided to write my own simple test case and check the output in AS2 and AS3. To simplify things I used simple characters instead of newlines:
trace("abbbbc".split("bbb").join("bb"));Run in AS2 I get “abbbc”, a reduction of 1 ‘b’ from the string, as expected. Run in AS3, however I get back the exact same string: “abbbbc”. Curious, I tried the following:
trace("abbbc".split("bb").join("b"));This produced the same result in both AS2 and AS3, “abbc”, a reduction of 1 ‘b’, as expected. I then tried the next case up:
trace("abbbbbc".split("bbbb").join("bbb"));This code returned “abbbbc” in AS2, again as expected, but in AS3 returned “abbbbbbc”, an increase of 1 ‘b’!
Well, I had obviously found a regression in AS3 vs AS2. I then tried regular expressions to see if the same thing happened with them:
trace("abbbbc".split(/bbb/).join("bb"));This, (in AS3 of course, AS2 doesn’t have regular expressions) produced the desired output of “abbbc”. In my searching I also noticed that String has a replace() method in AS3 which will do global replacements if the regex has a ‘g’ modifier. Now that I had found a solution I switched all of our split().join() mess to use replace() with a global regex. Furthermore I replaced the while loop above with a single replace call with a regular expression, as I had wanted to do when I first wrote it, hence:
str = str.replace(/\n\n\n+/, "\n\n");
I also tried replace to see if ir exhibited the same problem as split().join():
trace("abbbbc".replace("bbb", "bb"));AS3 reported “abbbc”, the wanted output, so the problem is specifically in split() in AS3 and only when used with a string and not a regular expression. Of course this is somewhat obvious as replace() only replaces the first occurance of a string unless a regex with ‘g’ is used.
Further testing revealed an interesting anomaly if my replacement text used a different character than the text I was searching for:
trace("abbbbc".split("bbb").join("zz"));
trace("abbbc".split("bb").join("z"));
trace("abbbbbc".split("bbbb").join("zzz"));In AS2 this outputs:
azzbc
azbc
azzzbc
leaving the last ‘b’ untouched, as we would expect; but in AS3:
azzzzc
azzc
azzzzzzc
….where did all of the ‘b’s go?
The more algorithmic among you (wait, is anyone actually reading this?) will have figured it out by now but a look at the output of split() makes this much more obvious. If you run:
trace("abbbbc".split("bbb"));
trace("abbbc".split("bb"));
trace("abbbbbc".split("bbbb"));you get this in AS2:
a,bc
a,bc
a,bc
and this in AS3:
a,,c
a,,c
a,,c
The arrays are the same for all 3 runs but the interesting part is that in AS3 all traces of ‘b’ disappear and we get an empty string in between the “bookend” ‘a’ and ‘b’. From this output we can deduce that AS3 is finding 2 copies of the search string as the resulting array is 3 elements long; vs. AS2’s 2 element array showing that it is finding 1 match. Logically, and from a brain teaser perspective, there are actually 2 “matches” for “bbb” in the string “abbbbc”. There’s the first 3 ‘b’s and the last 3 ‘b’s. This isn’t how we were expecting split() to function, though, as the semantics of split() are supposed to be such that if you do a split() followed by a join() with the same string you should get the same string back again. In the case of AS3 “abbbc”.split(”bb”).join(”bb”) will actually increase the length of the string by one ‘b’.
Another simple test shows that the same thing happens with a multicharacter repeating pattern:
trace("a-_-_-_-_d".split("-_-_-_").join("-_-_"));A little cogitation reveals how this could happen given a simple splitting algorithm. Here’s some AS code which exhibits the same problem as AS3’s String.split():
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| function brokenSplit(str : String, needle : String) : Array {
var output : Array = new Array();
var lastFound : Number = 0;
for (var i : Number = 0; i < str.length; ++i) {
if (str.substr(i, needle.length) == needle) {
output.push(str.substr(lastFound, i - lastFound));
lastFound = i + needle.length;
}
}
if (lastFound < i) {
output.push(str.substr(lastFound, i));
}
return output;
} |
and here is the same code with only 3 characters added which outputs the same as AS2’s String.split():
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| function fixedSplit(str : String, needle : String) : Array {
var output : Array = new Array();
var lastFound : Number = 0;
for (var i : Number = 0; i < str.length; ++i) {
if (str.substr(i, needle.length) == needle) {
output.push(str.substr(lastFound, i - lastFound));
lastFound = (i += needle.length);
}
}
if (lastFound < i) {
output.push(str.substr(lastFound, i));
}
return output;
} |
The only difference between the two is that fixedSplit() increments the iterator (i) by the length of the needle when a match is found. brokenSplit() naively keeps looking for a match on the very next character even though it is inside a match.
Here is a swf running the various test cases in AS2:
Here is a swf running the various test cases in AS3:
I’ve tested in Flash versions: 9,0,28,0 (debug player for Flex Builder 2), 9,0,47,0 (current release player as of this post), 9,0,60,235 (debug player for Flex Builder 3 beta 2), and 9,0,98,0 (MovieStar). The same output displays in all 3 versions.
I have opened a bug report in Adobe’s bug system: ASC-2936.
UPDATE: fixed the swf links so you should be able to see the tests working again.
UPDATE: My bug was marked a duplicate of ASC-1739, which is slated to be fixed in Flash Player 9.1. I’m glad that others found this issue too.
